Eye tracking device, near-eye display device, and eye tracking method

Abstract

The application relates to the technical field of near-eye display, and provides an eye movement tracking device, a near-eye display device and an eye movement tracking method. The eye movement tracking device comprises a waveguide substrate, a first in-coupling area, a first out-coupling area, at least one second in-coupling area and at least one second out-coupling area arranged on the waveguide substrate; the second in-coupling area is arranged close to the first in-coupling area relative to the first out-coupling area; the second out-coupling area is arranged close to the first out-coupling area relative to the first in-coupling area; the first in-coupling area is used for coupling incident light emitted by a projection module into the waveguide substrate; the first out-coupling area is used for coupling the incident light propagating in the waveguide substrate out of the waveguide substrate to a human eye; the second in-coupling area is used for receiving infrared light with different incident angles and coupling the infrared light into the waveguide substrate, so that the infrared light propagates in the waveguide substrate to the corresponding one or more second out-coupling areas and is coupled out of the waveguide substrate to the human eye, thereby improving the eye movement tracking convenience and the eye movement tracking accuracy of the human eye when the eyeball of the user is in different positions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of near-eye display, and particularly relates to an eye movement tracking device, a near-eye display device and an eye movement tracking method. BACKGROUND

[0002] In the related art, when eye movement tracking is performed on a user by using infrared light provided by an infrared light source, the infrared light provided by the infrared light source can form a corresponding light spot in the human eye, and an imaging device can capture the human eye to obtain a corresponding image for eye movement tracking of the human eye.

[0003] However, due to the limitations of the position of the infrared light source, the position of the human eye and the position of the imaging device, the light spot corresponding to the infrared light in the human eye image obtained by the imaging device capturing the human eye is likely to not be on the cornea of the human eye but on the sclera of the human eye. For example, as shown in FIG. 1, when the infrared light is normally incident on the cornea of the human eye, the cornea can reflect the infrared light to the imaging device, so that the image captured by the imaging device can contain a clear light spot on the cornea. The clear light spot on the cornea can be used for eye movement tracking of the human eye. As shown in FIG. 2, when the human eye is turned to the inner side of the eye socket, the cornea of the human eye is correspondingly turned to the inner side of the eye socket. Since the position of the infrared light source does not change, the infrared light provided by the infrared light source to the human eye is irradiated onto the sclera of the human eye. The sclera can only diffuse reflect the infrared light, so that the image obtained by the imaging device capturing the human eye can contain an unclear speckle on the sclera. The unclear speckle on the sclera cannot be used for eye movement tracking of the human eye. Figure 1 a Figure 1 b As shown in FIG. 1, when the infrared light is normally incident on the cornea of the human eye, the cornea can reflect the infrared light to the imaging device, so that the image captured by the imaging device can contain a clear light spot on the cornea. The clear light spot on the cornea can be used for eye movement tracking of the human eye. As shown in FIG. 2, when the human eye is turned to the inner side of the eye socket, the cornea of the human eye is correspondingly turned to the inner side of the eye socket. Since the position of the infrared light source does not change, the infrared light provided by the infrared light source to the human eye is irradiated onto the sclera of the human eye. The sclera can only diffuse reflect the infrared light, so that the image obtained by the imaging device capturing the human eye can contain an unclear speckle on the sclera. The unclear speckle on the sclera cannot be used for eye movement tracking of the human eye.

[0004] Based on this, under the limitations of the position of the infrared light source, the position of the human eye and the position of the imaging device, if the clear light spot on the cornea is not contained in the human eye image captured by the imaging device, the accuracy of eye movement tracking of the human eye is likely to be adversely affected. SUMMARY

[0005] The main purpose of the present application is to provide an eye movement tracking device, a near-eye display device and an eye movement tracking method, which aims to solve the technical problem that the clear light spot on the cornea of the human eye is not contained in the human eye image captured by the imaging device due to the limitations of the position of the infrared light source, the position of the human eye and the position of the imaging device, and thus the accuracy of eye movement tracking of the human eye is poor.

[0006] In a first aspect, the present application provides an eye movement tracking device, comprising:

[0007] a waveguide substrate;

[0008] ​a first in-coupling region, a first out-coupling region, at least one second in-coupling region and at least one second out-coupling region are arranged on the waveguide substrate; the second in-coupling region is arranged close to the first in-coupling region relative to the first out-coupling region; the second out-coupling region is arranged close to the first out-coupling region relative to the first in-coupling region;

[0009] The first in-coupling region is configured to couple the incident light emitted by the projection module into the waveguide substrate, and the first out-coupling region is configured to couple the incident light propagating in the waveguide substrate out of the waveguide substrate to the human eye. The second in-coupling region facilitates receiving infrared light of different incident angles and coupling the infrared light into the waveguide substrate, propagating in the waveguide substrate to the corresponding one or more second out-coupling regions, and coupling out of the waveguide substrate to the human eye through the second out-coupling region.

[0010] In a second aspect, the present application further provides a near-eye display device, comprising:

[0011] An eye movement tracking device, wherein the eye movement tracking device is the aforementioned eye movement tracking device.

[0012] An optical-mechanical device, wherein the optical-mechanical device comprises a projection module and an infrared module. The projection module is configured to emit incident light to a first in-coupling region of the eye movement tracking device, and the first in-coupling region is configured to couple the incident light into a waveguide substrate of the eye movement tracking device. A first out-coupling region of the eye movement tracking device is configured to couple the incident light propagating in the waveguide substrate out of the waveguide substrate to the human eye. The infrared module comprises an infrared light source and an angle adjusting member. The angle adjusting member is configured to adjust the light angle of the infrared light provided by the infrared light source, so that the adjusted infrared light irradiates one of the second in-coupling regions of the eye movement tracking device, propagates to the corresponding one or more second out-coupling regions of the eye movement tracking device after being coupled into the waveguide substrate through the second in-coupling region, and is coupled out of the waveguide substrate to the human eye through the second out-coupling region.

[0013] An imaging device, wherein the imaging device is configured to capture the image of the human eye of the user to realize eye movement tracking.

[0014] In a third aspect, the present application further provides an eye movement tracking method of a near-eye display device, wherein the eye movement tracking method is used for the aforementioned near-eye display device, and the eye movement tracking method comprises:

[0015] Controlling the infrared light source of the near-eye display device to provide infrared light;

[0016] Adjusting the light angle of the infrared light provided by the infrared light source through the angle adjusting member of the near-eye display device to provide the adjusted infrared light to the human eye of the user;

[0017] Capturing the image of the human eye of the user through the imaging device of the near-eye display device;

[0018] According to the human eye image, eye movement information of the user is determined.

[0019] The application provides an eye movement tracking device, a near-eye display device and an eye movement tracking method of the near-eye display device. The eye movement tracking device comprises a waveguide substrate, a first coupling-in area, a first coupling-out area, at least one second coupling-in area and at least one second coupling-out area arranged on the waveguide substrate. The second coupling-in area is arranged close to the first coupling-in area relative to the first coupling-out area. The second coupling-out area is arranged close to the first coupling-out area relative to the first coupling-in area. The first coupling-in area is used for coupling incident light emitted by a projection module into the waveguide substrate. The first coupling-out area is used for coupling the incident light propagating in the waveguide substrate out of the human eye. The second coupling-in area is used for receiving infrared light of different incident angles and coupling the infrared light into the waveguide substrate, propagating in the waveguide substrate to the corresponding one or more second coupling-out areas, and coupling out of the human eye through the second coupling-out area.

[0020] Based on the arrangement of the at least one second coupling-in area and the at least one second coupling-out area, when the human eye is in different positions, such as when the cornea of the human eye is not rotated to the medial side of the eye socket or when the cornea of the human eye is rotated to the medial side of the eye socket, the eye movement tracking device can couple the infrared light of the corresponding incident angle through different second coupling-in areas or the infrared light of the corresponding incident angle through the corresponding position of the same second coupling-in area, so that the infrared light coupled into the waveguide substrate can be coupled out of the human eye after propagating in the waveguide substrate to the corresponding one or more second coupling-out areas. The infrared light can be coupled out of the cornea of the human eye after being affected by the eye movement tracking device, so as to form a corresponding light spot on the cornea of the human eye. The light spot on the cornea of the human eye can be used for eye movement tracking of the human eye, which is beneficial to improving the convenience and accuracy of eye movement tracking of the human eye. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0022] Figures 1 a to 1 b is a schematic diagram of a human eye image involved in the related art;

[0023] Figure 2 is a structural schematic diagram of an eye movement tracking device provided by an embodiment of the application;

[0024] Figure 3 is Figure 2 is a wave vector schematic diagram of the eye movement tracking device provided by the application;

[0025] Figure 4is a structural schematic diagram of an eye tracking device involved in an embodiment of the present application;

[0026] Figure 5 is Figure 4 is a wave vector schematic diagram of the eye tracking device provided;

[0027] Figure 6 is a structural schematic diagram of an eye tracking device involved in another embodiment of the present application;

[0028] Figure 7 is a structural schematic diagram of a near-eye display device provided in an embodiment of the present application;

[0029] Figures 8a to 8c is an optical path schematic diagram of the near-eye display device involved in an embodiment of the present application;

[0030] Figure 9 is a structural schematic diagram of a near-eye display device involved in another embodiment of the present application;

[0031] Figures 10a to 10c is an optical path schematic diagram of the near-eye display device involved in another embodiment of the present application;

[0032] Figure 11 is a flow schematic diagram of an eye tracking method of a near-eye display device provided in an embodiment of the present application.

[0033] Legend: 10, near-eye display device; 100, eye tracking device; 110, waveguide substrate; 120, first in-coupling region; 130, first out-coupling region; 140, second in-coupling region; 150, second out-coupling region; 160, third in-coupling region; 170, third out-coupling region; 200, optical-mechanical device; 210, projection module; 220, infrared module; 221, infrared light source; 222, angle adjusting member; 2221, mirror; 2222, adjustable liquid crystal grating; 230, beam combining assembly; 300, imaging device. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0035] The flowcharts shown in the drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor do they necessarily be executed in the described order. For example, some operations / steps can be further decomposed, combined or partially merged, so that the actual execution order may be changed according to the actual situation.

[0036] Some embodiments of the present application will be described in detail with reference to the drawings. The following embodiments and features of the embodiments can be combined with each other in the case of no conflict.

[0037] Please refer to Figure 2 , Figure 2 is a structural schematic diagram of an eye tracking device 100 provided by an embodiment of the present application. The eye tracking device 100 can be arranged in a near-eye display device 10 to track the eye movement of a user wearing the near-eye display device 10. The near-eye display device 10 can include an augmented reality (AR) glasses, a virtual reality (VR) glasses, a mixed reality (MR) glasses, an AR helmet, a VR helmet, an MR helmet, etc., without limitation.

[0038] As Figure 2 shown, the eye tracking device 100 includes a waveguide substrate 110, a first in-coupling region 120, a first out-coupling region 130, at least one second in-coupling region 140, and at least one second out-coupling region 150 arranged on the waveguide substrate 110. The second in-coupling region 140 is arranged close to the first in-coupling region 120 with respect to the first out-coupling region 130. The second out-coupling region 150 is arranged close to the first out-coupling region 130 with respect to the first in-coupling region 120. The first in-coupling region 120 is configured to couple the incident light emitted by a projection module 210 into the waveguide substrate 110. The first out-coupling region 130 is configured to couple the incident light propagating in the waveguide substrate 110 out to a human eye. The second in-coupling region 140 is configured to receive infrared light with different incident angles and couple the infrared light into the waveguide substrate 110, propagate the infrared light to a corresponding one or more second out-coupling regions 150 in the waveguide substrate 110, and couple the infrared light out to the human eye through the second out-coupling region 150.

[0039] For example, the waveguide substrate 110 can include a plurality of substrate surfaces. The first in-coupling region 120, the first out-coupling region 130, the second in-coupling region 140, and the second out-coupling region 150 can be arranged on one of the substrate surfaces of the waveguide substrate 110, without limitation.

[0040] In some embodiments, the incident light can include visible light. For example, the wavelength range of the incident light can include 390 nanometers to 770 nanometers. In the case that the incident light emitted by the projection module 210 irradiates the first in-coupling region 120, the first in-coupling region 120 can couple the incident light into the waveguide substrate 110. The incident light coupled into the waveguide substrate 110 can propagate at a preset angle until it is coupled out to the human eye by the first out-coupling region 130. The preset angle can include an angle greater than the critical angle of total internal reflection at the interface between the eye tracking device 100 medium and air. In the case that the eye tracking device 100 can couple the incident light by the first in-coupling region 120 and couple the incident light out to the human eye by the first out-coupling region 130, the eye tracking device 100 can have a display function. Based on this, the eye tracking device 100 can be used to subsequently prepare a near-eye display device 10 to achieve the display function of the near-eye display device 10. In an exemplary embodiment, in the case that ambient light irradiates the eye tracking device 100, the ambient light can be incident on the human eye through the first out-coupling region 130, and then the user can view the surrounding environment when wearing the near-eye display device 10 prepared by the eye tracking device 100, which is not limited herein.

[0041] For example, the number of the second in-coupling regions 140 can include one or more, and the number of the second out-coupling regions 150 can also include one or more.

[0042] In some embodiments, the second in-coupling region 140 can have a plurality of positions, and each of the positions can have a different range of incident angles. For example, the second in-coupling region 140 can have at least one of a ring shape and a semi-ring shape, and can be disposed around the first in-coupling region 120. When the infrared light is incident on the eye tracking device 100 and the incident angle of the infrared light is within the range of incident angles of one of the positions of the second in-coupling region 140, the infrared light can be incident on the corresponding position of the second in-coupling region 140. Since the ranges of incident angles of different positions of the same second in-coupling region 140 are different, the plurality of second in-coupling regions 140 can facilitate receiving infrared light with different incident angles. Accordingly, the corresponding position of the second in-coupling region 140 that receives the infrared light can couple the infrared light into the waveguide substrate 110. The infrared light coupled into the waveguide substrate 110 can propagate at a preset angle until it reaches the corresponding one or more second out-coupling regions 150 and is coupled out of the waveguide substrate 110 by the second out-coupling region 150 to the human eye. Accordingly, the second out-coupling region 150 can also have a plurality of positions. For example, the second out-coupling region 150 can also have at least one of a ring shape and a semi-ring shape, and can be disposed around the first out-coupling region 130. When the infrared light is incident on one of the positions of the second in-coupling region 140, the infrared light can be sequentially coupled out of the waveguide substrate 110 by the second in-coupling region 140, the waveguide substrate 110, and the corresponding position of the second out-coupling region 150 to the human eye. Of course, the present disclosure is not limited thereto, and any other suitable configuration is also possible.

[0043] In some embodiments, the second in-coupling region 140 can have a plurality of positions, and the second out-coupling region 150 can also have a plurality of positions. The second in-coupling region 140 can have a preset range of incident angles. When the infrared light is incident on the eye tracking device 100 and the incident angle of the infrared light is within the range of incident angles of the second in-coupling region 140, the infrared light can be incident on the second in-coupling region 140. Since the ranges of incident angles of different second in-coupling regions 140 are different, the plurality of second in-coupling regions 140 can facilitate receiving infrared light with different incident angles. Accordingly, the second in-coupling region 140 that receives the infrared light can couple the infrared light into the waveguide substrate 110. The infrared light coupled into the waveguide substrate 110 can propagate at a preset angle until it reaches the corresponding one or more second out-coupling regions 150 and is coupled out of the waveguide substrate 110 by the second out-coupling region 150 to the human eye. When the eye tracking device 100 can couple the infrared light by the second in-coupling region 140 and couple the infrared light out of the waveguide substrate 110 by the second out-coupling region 150 to the human eye, the human eye can reflect the infrared light, and the human eye can form a corresponding light spot. The light spot can be used for eye tracking of the user. Based on this, the eye tracking device 100 can be used to subsequently manufacture the near-eye display device 10 to implement the eye tracking function of the near-eye display device 10.

[0044] Of course, it is not limited to this, the second coupling-in area 140 and the second coupling-out area 150 can have a one-to-one correspondence, can have a one-to-many relationship, and can also have a many-to-one relationship, which is not limited here.

[0045] For example, the second coupling-in area 140 is distributed around the first coupling-in area 120, and the second coupling-in area 140 is arranged close to the first coupling-in area 120 relative to the first coupling-out area 130. The second coupling-out area 150 is distributed around the first coupling-out area 130, and the second coupling-out area 150 is arranged close to the first coupling-out area 130 relative to the first coupling-in area 120.

[0046] In some embodiments, the second coupling-in area 140 is arranged around the first coupling-in area 120, and / or the second coupling-out area 150 is arranged around the first coupling-out area 130.

[0047] For example, in a case where one second coupling-in area 140 is arranged around the first coupling-in area 120, the second coupling-in area 140 can be at least one of a ring shape and a semi-ring shape, so that the second coupling-in area 140 is arranged close to the first coupling-in area 120. In a case where multiple second coupling-in areas 140 are arranged around the first coupling-in area 120, the second coupling-in areas 140 can be arranged close to the first coupling-in area 120. In a case where one second coupling-out area 150 is arranged around the first coupling-out area 130, the second coupling-out area 150 can be at least one of a ring shape and a semi-ring shape, so that the second coupling-out area 150 is arranged close to the first coupling-out area 130. In a case where multiple second coupling-out areas 150 are arranged around the first coupling-out area 130, the second coupling-out areas 150 can be arranged close to the first coupling-out area 130.

[0048] Based on this, in subsequent preparation of the near-eye display device 10, the optical-mechanical device 200 of the near-eye display device 10 can be arranged according to the respective arrangement positions of the first coupling-in area 120 and the second coupling-in area 140, for example, the projection module 210 and the infrared module 220 included in the optical-mechanical device 200 are arranged, so as to improve the structural distribution compactness of the optical-mechanical device 200, reduce the volume of the optical-mechanical device 200, and thus reduce the volume of the near-eye display device 10. Accordingly, in subsequent preparation of the near-eye display device 10, the position of the imaging device 300 of the near-eye display device 10 can be arranged according to the respective arrangement positions of the first coupling-out area 130 and the second coupling-out area 150, so as to reduce the volume of the near-eye display device 10.

[0049] For example, since the first out-coupling region 130 can couple the incident light rays propagating to the first in-coupling region 120 out to the human eye, the size of the first out-coupling region 130 can be matched with the size of the human eye, so that the user can better view the display content of the eye movement tracking device 100, thereby improving the display effect of the eye movement tracking device 100 on the user. Based on the size of the first out-coupling region 130 being matched with the size of the human eye, in a case where one or more second out-coupling regions 150 are arranged around the first out-coupling region 130, the second out-coupling regions 150 can be arranged close to the first out-coupling region 130. Accordingly, in a case where the second out-coupling regions 150 are arranged close to the first out-coupling region 130, it can be ensured that the infrared light rays coupled out by the second out-coupling regions 150 can be coupled out to the human eye, thereby facilitating subsequent improvement of the convenience and accuracy of eye movement tracking on the human eye.

[0050] In some embodiments, the size of the second in-coupling region 140 is smaller than the size of the first in-coupling region 120, and / or the size of the second out-coupling region 150 is smaller than the size of the first out-coupling region 130.

[0051] For example, in a case where the size of the second in-coupling region 140 is smaller than the size of the first in-coupling region 120, as many second in-coupling regions 140 as possible can be arranged around the first in-coupling region 120 to more comprehensively receive infrared light rays of different incident angles. In a case where the size of the second out-coupling region 150 is smaller than the size of the first out-coupling region 130, as many second out-coupling regions 150 as possible can be arranged around the first out-coupling region 130 to more comprehensively provide infrared light rays of different exit angles to the human eye. Accordingly, in a case where the number of second in-coupling regions 140 and / or the number of second out-coupling regions 150 increases, since the sizes of the second in-coupling regions 140 and / or the second out-coupling regions 150 are small, the volume of the eye movement tracking device 100 will not be additionally increased, thereby facilitating improvement of the structural compactness of the eye movement tracking device 100.

[0052] Since the corresponding incident angle range of different positions of the same second in-coupling region 140 can be different, and the incident angle range of each of the plurality of second in-coupling regions 140 can also be different, the infrared light rays irradiating the eye movement tracking device 100 at different incident angles can be respectively coupled into the waveguide substrate 110 after irradiating different positions of the same second in-coupling region 140 or different second in-coupling regions 140, and propagate in the waveguide substrate 110 to one or more second out-coupling regions 150 corresponding to different positions of the same second in-coupling region 140 or different second in-coupling regions 140 respectively, and then be out-coupled to different positions on the cornea of the human eye. The cornea of the human eye can reflect the infrared light rays respectively coupled out by the different second out-coupling regions 150, forming a plurality of corresponding light spots on the cornea of the human eye. The plurality of light spots can be used for eye movement tracking of the human eye in combination with the positions of the plurality of light spots on the cornea of the human eye, which is conducive to improving the convenience and accuracy of eye movement tracking of the human eye. Correspondingly, when the human eye moves to different positions, the infrared light rays can be coupled in through different positions of the same second in-coupling region 140 or different second in-coupling regions 140, and coupled out through the corresponding second out-coupling regions 150 to the human eye, so that the coupled-out infrared light rays can irradiate the human eye, such as the cornea of the human eye, which is conducive to improving the accuracy of eye movement tracking of the human eye.

[0053] For example, the first in-coupling region 120 and the first out-coupling region 130 satisfy the wave vector closure relationship.

[0054] Please refer to Figure 3 , Figure 3 is Figure 2 the wave vector diagram of the eye movement tracking device 100 provided.

[0055] For example, the eye movement tracking device 100 includes a plurality of second in-coupling regions 140 and second out-coupling regions 150. As Figure 3 shown, K1in is used to indicate the wave vector of the first in-coupling region 120. K1out1 and K1out2 are respectively used to indicate the wave vector of the first out-coupling region 130.

[0056] As Figure 3 can be seen, the first in-coupling region 120 and the plurality of second in-coupling regions 140 are all in the inner circle, and the first out-coupling region 130 and the plurality of second out-coupling regions 150 are all in the outer circle. The radius of the inner circle can be represented as n0 is used to indicate the refractive index of air. The radius of the outer circle can be represented as n is used to indicate the refractive index of the eye-tracking device 100. The annular region formed by the inner and outer rings is used to indicate the propagation of incident light rays coupled into the waveguide substrate 110 via the first coupling region 120 and infrared light rays coupled into the waveguide substrate 110 via the second coupling region 140 under total internal reflection conditions in the waveguide substrate 110. The rectangular region at the center (origin of the coordinate system) is used to indicate the field of view of the incident light rays emitted by the projection module 210, and the center of the circle is used to indicate that the incident light rays are perpendicularly incident into the first coupling region 120. The incident light rays can be shifted to the annular region by the K1in action of the first coupling region 120 to couple into the waveguide substrate 110. The incident light rays coupled into the waveguide substrate 110 can propagate in the waveguide substrate 110. Correspondingly, the incident light rays propagating in the waveguide substrate 110 can be shifted back into the inner ring by the K1out1 and K1out2 actions of the first coupling region 130 to couple out to the human eye.

[0057] Accordingly, when multiple second coupling regions 140 and multiple second coupling regions 150 are provided on the waveguide substrate 110, the corresponding second coupling regions 140 and second coupling regions 150 also conform to the wave vector closure relationship.

[0058] In some implementations, the second coupling-in region 140 and the second coupling-out region 150 may correspond one-to-one.

[0059] like Figure 3 As shown, taking multiple second coupling regions 140 including second coupling regions 1 to 2 coupling regions p, and multiple second coupling regions 150 including second coupling regions 1 to 2 coupling regions p as an example, the second coupling region 1 and the second coupling region p correspond one-to-one, the second coupling region p and the second coupling region p correspond one-to-one, and so on. K2in1 is used to indicate the wave vector of the second coupling region 1, K2out1 is used to indicate the wave vector of the second coupling region 1, K2inp is used to indicate the wave vector of the second coupling region p, K2outp is used to indicate the wave vector of the second coupling region p, and so on.

[0060] Depend on Figure 2 as well as Figure 3It can be known that the incident angle ranges of the second coupling-in regions 140 at different positions on the waveguide substrate 110 can be different, and the infrared light has a corresponding field of view angle. In the case that the infrared light irradiates the second coupling-in region 1, the infrared light can be moved to the annular region by the K2in1 of the second coupling-in region 1, so as to be coupled into the waveguide substrate 110. The infrared light coupled into the waveguide substrate 110 can propagate in the waveguide substrate 110. Correspondingly, the infrared light propagating in the waveguide substrate 110 can be moved back to the inner circle by the K2out1 of the second coupling-out region 1, so as to be coupled out to the human eye. By analogy, in the case that the infrared light irradiates the second coupling-in region p, the infrared light can be moved to the annular region by the K2inp of the second coupling-in region p, and moved back to the inner circle by the K2outp of the second coupling-out region p, so as to be coupled out to the human eye.

[0061] In some embodiments, the second coupling-in region 140 and the second coupling-out region 150 can have a one-to-many corresponding relationship.

[0062] Please refer to Figure 4 , Figure 4 is a structural schematic diagram of an eye movement tracking device 100 involved in an embodiment of the present application.

[0063] As Figure 4 shown, the waveguide substrate 110 of the eye movement tracking device 100 can be provided with at least one second coupling-in region 140 and a plurality of second coupling-out regions 150. In the case that the infrared light irradiates one of the second coupling-in regions 140, the second coupling-in region 140 can generate a plurality of diffraction orders, and then the infrared light can propagate to the corresponding plurality of second coupling-out regions 150 through the respective propagation paths of the plurality of diffraction orders after being coupled into the waveguide substrate 110 by the second coupling-in region 140, and be coupled out to the human eye by the plurality of second coupling-out regions 150.

[0064] Please refer to Figure 5 , Figure 5 is Figure 4 a wave vector schematic diagram of the eye movement tracking device 100 provided.

[0065] As Figure 5As shown, the relationship between the wave vector of the infrared light and the wave vectors of the second coupling region 140 and the second coupling region 150 can be expressed as K2out = Kred + K2in. Kred indicates the wave vector of the infrared light, K2in indicates the wave vector of the second coupling region 140, and K2out indicates the wave vector of the second coupling region 150. Taking at least one second coupling region 140 including second coupling region 1 and multiple second coupling regions 150 including second coupling regions 1 to q as an example, the second coupling region 1 and the second coupling regions 1 to q have a one-to-many correspondence. K2in can include K2in1, which indicates the wave vector of the second coupling region 1. K2out can include K2out1 to K2outq, where K2out1 indicates the wave vector of the second coupling region 1, K2outq indicates the wave vector of the second coupling region q, and so on.

[0066] Depend on Figure 4 as well as Figure 5 It can be seen that when infrared light illuminates the second coupling region 1, since the second coupling region 1 can generate multiple diffraction orders, the infrared light can be shifted to multiple sub-regions of the annular region by the K2in1 effect of the second coupling region 1, and coupled into the waveguide substrate 110. Correspondingly, infrared light of different diffraction orders coupled into the waveguide substrate 110 can propagate in different directions to the corresponding second coupling region 150, such as from the second coupling region 1 to the second coupling region q. When infrared light of one diffraction order propagates to the second coupling region 1, it can be shifted back to the inner circle by the K2out1 effect of the second coupling region 1, and coupled out to the human eye. When infrared light of another diffraction order propagates to the second coupling region q, it can be shifted back to the inner circle by the K2outq effect of the second coupling region q, and coupled out to the human eye. And so on.

[0067] In some embodiments, the eye-tracking device 100 further includes at least one third coupling-in region 160 and at least one third coupling-out region 170 disposed on the waveguide substrate 110; the third coupling-in region 160 is disposed close to the first coupling-in region 120 relative to the first coupling-out region 130, and the location of the third coupling-in region 160 is different from the location of the second coupling-in region 140; the third coupling-out region 170 is disposed close to the first coupling-out region 130 relative to the first coupling-in region 120, and the location of the third coupling-out region 170 is different from the location of the second coupling-out region 150; wherein, at least one third coupling-in region 160 is used to receive infrared light and couple the infrared light into the waveguide substrate 110, propagate it in the waveguide substrate 110 to one or more corresponding third coupling-out regions 170, and couple it out to the human eye via the third coupling-out region 170.

[0068] Exemplarily, the number of the third in-coupling regions 160 can include one or more, and the number of the third out-coupling regions 170 can also include one or more.

[0069] For example, in the case that the number of the third in-coupling regions 160 is one, different positions of the third in-coupling region 160 can have different ranges of incident angles. The third in-coupling region 160 can be at least one of a ring shape, a semi-ring shape, and the like, for example, and can be arranged to surround the first in-coupling region 120. When infrared light is incident on the eye movement tracking device 100 and the incident angle of the infrared light is within the range of incident angles corresponding to one of the positions of the third in-coupling region 160, the infrared light can be incident on the corresponding position of the third in-coupling region 160. The range of incident angles corresponding to each of the different positions of the third in-coupling region 160 is different from the range of incident angles of the second in-coupling region 140. Based on this, the second in-coupling region 140 and the third in-coupling region 160 included in the eye movement tracking device 100 facilitate receiving infrared light having different incident angles. Accordingly, the corresponding position of the third in-coupling region 160 that receives the infrared light can couple the infrared light into the waveguide substrate 110. The infrared light coupled into the waveguide substrate 110 can propagate at a predetermined angle until it propagates to the corresponding one or more third out-coupling regions 170 and is coupled out of the human eye by the third out-coupling region 170.

[0070] For example, the third in-coupling region 160 can have a predetermined range of incident angles. When infrared light is incident on the eye movement tracking device 100 and the incident angle of the infrared light is within the range of incident angles of the third in-coupling region 160, the infrared light can be incident on the third in-coupling region 160. The range of incident angles of the third in-coupling region 160 is different from the range of incident angles of the second in-coupling region 140. Based on this, the second in-coupling region 140 and the third in-coupling region 160 included in the eye movement tracking device 100 facilitate receiving infrared light having different incident angles. Accordingly, the third in-coupling region 160 that receives the infrared light can couple the infrared light into the waveguide substrate 110. The infrared light coupled into the waveguide substrate 110 can propagate at a predetermined angle until it propagates to the corresponding one or more third out-coupling regions 170 and is coupled out of the human eye by the third out-coupling region 170.

[0071] Of course, the third in-coupling region 160 and the third out-coupling region 170 can also have a one-to-one relationship, a one-to-many relationship, or a many-to-one relationship, without being limited thereto.

[0072] In some embodiments, the third in-coupling region 160 is distributed around the first in-coupling region 120, and the third in-coupling region 160 is disposed closer to the first in-coupling region 120 than to the first out-coupling region 130. The third out-coupling region 170 is distributed around the first out-coupling region 130, and the third out-coupling region 170 is disposed closer to the first out-coupling region 130 than to the first in-coupling region 120. Since the third in-coupling region 160 has a different range of incident angles than the second in-coupling region 140, the third in-coupling region 160 is disposed differently than the second in-coupling region 140. Accordingly, to facilitate the formation of a spot on different locations of the cornea of a human eye, the third out-coupling region 170 can have a different range of exit angles for infrared light than the second out-coupling region 150, and the third out-coupling region 170 is disposed differently than the second out-coupling region 150.

[0073] In some embodiments, the third in-coupling region 160 is disposed around the first in-coupling region 120, and / or the third out-coupling region 170 is disposed around the first out-coupling region 130. For example, the third in-coupling region 160 can have a number of one, and the third in-coupling region 160 can have at least one of a ring shape and a semi-ring shape, and the third in-coupling region 160 can be disposed around the first in-coupling region 120. The third in-coupling region 160 can also have a number of multiple, and be disposed around the first in-coupling region 120. The third out-coupling region 170 can have a number of one, and the third out-coupling region 170 can have at least one of a ring shape and a semi-ring shape, and the third out-coupling region 170 can be disposed around the first out-coupling region 130. The third out-coupling region 170 can also have a number of multiple, and be disposed around the first out-coupling region 130, without limitation.

[0074] In some embodiments, the third in-coupling region 160 has a size smaller than a size of the first in-coupling region 120, and / or the third out-coupling region 170 has a size smaller than a size of the first out-coupling region 130.

[0075] When the eye-tracking device 100 couples infrared light into the human eye through the third coupling region 160 and couples infrared light out through the third coupling region 170, and / or couples infrared light into the human eye through the second coupling region 140 and couples infrared light out through the second coupling region 150, different positions of the human eye can reflect infrared light at different emission angles, thus forming a corresponding light spot in the human eye. When the light spot is located on the cornea of ​​the human eye, the light spot can be used for eye tracking of the user. Based on this, the eye-tracking device 100 can be used to subsequently manufacture a near-eye display device 10 to realize the eye-tracking function of the near-eye display device 10. Accordingly, when the user's eye moves to different positions, infrared light can be coupled into the eye through at least one of the second coupling region 140 and the third coupling region 160, and infrared light can be coupled out to the eye through the corresponding second coupling region 150 or the third coupling region 170, so that the coupled infrared light can illuminate the cornea of ​​the eye, which helps to improve the accuracy of eye movement tracking.

[0076] For example, when at least one third coupling region 160 and at least one third coupling region 170 are provided on the waveguide substrate 110, the corresponding third coupling region 160 and third coupling region 170 conform to the wave vector closure relationship.

[0077] In some implementations, the third coupling in region 160 and the third coupling out region 170 may correspond one-to-one.

[0078] like Figure 6 As shown, taking at least one third coupling region 160 including third coupling regions 1 to 1 to 1 u, and multiple third coupling regions 170 including third coupling regions 1 to 1 to 1 u as an example, the third coupling region 1 and the third coupling region 1 correspond one-to-one, the third coupling region u and the third coupling region u correspond one-to-one, and so on. K3in1 is used to indicate the wave vector of the third coupling region 1, K3out1 is used to indicate the wave vector of the third coupling region 1, K3inu is used to indicate the wave vector of the third coupling region u, K3outu is used to indicate the wave vector of the third coupling region u, and so on.

[0079] Since the range of incident angles of the third in-coupling regions 160 at different positions on the waveguide substrate 110 can be different, and the infrared light has a corresponding field of view angle. In the case that the infrared light irradiates the third in-coupling region 1, the infrared light can be moved to the annular region by the action of K3in1 of the third in-coupling region 1, to be in-coupled into the waveguide substrate 110. The infrared light in-coupled into the waveguide substrate 110 can propagate in the waveguide substrate 110. Correspondingly, the infrared light propagating in the waveguide substrate 110 can be moved back into the inner circle by the action of K3out1 of the third out-coupling region 1, to be out-coupled to the human eye. By analogy, in the case that the infrared light irradiates the third in-coupling region u, the infrared light can be moved to the annular region by the action of K3inu of the third in-coupling region u, and moved back into the inner circle by the action of K3outu of the third out-coupling region u, to be out-coupled to the human eye.

[0080] In some embodiments, the third in-coupling region 160 and the third out-coupling region 170 can have a one-to-many correspondence.

[0081] As shown in Figure 4 , the waveguide substrate 110 of the eye movement tracking device 100 can be provided with at least one third in-coupling region 160 and a plurality of third out-coupling regions 170. In the case that the infrared light irradiates one of the third in-coupling regions 160, the third in-coupling region 160 can generate a plurality of diffraction orders, and then the infrared light can propagate to the corresponding plurality of third out-coupling regions 170 through the waveguide substrate 110 by a plurality of diffraction orders respectively corresponding to the propagation paths, and be out-coupled to the human eye through the plurality of third out-coupling regions 170.

[0082] As shown in Figure 5 , the relationship between the wave vector of the infrared light and the wave vectors of the third in-coupling region 160 and the third out-coupling region 170 can be represented as K3out = Kred + K3in. Kred is used to indicate the wave vector of the infrared light, K3in is used to indicate the wave vector of the third in-coupling region 160, and K3out is used to indicate the wave vector of the third out-coupling region 170. Taking the case that the at least one third in-coupling region 160 includes the third in-coupling region 1, and the plurality of third out-coupling regions 170 include the third out-coupling region 1 to the third out-coupling region v as an example, the third in-coupling region 1 has a one-to-many correspondence with the third out-coupling region 1 to the third out-coupling region v. Wherein, K3in can include K3in1, and K3in1 is used to indicate the wave vector of the third in-coupling region 1. K3out can include K3out1 to K3outv, K3out1 is used to indicate the wave vector of the third out-coupling region 1, and K2outv is used to indicate the wave vector of the third out-coupling region v. By analogy.

[0083] By Figure 4 and Figure 5It can be known that, in the case that the infrared light irradiates the third coupling-in region 1, since the third coupling-in region 1 can generate multiple diffraction orders, the infrared light can be shifted to multiple sub-regions of the annular region by the action of K3in1 of the third coupling-in region 1 to be coupled into the waveguide substrate 110. Correspondingly, the infrared light of different diffraction orders of the coupling-in waveguide substrate 110 can propagate to corresponding third coupling-out regions 170, such as the third coupling-out region 1 to the third coupling-out region v, in different directions. In the case that the infrared light of one of the diffraction orders propagates to the third coupling-out region 1, it can be shifted back to the inner circle by the action of K3out1 of the third coupling-out region 1 to be coupled out to the human eye. In the case that the infrared light of another diffraction order propagates to the third coupling-out region v, it can be shifted back to the inner circle by the action of K2outv of the third coupling-out region v to be coupled out to the human eye. Similarly, the infrared light of other diffraction orders can be shifted back to the inner circle by the action of the corresponding third coupling-out region to be coupled out to the human eye.

[0084] The eye movement tracking device 100 provided by the embodiment of the present application comprises a waveguide substrate 110, a first coupling-in region 120, a first coupling-out region 130, at least one second coupling-in region 140 and at least one second coupling-out region 150 arranged on the waveguide substrate 110, the second coupling-in region 140 is arranged close to the first coupling-in region 120 relative to the first coupling-out region 130, the second coupling-out region 150 is arranged close to the first coupling-out region 130 relative to the first coupling-in region 120, wherein the first coupling-in region 120 is used to couple the incident light emitted by the projection module 210 out to the human eye, the second coupling-in region 140 is used to receive infrared light of different incident angles and couple the infrared light into the waveguide substrate 110, propagate in the waveguide substrate 110 to the corresponding one or more second coupling-out regions 150, and be coupled out to the human eye through the second coupling-out region 150.

[0085] Based on the arrangement of the at least one second coupling-in region 140 and the at least one second coupling-out region 150, when the human eye is in different positions, such as the cornea of the human eye is not rotated to the inner side of the eye socket or the cornea of the human eye is rotated to the inner side of the eye socket, the eye movement tracking device 100 can couple the infrared light of the corresponding incident angle through the different second coupling-in regions 140 or the corresponding positions of the same second coupling-in region 140, so that the infrared light coupled into the waveguide substrate 110 can be coupled out to the human eye after propagating in the waveguide substrate 110 to the corresponding one or more second coupling-out regions 150. The infrared light can be coupled out to the cornea of the human eye after the action of the eye movement tracking device 100, so as to form a corresponding light spot on the cornea of the human eye. The light spot on the cornea of the human eye can be used for eye movement tracking of the human eye, which is beneficial to improve the convenience and accuracy of eye movement tracking of the human eye.

[0086] Please refer to Figure 7 , Figure 7 which is a structural schematic diagram of a near-eye display device 10 provided by the embodiment of the present application.

[0087] As shown in Figure 7 The near-eye display device 10 includes an eye movement tracking apparatus 100, an optical-mechanical apparatus 200, and an imaging apparatus 300.

[0088] The eye movement tracking apparatus 100 is the aforementioned eye movement tracking apparatus 100. The specific principles and implementation manners of the eye movement tracking apparatus 100 can refer to the eye movement tracking apparatus 100 of the aforementioned embodiments, which will not be described here again.

[0089] The optical-mechanical apparatus 200 includes a projection module 210 and an infrared module 220. The projection module 210 is configured to emit incident light to the first coupling-in region 120 of the eye movement tracking apparatus 100. The first coupling-in region 120 is configured to couple the incident light into the waveguide substrate 110 of the eye movement tracking apparatus 100. The first coupling-out region 130 of the eye movement tracking apparatus 100 is configured to couple the incident light propagating in the waveguide substrate 110 out to the human eye. The infrared module 220 includes an infrared light source 221 and an angle adjusting member 222. The angle adjusting member 222 is configured to adjust the light angle of the infrared light provided by the infrared light source 221, so that the adjusted infrared light irradiates one of the second coupling-in regions 140 of the eye movement tracking apparatus 100. After being coupled into the waveguide substrate 110 through the second coupling-in region 140, the infrared light propagates to the corresponding one or more second coupling-out regions 150, and is coupled out to the human eye through the second coupling-out region 150.

[0090] The imaging apparatus 300 is configured to capture the image of the human eye of the user, so as to realize eye movement tracking.

[0091] For example, corresponding to the human eye of the user, the near-eye display device 10 can include a left eye movement tracking apparatus and a right eye movement tracking apparatus. The left eye movement tracking apparatus can be configured to track the left eye of the user, and the right eye movement tracking apparatus can be configured to track the right eye of the user. The structure of the left eye movement tracking apparatus can be consistent with or different from the structure of the right eye movement tracking apparatus. For example, the first coupling-in region 120, the first coupling-out region 130, the at least one second coupling-in region 140, and the at least one second coupling-out region 150 can be arranged on the left eye movement tracking apparatus, and the second coupling-in region 140 and the second coupling-out region 150 have a one-to-one correspondence. Correspondingly, the first coupling-in region 120, the first coupling-out region 130, the at least one second coupling-in region 140, and the at least one second coupling-out region 150 can also be arranged on the right eye movement tracking apparatus, and the second coupling-in region 140 and the second coupling-out region 150 also have a one-to-one correspondence. Alternatively, the second coupling-in region 140 and the second coupling-out region 150 arranged on the right eye movement tracking apparatus 100 have a one-to-many correspondence. For another example, the first coupling-in region 120 and the at least one second coupling-in region 140 can be shared by the left eye movement tracking apparatus and the right eye movement tracking apparatus, which is not limited herein.

[0092] In some embodiments, the near-eye display device 10 can provide the incident light and the infrared light to the eye movement tracking device 100 through the light engine 200. For example, the light engine 200 can include a projection module 210 and an infrared module 220. The projection module 210 and the infrared module 220 can be integrally arranged or separately arranged, which is not limited herein. The angle adjustment member 222 included in the infrared module 220 can be arranged inside the light engine 200 or outside the light engine 200, which is not limited herein.

[0093] For example, the projection module 210 can emit the incident light to the first in-coupling region 120 of the eye movement tracking device 100, so as to couple the incident light into the waveguide substrate 110 through the first in-coupling region 120, propagate in the waveguide substrate 110 to the first out-coupling region 130, and then out-couple to the human eye. Based on this, the near-eye display device 10 can have a display function.

[0094] For example, the infrared module 220 can provide the infrared light to the eye movement tracking device 100. The infrared module 220 can include an infrared light source 221 and the angle adjustment member 222. The infrared light source 221 can provide the infrared light. The infrared light provided by the infrared light source 221 can irradiate one of the second in-coupling regions 140 after the action of the angle adjustment member 222. Correspondingly, the second in-coupling region 140 receiving the infrared light can couple the infrared light into the waveguide substrate 110, propagate in the waveguide substrate 110 to the corresponding one or more second out-coupling regions 150, and then out-couple to the human eye, such as the cornea of the human eye. The cornea of the human eye can reflect the out-coupled infrared light, and then a corresponding light spot can be formed on the cornea of the human eye in the human eye image captured by the imaging device 300. The light spot on the cornea of the human eye can be used for eye movement tracking of the human eye.

[0095] In some embodiments, when the human eye rotates to different positions, the light angle of the infrared light can be adjusted through the angle adjustment member 222, so that the adjusted infrared light can irradiate different positions of the same second in-coupling region 140 or different second in-coupling regions 140, thereby coupling into the waveguide substrate 110 through the different positions of the same second in-coupling region 140 or the different second in-coupling regions 140, propagating in the waveguide substrate 110 to the corresponding one or more second out-coupling regions 150 of the different positions of the same second in-coupling region 140 or the different second in-coupling regions 140, and then out-coupling to different positions of the human eye, such as different positions of the cornea of the human eye. Correspondingly, in the case that the out-coupled infrared light irradiates the cornea of the human eye, it is beneficial to improve the accuracy of eye movement tracking of the human eye.

[0096] For example, the angle adjusting member 222 is further configured to adjust the angle of the infrared light provided by the infrared light source 221, so that the adjusted infrared light is incident on one of the third in-coupling regions 160, is coupled into the waveguide substrate 110 via the third in-coupling region 160, propagates to one or more corresponding third out-coupling regions 170 via the waveguide substrate 110, and is coupled out of the waveguide substrate 110 via the third out-coupling region 170 to the human eye.

[0097] For example, in the case where the waveguide substrate 110 of the eye tracking device 100 is provided with the third in-coupling regions 160 and the third out-coupling regions 170, the infrared light provided by the infrared light source 221 can be incident on one of the third in-coupling regions 160 via the angle adjusting member 222. Accordingly, the third in-coupling region 160 receiving the infrared light can couple the infrared light into the waveguide substrate 110, propagate to one or more corresponding third out-coupling regions 170 via the waveguide substrate 110, and be coupled out of the waveguide substrate 110 via the corresponding third out-coupling region 170 to the human eye, such as the cornea of the human eye. The cornea of the human eye can reflect the coupled-out infrared light, and a corresponding light spot can be formed on the cornea of the human eye in the human eye image captured by the imaging device 300. The light spot on the cornea of the human eye can be used for eye tracking of the human eye.

[0098] In some embodiments, when the human eye moves to different positions, the angle of the infrared light can be adjusted by the angle adjusting member 222, so that the adjusted infrared light can be incident on different positions of the same third in-coupling region 160 or different third in-coupling regions 160, thereby being coupled into the waveguide substrate 110 via the different positions of the same third in-coupling region 160 or the different third in-coupling regions 160, propagating to one or more corresponding third out-coupling regions 170 of the different positions of the same third in-coupling region 160 or the different third in-coupling regions 160 via the waveguide substrate 110, and being coupled out of the waveguide substrate 110 to different positions of the human eye, such as different positions of the cornea of the human eye. In other embodiments, when the human eye moves to different positions, the angle of the infrared light can be adjusted by the angle adjusting member 222, so that the adjusted infrared light can be incident on different second in-coupling regions 140 and third in-coupling regions 160, thereby being coupled into the waveguide substrate 110 via the second in-coupling regions 140, propagating to one or more corresponding second out-coupling regions 150 via the waveguide substrate 110, and being coupled out of the waveguide substrate 110 to the human eye, and being coupled into the waveguide substrate 110 via the third in-coupling regions 160, propagating to one or more corresponding third out-coupling regions 170 via the waveguide substrate 110, and being coupled out of the waveguide substrate 110 to the human eye. Accordingly, in the case where the coupled-out infrared light is incident on the cornea of the human eye, the accuracy of eye tracking of the human eye can be improved.

[0099] The imaging device 300 can capture an image of the human eye of the user, and the near-eye display device 10 can subsequently perform eye tracking of the human eye based on the image of the human eye.

[0100] In some embodiments, the angle adjustment member 222 includes at least one of a reflector 2221 and an adjustable liquid crystal grating 2222.

[0101] like Figure 7 As shown, the angle adjustment element 222 may include a reflector 2221. The reflector 2221 may be disposed inside the optomechanical device 200. The position of the reflector 2221 within the optomechanical device 200 is changeable. The reflector 2221 in different positions can illuminate different locations within the same second coupling region 140, different second coupling regions 140, different locations within the same third coupling region 160, different third coupling regions 160, or at least one of at least one second coupling region 140 and at least one third coupling region 160 with infrared light provided by the infrared light source 221.

[0102] like Figure 8a as well as Figure 8c As shown, when the reflector 2221 is set in the first position, the infrared light provided by the infrared light source 221 can illuminate one of the second coupling regions 140 after being acted upon by the reflector 2221 in the first position. After being coupled into the waveguide substrate 110 through the second coupling region 140, it propagates through the waveguide substrate 110 to one or more corresponding second coupling regions 150, and is coupled out through the second coupling region 150 to the cornea of ​​the human eye.

[0103] like Figure 8b as well as Figure 8c As shown, when the reflector 2221 is set in the second position, the infrared light provided by the infrared light source 221 can illuminate another second coupling region 140 after being acted upon by the reflector 2221 in the second position. After being coupled into the waveguide substrate 110 through the second coupling region 140, it propagates through the waveguide substrate 110 to one or more corresponding second coupling regions 150, and is coupled out through the second coupling region 150 to the cornea of ​​the human eye.

[0104] Of course, the position of the reflector 2221 is not limited to the first position or the second position, and no restrictions are made here.

[0105] like Figure 9As shown, the angle adjustment element 222 may include an adjustable liquid crystal grating 2222. The adjustable liquid crystal grating 2222 may be disposed outside the optomechanical device 200. Since the alignment direction of the liquid crystal molecules in the adjustable liquid crystal grating 2222 is adjustable, the adjustable liquid crystal grating 2222 can have different operating states. For example, an external voltage may be applied to the adjustable liquid crystal grating 2222 to adjust the alignment direction of the liquid crystal molecules, thereby changing the operating state of the adjustable liquid crystal grating 2222. The reflector 2221 in different operating states can irradiate the infrared light provided by the infrared light source 221 onto at least one of the following: different positions of the same second coupling region 140, different second coupling regions 140, different positions of the same third coupling region 160, different third coupling regions 160, at least one second coupling region 140, and at least one third coupling region 160.

[0106] like Figure 10a as well as Figure 10c As shown, when the adjustable liquid crystal grating 2222 is in the first working state, the infrared light provided by the infrared light source 221 can illuminate one of the third coupling regions 160 after being acted upon by the first working state. After being coupled into the waveguide substrate 110 through the third coupling region 160, it propagates through the waveguide substrate 110 to one or more corresponding third coupling regions 170, and is coupled out to the cornea of ​​the human eye through the third coupling region 170.

[0107] like Figure 10a as well as Figure 10c As shown, when the adjustable liquid crystal grating 2222 is in the second working state, the infrared light provided by the infrared light source 221 can illuminate one of the second coupling regions 140 after being coupled into the waveguide substrate 110 through the second working state, and then propagate through the waveguide substrate 110 to one or more corresponding second coupling regions 150, and then be coupled out through the second coupling regions 150 to the cornea of ​​the human eye.

[0108] Of course, the working states of the adjustable liquid crystal grating 2222 are not limited to the first working state and the second working state, and no restrictions are made here.

[0109] For example, the angle adjustment element 222 may include a reflector 2221 and an adjustable liquid crystal grating 2222. Then, multiple infrared rays provided by the infrared light source 221 can be irradiated by the combined action of the reflector 2221 in the corresponding setting position and the adjustable liquid crystal grating 2222 in the corresponding working state to at least one of the following: different positions of the same second coupling region 140, different second coupling regions 140, different positions of the same third coupling region 160, different third coupling regions 160, at least one second coupling region 140 and at least one third coupling region 160, thereby emitting adjusted infrared rays to the human eye. No limitation is imposed here.

[0110] In some embodiments, the adjustable liquid crystal grating 2222 is disposed adjacent to the projection module 210 and the infrared light source 221, respectively. The optomechanical device 200 also includes a beam combiner 230, which is used to combine the incident light emitted by the projection module 210 and the infrared light provided by the infrared light source 221 and then project them onto the adjustable liquid crystal grating 2222.

[0111] like Figures 10a to 10c As shown, the projection module 210 and the infrared light source 221 can be separately configured. The direction of the incident light emitted by the projection module 210 is different from the direction of the infrared light provided by the infrared light source 221. The projection module 210 and the infrared light source 221 are located on one side of the adjustable liquid crystal grating 2222, and the first coupling region 120 is located on the other side of the adjustable liquid crystal grating 2222. Therefore, the near-eye display device 10 needs to ensure that the incident light can illuminate the first coupling region 120 and that the infrared light can be incident on the adjustable liquid crystal grating 2222. Based on this, the near-eye display device 10 can control the respective light directions of the incident light and the infrared light through the beam combining component 230 included in the optomechanical device 200. The beam combining component 230 can be located on the light-emitting side of the infrared light source 221 and the light-emitting side of the projection module 210. The beam combining component 230 can combine the incident light and the infrared light and then project them onto the adjustable liquid crystal grating 2222. For example, infrared light provided by infrared light source 221 can be reflected onto adjustable liquid crystal grating 2222 by beam combining component 230, so that adjustable liquid crystal grating 2222 can adjust the direction of infrared light. Correspondingly, incident light provided by projection module 210 can be transmitted onto adjustable liquid crystal grating 2222 by beam combining component 230, so that adjustable liquid crystal grating 2222 can transmit incident light, thereby allowing the incident light to illuminate the first coupling region 120.

[0112] In one exemplary embodiment, the beam combiner 230 may include a semi-reflective mirror. The semi-reflective mirror may be derived from a corresponding design of the reflector 2221, and is not limited thereto.

[0113] The near-eye display device 10 provided by the embodiment of the present application comprises: an eye movement tracking device 100, which is the aforementioned eye movement tracking device 100; an optical-mechanical device 200, which comprises a projection module 210 and an infrared module 220, the projection module 210 is used for emitting incident light to a first coupling-in area 120 of the eye movement tracking device 100, the first coupling-in area 120 is used for coupling the incident light into a waveguide substrate 110 of the eye movement tracking device 100, and a first coupling-out area 130 of the eye movement tracking device 100 is used for coupling the incident light propagating in the waveguide substrate 110 out to a human eye; the infrared module 220 comprises an infrared light source 221 and an angle adjusting member 222, the angle adjusting member 222 is used for adjusting the light angle of the infrared light provided by the infrared light source 221, so that the adjusted infrared light irradiates one of the second coupling-in areas 140 of the eye movement tracking device 100, propagates to the corresponding one or more second coupling-out areas 150 in the waveguide substrate 110 after being coupled into the waveguide substrate 110 through the second coupling-in area 140, and is coupled out to the human eye through the second coupling-out area 150; and an imaging device 300, which is used for shooting the image of the human eye of the user to realize eye movement tracking.

[0114] Based on the settings of the eye movement tracking device 100 and the infrared module 220, the light angle of the infrared light provided by the infrared light source 221 can be adjusted through the angle adjusting member 222, so that the adjusted infrared light can be coupled out to the human eye through the second coupling-in area 140, the waveguide substrate 110, and the corresponding one or more second coupling-out areas 150 of the second coupling-in area 140 in sequence. Accordingly, when the human eye is in different positions, the infrared light is irradiated to different second coupling-in areas 140 through the action of the angle adjusting member 222, and then can be coupled out to different positions of the human eye, such as different positions of the cornea of the human eye, through the actions of different second coupling-in areas 140, the waveguide substrate 110, and the corresponding one or more second coupling-out areas 150 of different second coupling-in areas 140 respectively, thereby forming light spots at different positions on the cornea of the human eye. The light spots at different positions on the cornea of the human eye can be used for eye movement tracking of the human eye, thereby being beneficial to improving the convenience and accuracy of eye movement tracking of the human eye.

[0115] Please refer to Figure 9 , Figure 9 is a flowchart of an eye movement tracking method of a near-eye display device 10 provided by the embodiment of the present application. The eye movement tracking method is used for the aforementioned near-eye display device 10. The specific principles and implementation manners of the near-eye display device 10 involved in the embodiment of the present application can be referred to the aforementioned eye movement tracking device 100 and the near-eye display device 10, which will not be described herein again.

[0116] As Figure 9As shown, the eye movement tracking method of the near-eye display device 10 includes steps S101 to S104.

[0117] S101, control the infrared light source 221 of the near-eye display device 10 to provide infrared light rays.

[0118] For example, the near-eye display device 10 can provide the infrared light rays through the infrared light source 221 of the infrared module 220. The infrared light rays can be irradiated on the angle adjustment member 222 of the near-eye display device 10.

[0119] S102, adjust the ray angle of the infrared light rays provided by the infrared light source 221 through the angle adjustment member 222 of the near-eye display device 10 to provide the adjusted infrared light rays to the human eye of the user.

[0120] In some embodiments, the angle adjustment member 222 can adjust the ray angle of the infrared light rays provided by the infrared light source 221 in response to the rotation of the eyeball of the user, which is not limited herein.

[0121] For example, the waveguide substrate 110 of the eye movement tracking device 100 can be provided with at least one second in-coupling region 140 and at least one second out-coupling region 150. The near-eye display device 10 can adjust the ray angle of the infrared light rays provided by the infrared light source 221 through the angle adjustment member 222 to different ray angles. The adjusted infrared light rays can be irradiated on one of the second in-coupling regions 140, coupled into the waveguide substrate 110 through the second in-coupling region 140, propagated in the waveguide substrate 110 to the corresponding one or more second out-coupling regions 150, and coupled out of the human eye through the second out-coupling region 150, thereby providing the adjusted infrared light rays to the human eye. Accordingly, in the case of the adjusted infrared light rays at different ray angles, the infrared light rays at different ray angles can be irradiated on different positions of the same second in-coupling region 140 or different second in-coupling regions 140, thereby being coupled out of the human eye through the different positions of the same second in-coupling region 140 or different second in-coupling regions 140, the waveguide substrate 110, and the corresponding one or more second out-coupling regions 150 of the different positions of the same second in-coupling region 140 or different second in-coupling regions 140, thereby providing the adjusted infrared light rays to different positions of the human eye, such as different positions of the cornea of the human eye.

[0122] For example, the waveguide substrate 110 of the eye movement tracking device 100 can be provided with at least one second in-coupling region 140, at least one second out-coupling region 150, at least one third in-coupling region 160, and at least one third out-coupling region 170. The near-eye display device 10 can adjust the light ray angle of the infrared light provided by the infrared light source 221 to different light ray angles through the angle adjustment member 222. The adjusted infrared light can irradiate one of the second in-coupling regions 140, be coupled into the waveguide substrate 110 through the second in-coupling region 140, propagate in the waveguide substrate 110, be coupled out of the human eye through the corresponding one or more second out-coupling regions 150, and thus provide the adjusted infrared light to the user's human eye. Alternatively, the adjusted infrared light can irradiate one of the third in-coupling regions 160, be coupled into the waveguide substrate 110 through the third in-coupling region 160, propagate in the waveguide substrate 110, be coupled out of the human eye through the corresponding one or more third out-coupling regions 170, and thus provide the adjusted infrared light to the user's human eye. In the presence of adjusted infrared light at different light ray angles, the infrared light at different light ray angles can irradiate at least one of different positions of the same second in-coupling region 140, different second in-coupling regions 140, different positions of the same third in-coupling region 160, different third in-coupling regions 160, at least one second in-coupling region 140, and at least one third in-coupling region 160, and thus be coupled out of the human eye through the eye movement tracking device 100, and thus provide the adjusted infrared light to different positions of the user's human eye.

[0123] S103, acquiring an image of the user's human eye captured by the imaging device 300 of the near-eye display device 10.

[0124] For example, the imaging device 300 can capture the human eye to obtain an image of the user's human eye. The near-eye display device 10 can acquire the image of the human eye captured by the imaging device 300 for subsequent eye movement tracking of the user based on the image of the user's human eye.

[0125] S104, determining eye movement information of the user according to the image of the human eye.

[0126] For example, the eye movement information of the user can be determined according to the light spot in the image of the human eye. For example, the position offset of the light spot relative to the pupil of the user can be determined according to the image of the human eye; and the eye movement information of the user can be determined according to the position offset.

[0127] In some embodiments, image recognition can be performed on the human eye image to determine the position of the user's pupil in the human eye image and the position of the light spot in the human eye image. According to the position of the pupil in the human eye image and the position of the light spot in the human eye image, the positional offset of the light spot relative to the user's pupil is determined. For example, the positional offset in the X-axis direction can be obtained by subtracting the X-axis coordinate of the light spot in the coordinate system corresponding to the human eye image from the X-axis coordinate of the pupil in the coordinate system corresponding to the human eye image. The positional offset in the Y-axis direction can be obtained by subtracting the Y-axis coordinate of the light spot in the coordinate system corresponding to the target image from the Y-axis coordinate of the pupil in the coordinate system corresponding to the human eye image. This is not limited herein.

[0128] In the case where the human eye image includes multiple light spots, the positional offset of each light spot relative to the user's pupil is determined according to the average of the positional offset of each light spot relative to the user's pupil. Of course, this is not limited herein. In the case where the human eye image includes multiple light spots, the average of the positions of all light spots in the human eye image can be determined. According to the average of the positions of all light spots in the human eye image and the position of the pupil in the human eye image, the positional offset of the light spot relative to the user's pupil is determined, and the eye movement information of the user is determined. This is not limited herein.

[0129] For example, the adjustment parameter of the angle adjustment member 222 is adjusted according to the first preset frequency, and the light angle of the infrared light is adjusted by the angle adjustment member 222 under different adjustment parameters, respectively.

[0130] In some embodiments, the imaging device 300 is controlled to capture the human eye image of the user according to a second preset frequency, and the second preset frequency is less than the first preset frequency.

[0131] For example, the adjustment parameter of the angle adjustment member 222 includes the setting position of the mirror 2221. As shown in Figure 8a At time t1, the near-eye display device 10 can irradiate the infrared light provided by the infrared light source 221 to one of the second coupling-in regions 140 through the mirror 2221 with the first position, and the infrared light is coupled out to the human eye through the action of the eye movement tracking device 100, thereby forming a light spot 1 at the corresponding position of the human eye. As shown in Figure 8b At time t2, the near-eye display device 10 can irradiate the infrared light provided by the infrared light source 221 to the other second coupling-in region 140 through the mirror 2221 with the second position, and the infrared light is coupled out to the human eye through the action of the eye movement tracking device 100, thereby forming a light spot 2 at the corresponding position of the human eye. As shown in Figure 8cAs shown, the near-eye display device 10 can adjust the setting position of the mirror 2221 at a first preset frequency, and adjust the light angle of the infrared light through the mirror 2221 at different setting positions, so that when the imaging device 300 shoots the human eye image of the user at a second preset frequency, the human eye image can include the light spot 1 and the light spot 2, so as to track the eye movement of the user. The first preset frequency is greater than the second preset frequency.

[0132] The angle adjusting member 222 includes an adjustable liquid crystal grating 2222, and the adjustment parameter of the angle adjusting member 222 can include the working state of the adjustable liquid crystal grating 2222. Figure 10a As shown, at time t3, the near-eye display device 10 can irradiate the infrared light provided by the infrared light source 221 to one of the third coupling-in areas 160 through the adjustable liquid crystal grating 2222 in the first working state, and the infrared light is coupled out to the human eye through the action of the eye movement tracking device 100, so as to form a light spot 3 at the corresponding position of the human eye. Figure 10b As shown, at time t4, the near-eye display device 10 can irradiate the infrared light provided by the infrared light source 221 to one of the second coupling-in areas 140 through the adjustable liquid crystal grating 2222 in the second working state, and the infrared light is coupled out to the human eye through the action of the eye movement tracking device 100, so as to form a light spot 4 at the corresponding position of the human eye. Figure 10c As shown, the near-eye display device 10 can adjust the working state of the adjustable liquid crystal grating 2222 at a first preset frequency, and adjust the light angle of the infrared light through the adjustable liquid crystal grating 2222 at different working states, so that when the imaging device 300 shoots the human eye image of the user at a second preset frequency, the human eye image can include the light spot 3 and the light spot 4, so as to track the eye movement of the user. The first preset frequency is greater than the second preset frequency.

[0133] The eye movement tracking method of the near-eye display device 10 provided by the embodiment of the application includes: controlling the infrared light source 221 of the near-eye display device 10 to provide infrared light; adjusting the light angle of the infrared light provided by the infrared light source 221 through the angle adjusting member 222 of the near-eye display device 10, so as to provide the adjusted infrared light to the human eye of the user; acquiring the human eye image of the user shot by the imaging device 300 of the near-eye display device 10; and determining the eye movement information of the user according to the human eye image, so as to improve the convenience and accuracy of the eye movement tracking of the human eye.

[0134] The methodologies of the present application can be employed in a variety of computer system contexts. For example, personal computers, server computers, hand-held or laptop devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like, that perform particular tasks or implement particular abstract data types. The present application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.

[0135] By way of example, the above-described methods and apparatus can be implemented in the form of a computer program that can be run on a near-eye display device 10 to perform eye tracking on a user by the near-eye display device 10. By way of example, the near-eye display device 10 can include a VR headset, an AR headset, an MR headset, and the like, without limitation.

[0136] By way of example, the present application also provides a near-eye display device 10. The near-eye display device 10 includes a memory and a processor. The memory and the processor can be connected by a system bus. The memory can include a storage medium and an internal memory.

[0137] The storage medium can store an operating system and a computer program. The computer program, when executed, can cause the processor to perform any of the eye tracking methods of the near-eye display device 10.

[0138] The processor is configured to provide computing and control capabilities to support the operation of the near-eye display device 10.

[0139] The internal memory provides an environment for the execution of the computer program in the storage medium. The computer program, when executed by the processor, can cause the processor to perform any of the eye tracking methods of the near-eye display device 10.

[0140] It should be appreciated that the processor can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like. The general-purpose processor can be a microprocessor or the processor can be any conventional processor.

[0141] In one embodiment, the processor is configured to execute the computer program and implement the following steps when executing the computer program:

[0142] The infrared light source 221 of the near-eye display device 10 provides infrared light rays;

[0143] The angle adjusting member 222 of the near-eye display device 10 adjusts the light ray angle of the infrared light rays provided by the infrared light source 221 to provide adjusted infrared light rays to the user's eyes;

[0144] The imaging device 300 of the near-eye display device 10 captures an image of the user's eyes;

[0145] According to the image of the eyes, the eye movement information of the user is determined.

[0146] It should be noted that, for the convenience and brevity of description, the above description of the specific working process of the eye movement tracking of the near-eye display device 10 can refer to the corresponding process in the foregoing embodiments of the eye movement tracking method of the near-eye display device 10, and will not be repeated here.

[0147] The embodiments of the present application also provide a computer readable storage medium, and the computer readable storage medium stores a computer program. The method implemented by the computer program executed by the processor can refer to each embodiment of the eye movement tracking method of the near-eye display device 10.

[0148] The computer readable storage medium can be an internal storage unit of the near-eye display device 10, such as a hard disk or a memory of the near-eye display device 10. The computer readable storage medium can also be an external storage device of the near-eye display device 10, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like.

[0149] It should be understood that the terms used herein are for the purpose of describing particular embodiments and are not intended to limit the application. As used in the specification and the appended claims, the singular forms "a," "an" and "the" are intended to include plural forms as well, unless the context clearly indicates otherwise.

[0150] It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items, and that the term "at least one of" followed by a list of two or more items means any single one of the items in the list, and that the term "one or more of" followed by a list of two or more items means any single one or plurality of the items in the list. It will be further understood that the terms "comprises" and / or "comprising," or "includes" and / or "including" when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0151] The above-mentioned sequence numbers of the embodiments of the present application are only for description, and do not represent advantages or disadvantages of the embodiments. The above describes only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An eye-tracking device, characterized in that, include: Waveguide substrate: A first coupling-in region, a first coupling-out region, at least one second coupling-in region, and at least one second coupling-out region are disposed on the waveguide substrate; the second coupling-in region is disposed close to the first coupling-in region relative to the first coupling-out region; the second coupling-out region is disposed close to the first coupling-out region relative to the first coupling-in region; the second coupling-in region is formed in at least one of annular and semi-annular shapes to surround the first coupling-in region; the second coupling-out region is formed in at least one of annular and semi-annular shapes to surround the first coupling-out region; the size of the second coupling-in region is smaller than the size of the first coupling-in region, and the size of the second coupling-out region is smaller than the size of the first coupling-out region. The first coupling region is used to couple the incident light emitted by the projection module into the waveguide substrate, and the first coupling region is used to couple the incident light propagating in the waveguide substrate out to the human eye; the second coupling region is used to receive infrared light with different incident angles and couple the infrared light into the waveguide substrate, propagate in the waveguide substrate to one or more corresponding second coupling regions, and couple out through the second coupling region to the cornea of ​​the human eye, so as to realize eye tracking by capturing images of the user's eyes.

2. The eye-tracking device according to claim 1, characterized in that, The eye-tracking device further includes at least one third coupling-in region and at least one third coupling-out region disposed on the waveguide substrate; relative to the first coupling-out region, the third coupling-in region is disposed close to the first coupling-in region, and the location of the third coupling-in region is different from the location of the second coupling-in region; relative to the first coupling-in region, the third coupling-out region is disposed close to the first coupling-out region, and the location of the third coupling-out region is different from the location of the second coupling-out region; In this process, at least one third coupling-in region is used to receive infrared light and couple the infrared light into the waveguide substrate, propagate it in the waveguide substrate to one or more corresponding third coupling-out regions, and couple it out through the third coupling-out regions to the human eye.

3. A near-eye display device, characterized in that, include: An eye-tracking device, wherein the eye-tracking device is the eye-tracking device according to any one of claims 1 to 2; An optomechanical device includes a projection module and an infrared module. The projection module emits incident light into a first coupling region of the eye-tracking device. The first coupling region couples the incident light into a waveguide substrate of the eye-tracking device. A first coupling region of the eye-tracking device couples the incident light propagating in the waveguide substrate out to the human eye. The infrared module includes an infrared light source and an angle adjustment component. The angle adjustment component adjusts the angle of the infrared light provided by the infrared light source so that the adjusted infrared light illuminates one of the second coupling regions of the eye-tracking device, is coupled into the waveguide substrate through the second coupling region, propagates to one or more corresponding second coupling regions, and is coupled out to the human eye through the second coupling region. An imaging device for capturing images of a user's eyes to achieve eye tracking.

4. The near-eye display device according to claim 3, characterized in that, The angle adjustment component is also used to adjust the angle of the infrared light provided by the infrared light source, so that the adjusted infrared light illuminates one of the third coupling regions, is coupled into the waveguide substrate through the third coupling region, propagates to one or more corresponding third coupling regions, and is coupled out to the human eye through the third coupling region.

5. The near-eye display device according to claim 3 or 4, characterized in that, The angle adjustment component includes at least one of a reflector and an adjustable liquid crystal grating.

6. The near-eye display device according to claim 5, characterized in that, The adjustable liquid crystal grating is disposed adjacent to the projection module and the infrared light source, respectively. The optomechanical device further includes a beam combining component, which is used to combine the incident light emitted by the projection module and the infrared light provided by the infrared light source and then project them onto the adjustable liquid crystal grating.

7. An eye-tracking method for a near-eye display device, characterized in that, For a near-eye display device as described in any one of claims 3 to 6, the eye-tracking method comprises: The infrared light source of the near-eye display device provides infrared light; The angle of the infrared light provided by the infrared light source is adjusted by the angle adjustment component of the near-eye display device to provide the adjusted infrared light to the user's eyes; Acquire images of the user's eye captured by the imaging device of the near-eye display device; Based on the human eye image, determine the user's eye movement information.

8. The eye-tracking method according to claim 7, characterized in that, The adjustment of the angle of the infrared light provided by the infrared light source through the angle adjustment component of the near-eye display device includes: Adjust the adjustment parameters of the angle adjustment component according to the first preset frequency; The angle of the infrared light is adjusted by the angle adjustment component under different adjustment parameters; The acquisition of the user's eye image captured by the imaging device of the near-eye display device includes: The imaging device is controlled to capture images of the user's eyes at a second preset frequency; the second preset frequency is less than the first preset frequency.

Images