Eyeball tracking structure and glasses used in AR (Augmented Reality) technology

Abstract

The utility model discloses an eyeball tracking structure and eyeball tracking glasses used in AR (Augmented Reality) technology, which relate to the technical field of eyeball tracking of AR display devices and comprise a visual tracking mechanism arranged on head wearing equipment. The visual tracking mechanism comprises a side light-emitting optical fiber used for collecting the eyeball watching position and a signal collecting piece used for receiving optical signals transmitted by the side light-emitting optical fiber, and the signal collecting piece is electrically connected with a control piece. During use, speckle patterns can be captured through the focusing lens, transmitted to the complementary metal oxide semiconductor device (CMOS) through the side light-emitting optical fiber and transmitted to the control piece through the complementary metal oxide semiconductor device, and compared with a traditional mode that a camera is used for collecting eyeball pictures, data needing to be processed are less, and the accuracy of the eyeball pictures is improved. The requirement for hardware configuration is lower, more image acquisition devices do not need to be installed on head wearing equipment, the overall size and weight are reduced, and therefore the experience of a wearer is improved.

Description

Technical Field

[0001] This utility model relates to the field of eye-tracking technology for AR display devices, specifically to an eye-tracking structure and glasses used in AR technology. Background Technology

[0002] Eye tracking in AR displays is an advanced technology that uses sensors to capture and extract eye feature information and measure eye movement in real time. This allows the system to estimate the direction of the gaze or the location of the eye's gaze point, enabling the display system to track the direction of the gaze in real time. This allows the device to prioritize rendering objects near the user's gaze point, reducing scene rendering time.

[0003] Existing eye-tracking systems primarily rely on cameras built into the display system to capture images of the eyes and then track their position. The system first captures various states of the wearer's eye movements using the camera, trains a neural network through mathematical modeling or deep learning, and then uses the captured information. During use, the system receives the captured information and, after a series of judgments and analyses, achieves precise tracking of eye movements.

[0004] The shortcomings of the above-mentioned existing technical solutions are: setting up a relatively bulky signal processing device on the AR glasses frame makes the AR glasses frame appear large and inconvenient to wear; at the same time, the information captured by the camera is relatively large, requiring a particularly complex processing system for processing, which places high demands on the processing system. Utility Model Content

[0005] The purpose of this invention is to provide an eye-tracking structure and glasses for AR technology, in order to solve the technical problem that the existing AR glasses frame is relatively bulky and inconvenient to wear because it has a bulky image receiving device on it.

[0006] The technical problem to be solved by this utility model can be achieved through the following technical solution:

[0007] An eye-tracking structure for AR technology includes a visual tracking mechanism mounted on a head-mounted device. The visual tracking mechanism includes a side-emitting optical fiber for collecting the eye's gaze position and a signal collection element for receiving the light signal transmitted by the side-emitting optical fiber. The signal collection element is electrically connected to a control element.

[0008] As a further embodiment of this utility model: the signal collection device includes a complementary metal-oxide-semiconductor device, the side-emitting optical fiber is connected to the output end of the complementary metal-oxide-semiconductor device, and the complementary metal-oxide-semiconductor device is electrically connected to the control device.

[0009] As a further embodiment of this invention, a focusing lens for focusing the light beam is fixedly disposed at the end of the side-emitting optical fiber away from the complementary metal-oxide-semiconductor device.

[0010] As a further embodiment of this utility model: at least one set of the side-emitting optical fibers is provided, and the at least one set of the side-emitting optical fibers is used to collect eye information, wherein the eye information includes the eye's gaze position.

[0011] A pair of glasses with the aforementioned eye-tracking structure for AR technology includes a frame, imaging lenses symmetrically fixed on both sides of the frame, a projection component for projecting onto the imaging lenses on the inner side of the frame, wearing components for easy wearing by the user on both sides of the frame, a complementary metal-oxide-semiconductor device disposed on the wearing components, and a focusing lens disposed on the inner side of the frame.

[0012] As a further embodiment of this utility model: the projection component includes a liquid crystal light modulator fixedly disposed on the upper side of the frame, and a reflective lens that cooperates with the liquid crystal light modulator is fixedly disposed on the lower side of the frame. The light beam output by the liquid crystal light modulator is transmitted to the imaging lens through the reflective lens, and is reflected or diffracted into the eyeball by the imaging lens.

[0013] As a further embodiment of this invention: the wearing component includes feet fixedly disposed on both sides of the frame, and the complementary metal-oxide-semiconductor device is fixed on the feet.

[0014] As a further embodiment of this utility model: vertically arranged light-shielding prisms are provided on both sides of the frame, and the focusing lens is located inside the light-shielding prisms.

[0015] As a further embodiment of this utility model: a data line is provided on the support leg, and the liquid crystal light modulator and the complementary metal oxide semiconductor device are electrically connected to the control component through the data line.

[0016] As a further embodiment of this utility model: a nose pad is provided at the bottom of the eyeglass frame, and the eyeglass frame, nose pad, light-shielding prism, and reflective lens form a space that is relatively isolated from the outside world.

[0017] The beneficial effects of this utility model are:

[0018] 1. When in use, this utility model can capture speckle patterns through a focusing lens and transmit the speckle patterns to a complementary metal-oxide-semiconductor (CMOS) device through a side-emitting optical fiber. The CMOS device then transmits the speckle patterns to the control unit. Compared with the traditional method of using a camera to capture images of the eye, it requires less data processing, has lower hardware requirements, and does not require installing more image acquisition devices on the head-mounted device, thus reducing the overall size and weight and improving the wearer's experience.

[0019] 2. In use, the complementary metal-oxide-semiconductor device transmits the received signal to the controller, which then compares the speckle pattern to determine the state of the eye. The device then prioritizes rendering objects near the user's gaze point, reducing scene rendering time. Attached Figure Description

[0020] The present invention will be further described below with reference to the accompanying drawings.

[0021] Figure 1 This is a schematic diagram of the overall structure of this utility model;

[0022] Figure 2 This is a schematic diagram of the focusing lens structure of this utility model;

[0023] Figure 3 This is a schematic diagram of the structure of this utility model after configuring the data cable;

[0024] Figure 4 This is a schematic diagram of the beam propagation path of this utility model;

[0025] Figure 5 This is a schematic diagram of the visual tracking mechanism of this utility model.

[0026] In the diagram: 101, frame; 102, foot; 103, liquid crystal light modulator; 104, reflecting lens; 105, imaging lens; 106, data cable; 107, light-shielding prism; 2, visual tracking mechanism; 201, complementary metal-oxide-semiconductor device; 202, side-emitting fiber; 203, focusing lens; 3, eyeball. Detailed Implementation

[0027] The technical solutions of the present utility model will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present utility model, and not all embodiments. Based on the embodiments of the present utility model, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present utility model.

[0028] like Figures 1-5 As shown, an eye-tracking structure for AR technology includes a visual tracking mechanism 2 mounted on a head-mounted device. The visual tracking mechanism 2 includes a side-emitting optical fiber 202 for collecting the eye's gaze position and a signal collection device for receiving the light signal transmitted by the side-emitting optical fiber 202. The signal collection device may be a complementary metal-oxide-semiconductor device 201 (i.e., CMOS, a photosensitive component). The side-emitting optical fiber 202 is connected to the output end of the complementary metal-oxide-semiconductor device 201. The signal collection device is electrically connected to a control device.

[0029] The control unit contains a function that functions as a data relationship management system, which is existing technology. Before use, a white light source, a mixture of red, green, and blue lasers, is modulated by a spatial light modulator (SLM) to create a display image. The SLM can display individual bright spots at different locations, using a pixel-by-pixel traversal method. When the eye 3 focuses on different pixels, the reflected light field of the eye 3 will also be different. At this time, the light field collected by the side-emitting fiber 202 will also be different, and the speckle pattern received by the camera will also be different. The speckle patterns from different focusing positions are collected and used to create training and testing sets. The pixel positions are used as labels, thus completing the system recording. By determining the relationship between each speckle pattern and the corresponding pupil deflection, the system can determine the wearer's pupil focusing direction during use. The above function is existing technology and is not a point of protection for this patent.

[0030] Additionally, it's necessary to understand the existing technology regarding optical fibers, including: different reflected light fields from the eye excite different modes upon entering the optical fiber, and these excitation modes have different amplitude and phase distributions. This results in a speckle pattern at the output end of the fiber after transmission. The complex amplitude ψ(x, y) of the speckle pattern at the fiber output end is represented as...

[0031]

[0032] Where a m (x,y) and φ m (x,y) represents the amplitude and phase distribution of the m-th mode in the xy-plane, where the xy-plane is the fiber endface. The intensity I(x,y) of the speckle pattern can be expressed as...

[0033]

[0034] The intensity distribution of the speckle pattern received by the camera is the sum of the intensities of the M propagation modes in the optical fiber.

[0035] Head-worn equipment may include glasses, helmets, or other equipment worn on the head that can collect information from the eyes.

[0036] The complementary metal-oxide-semiconductor device 201 is electrically connected to the control unit. The complementary metal-oxide-semiconductor device 201 can transmit the collected beam signal to the control unit, thereby facilitating the control unit to judge the detection information.

[0037] In this embodiment, a focusing lens 203 for focusing the light beam is fixedly disposed at the end of the side-emitting optical fiber 202 away from the complementary metal-oxide-semiconductor device 201. The focusing lens 203 can focus the light, thereby expanding the information collection range and improving the detection accuracy.

[0038] In one specific embodiment, a set of side-emitting optical fibers 202 is provided, and a set of eyeball 3 information is collected through the set of side-emitting optical fibers 202, wherein the eyeball 3 information includes the position of eye gaze, i.e. speckle pattern.

[0039] In another specific embodiment, two sets of side-emitting optical fibers 202 are provided. These two sets of side-emitting optical fibers 202 respectively collect information from two sets of eyeballs 3, thereby providing more accurate information. It should be noted that the complementary metal-oxide-semiconductor device 201 can simultaneously receive information from both sets of side-emitting optical fibers 202. The complementary metal-oxide-semiconductor device 201 uniformly transmits the collected information to the control unit for learning and training, recording the relationship between the speckle pattern and the observation position. Subsequently, during wear, the trained data can be compared to determine the wearer's observation direction.

[0040] This invention also provides glasses with the aforementioned eye-tracking structure for AR technology, including a frame 101. Imaging lenses 105 are symmetrically fixed on both sides of the frame 101. A projection assembly for projecting onto the imaging lenses 105 is disposed on the inner side of the frame 101. The projection assembly includes a liquid crystal light modulator 103 (i.e., SLM) fixedly disposed on the upper side of the frame 101, and a reflective lens 104 cooperating with the liquid crystal light modulator 103 is fixedly disposed on the lower side of the frame 101. Figure 4 As shown, the light beam output by the liquid crystal light modulator 103 is transmitted to the imaging lens 105 through the reflecting lens 104, and is reflected or diffracted into the eyeball 3 by the imaging lens 105.

[0041] The frame 101 has wearing components on both sides for easy wearing by the user. The wearing components include feet 102 fixedly mounted on both sides of the frame 101, and complementary metal oxide semiconductor devices 201 fixed on the feet 102. By mounting the complementary metal oxide semiconductor devices 201 that receive speckle patterns on the feet 102, the installation space on the frame 101 can be saved. At the same time, compared with the traditional method of using a camera to collect photos of the eyeball 3, less data needs to be processed and the requirements for hardware configuration are lower.

[0042] The frame 101 has vertically arranged light-shielding prisms 107 on both sides, and the focusing lens 203 is located inside the light-shielding prisms 107, so that the focusing lens 203 can receive the light beam information reflected from the eyeball 3. The frame 101 has a nose pad at the bottom. The frame 101, nose pad, light-shielding prisms 107, and reflecting lens 104 form a space that is relatively isolated from the outside world, thereby maintaining the light-sensing environment of the eye and reducing the impact of external light beams on the wearer.

[0043] In one specific embodiment, a data line 106 is provided on the support leg 102. The liquid crystal light modulator 103 and the complementary metal oxide semiconductor device 201 are electrically connected to the control device through the data line 106. The control device is located on the outside of the frame 101. By being linearly connected to the control device, the weight and volume of the glasses 1 itself are reduced, thereby improving the wearer's experience.

[0044] To facilitate understanding of the embodiments of this solution by those skilled in the art, the working principle of the embodiments of this solution will now be explained in conjunction with specific application scenarios:

[0045] In use, the glasses 1 frame 101 is worn on the eyes. The light beam output by the liquid crystal light modulator 103 is transmitted to the imaging lens 105 through the reflecting lens 104, and is reflected or diffracted into the eyeball 3 through the imaging lens 105, thus realizing the transmission of information. When the person's eyeball 3 moves, the focusing lens 203 captures the speckle pattern and transmits the captured light signal to the side-emitting fiber 202. The side-emitting fiber 202 transmits the signal to the complementary metal-oxide-semiconductor device 201. The complementary metal-oxide-semiconductor device 201 transmits the received signal to the controller. The controller compares the speckle pattern to determine the state of the eyeball 3, and then allows the device to prioritize rendering objects near the user's gaze point, reducing scene rendering time.

[0046] The above description provides a detailed account of one embodiment of the present invention. However, this description is merely a preferred embodiment and should not be construed as limiting the scope of the present invention. All equivalent variations and improvements made within the scope of the claims of the present invention should still fall within the patent coverage of the present invention.

Claims

1. An eye tracking structure for use in AR technology, characterized by The application relates to a visual tracking mechanism (2) arranged on a head-mounted device, the visual tracking mechanism (2) comprising a side-emitting optical fiber (202) for collecting an eye gaze position and a signal collecting unit for receiving an optical signal transmitted by the side-emitting optical fiber (202), and the signal collecting unit is electrically connected with a control unit.

2. The eye tracking structure for use in AR technology according to claim 1, characterized in that, The signal collecting unit comprises a complementary metal oxide semiconductor device (201), the side-emitting optical fiber (202) is connected with an output end of the complementary metal oxide semiconductor device (201), and the complementary metal oxide semiconductor device (201) is electrically connected with the control unit.

3. The eye tracking structure for use in AR technology according to claim 2, characterized in that, One end of the side-emitting optical fiber (202) away from the complementary metal oxide semiconductor device (201) is fixedly provided with a focusing lens (203) for gathering a light beam.

4. The eye tracking structure for use in AR technology according to claim 3, characterized in that, At least one group of the side-emitting optical fibers (202) is arranged for collecting eye (3) information, wherein the eye (3) information comprises an eye gaze position.

5. Glasses comprising the eye tracking structure according to claim 3, characterized in that The application further relates to a frame (101), the frame (101) is symmetrically fixed with imaging lenses (105) on both sides, the frame (101) is internally provided with a projection assembly for projecting on the imaging lenses (105), the frame (101) is provided with wearing assemblies on both sides for facilitating a user to wear, the complementary metal oxide semiconductor device (201) is arranged on the wearing assemblies, and the focusing lens (203) is arranged on the inner side of the frame (101).

6. The eyeglasses of claim 5, wherein, The projection assembly comprises a liquid crystal light modulator (103) fixed on the upper side of the frame (101), the frame (101) is fixed with a reflecting lens (104) matched with the liquid crystal light modulator (103) on the lower side, a light beam output by the liquid crystal light modulator (103) is transmitted to the imaging lenses (105) through the reflecting lens (104), and the light beam is reflected or diffracted into the eye (3) through the imaging lenses (105).

7. The eyeglasses of claim 6, wherein, The wearing assemblies comprise supporting legs (102) fixed on both sides of the frame (101), and the complementary metal oxide semiconductor device (201) is fixed on the supporting legs (102).

8. The eyeglasses of claim 7, wherein, The frame (101) is provided with vertically arranged light shielding rib plates (107) on both sides, and the focusing lens (203) is located on the inner side of the light shielding rib plates (107).

9. The eyeglasses of claim 7, wherein, The supporting legs (102) are provided with data lines (106), and the liquid crystal light modulator (103) and the complementary metal oxide semiconductor device (201) are electrically connected with the control unit through the data lines (106).

10. The eyeglasses of claim 8, wherein, The frame (101) is provided with a nose pad at the bottom, and the frame (101), the nose pad, the light shielding rib plates (107) and the reflecting lens (104) form a space isolated from the outside.

Images