Eye movement tracking system and wearing device

Abstract

The embodiment of the invention discloses an eye movement tracking system and wearing equipment, a light source module is used for irradiating a first eye and a second eye, and a first light guide module and a second light guide module which form a certain angle are used for receiving respectively. Therefore, the first light guide module, the second light guide module and the light combination component form a first light path and a second light path respectively, so that the same camera can collect the first reflected light and the second reflected light in different directions at the same time or in a time-sharing manner, the manufacturing cost of the eye movement tracking system is simplified, and the calibration procedure in a dual-camera system is omitted. And on the other hand, the first reflected light and the second reflected light pass through the first light path and the second light path and are emitted into the light sensing area of the same camera, so that the image processing module can estimate the fixation point information according to the difference and the relative position change of the first image information and the second image information.

Description

Technical Field

[0001] The present invention relates to the field of eye tracking technology, and in particular to an eye tracking system and a wearable device. Background Art

[0002] Eye tracking is a core technology that captures and analyzes user eye movements, enabling visual perception and human-computer interaction. Based on optical imaging, computer vision algorithms, and artificial intelligence models, it detects pupil position, corneal reflection, or eye movement characteristics to calculate the user's gaze point in real time and map it to the interactive scene. This technology has evolved from an early psychological research tool to a key functional module in wearable devices. Current mainstream solutions rely on infrared optical tracking or RGB camera-based AI algorithms, but these solutions often suffer from complex structures, high costs, and difficult architecture implementation.

[0003] Taking smart glasses as an example, a dual-camera solution is commonly used to achieve eye tracking, that is, a separate camera is configured for each eye. Although this design can ensure the independence of binocular data, it also brings significant limitations: hardware redundancy, increased costs, and industrial design restrictions. Specifically, each eye requires at least one camera, which doubles the hardware cost and requires additional circuit board space and power management modules. The dual-camera system requires complex synchronous calibration processes (such as timestamp alignment and view angle calibration), which increases production complexity and reduces the risk of yield reduction. How to reduce the manufacturing cost of eye tracking systems and reduce the difficulty of configuration has become a problem that needs to be solved. Summary of the Invention

[0004] In view of this, an embodiment of the present invention provides an eye tracking system and a wearable device, which utilizes an image acquisition module to respectively capture the first reflected light and the second reflected light of the first eye and the second eye, so as to reduce the manufacturing cost of the eye tracking system and reduce the configuration difficulty.

[0005] According to a first aspect of an embodiment of the present invention, an eye tracking system is provided, the eye tracking system comprising:

[0006] a first light guide module;

[0007] a second light guide module;

[0008] a light source module, irradiating the first eye and the second eye to form a first reflected light and a second reflected light respectively;

[0009] An image acquisition module, comprising a light combining component and a camera, wherein the first light guide module and the second light guide module are arranged at a preset angle corresponding to the first eye and the second eye, respectively; the camera comprises a photosensitive area, the first light guide module and the light combining component form a first light path, and the second light guide module and the light combining component form a second light path, the photosensitive area receives the first reflected light through the first light path and generates first image information, and the photosensitive area receives the second reflected light through the second light path and generates second image information; and

[0010] The image processing module is configured to determine gaze point information based on the first image information and the second image information.

[0011] Furthermore, the light source module includes at least one infrared lamp;

[0012] The camera is an infrared camera.

[0013] Furthermore, the light source module further includes a light splitting component, and the light splitting component includes an incident surface, a first irradiation surface, and a second irradiation surface;

[0014] The number of the infrared lamp is one, and the infrared light of the infrared lamp passes through the incident surface and is emitted toward the first eye and the second eye from the first irradiation surface and the second irradiation surface respectively.

[0015] Furthermore, the number of the infrared lamps is multiple, and the multiple infrared lamps are arranged at intervals to form a first lamp group and a second lamp group. The multiple infrared lamps of the first lamp group and the multiple infrared lamps of the second lamp group are symmetrically arranged relative to the light combining component, and the first lamp group forms multiple light spots on the first eye, and the second lamp group forms multiple light spots on the second eye;

[0016] The first image information and the second image information both include pupil position information and light spot position information;

[0017] The image processing module is further configured to determine the rotation direction of the first eye and the second eye according to the relative position change of the pupil position information and the light spot position information.

[0018] Furthermore, the image acquisition module is configured such that the first light guide module receives a first reflected light toward the first eye, and the second light guide module receives a second reflected light toward the second eye.

[0019] Furthermore, the eye tracking system further includes:

[0020] Two coupling gratings are respectively arranged corresponding to the first eye and the second eye;

[0021] Two waveguide plates are respectively arranged corresponding to the two coupling-in gratings, and the waveguide plates are arranged on a side of the two coupling-in gratings close to the infrared lamp, and the waveguide plates have a coupling-out surface;

[0022] The first light guide module and the second light guide module are located between the two outcoupling surfaces, and the first light guide module faces one outcoupling surface, and the second light guide module faces the other outcoupling surface. The first reflected light and the second reflected light are incident on the corresponding waveguide plates. The first reflected light is reflected by the incoupling grating and is incident on the first light guide module from the outcoupling surface. The second reflected light is reflected by the incoupling grating and is incident on the second light guide module from the outcoupling surface.

[0023] Furthermore, the photosensitive area includes a first photosensitive area and a second photosensitive area, the first photosensitive area receives the first reflected light, and the second photosensitive area receives the second reflected light.

[0024] Furthermore, the image acquisition module further includes a shielding body, and the image acquisition module is configured to drive the shielding body to be movably disposed in the first optical path or the second optical path;

[0025] The image processing module is configured to collect the first image information through the camera in response to the blocking body blocking the second light path; and collect the second image information through the camera in response to the blocking body blocking the first light path.

[0026] In a second aspect, an embodiment of the present invention further provides a wearable device, the wearable device comprising:

[0027] A glasses frame comprising a nose bridge and two glasses frames, wherein the nose bridge is connected between the two glasses frames; and

[0028] An eye tracking system includes two waveguides, a light source module, an image acquisition module, and an image processing module, wherein the two waveguides are respectively fixed to the two frames, the light source module irradiates the first eye and the second eye to form a first reflected light and a second reflected light, respectively, the image acquisition module is arranged on the bridge of the nose and includes a first light guide module, a second light guide module, a light combining component, and a camera, the first light guide module and the second light guide module are arranged at a preset angle corresponding to the first eye and the second eye, respectively, the camera includes a photosensitive area, the photosensitive area faces the light combining component, the first light guide module and the light combining component form a first light path, the second light guide module and the light combining component form a second light path, the photosensitive area receives the first reflected light through the first light path and generates first image information, and the photosensitive area receives the second reflected light through the second light path and generates first image information;

[0029] The image processing module is configured to determine gaze point information based on the first image information and the second image information.

[0030] Furthermore, the camera is an infrared camera;

[0031] The light source module is arranged on the nose bridge and includes an infrared lamp and a light splitting component, and the light splitting component includes an incident surface, a first irradiation surface and a second irradiation surface;

[0032] The infrared light of the infrared lamp passes through the incident surface and is emitted toward the first eye and the second eye from the first irradiation surface and the second irradiation surface respectively.

[0033] Furthermore, the camera is an infrared camera;

[0034] The light source module includes a plurality of infrared lamps, which are arranged at intervals to form a first lamp group and a second lamp group. The plurality of infrared lamps in the first lamp group and the plurality of infrared lamps in the second lamp group are symmetrically arranged relative to the light combining component. The first lamp group is arranged on one of the frames, and the second lamp group is arranged on the other frame. The first lamp group forms a plurality of light spots on the first eye, and the second lamp group forms a plurality of light spots on the second eye.

[0035] The first image information and the second image information both include pupil position information and light spot position information;

[0036] The image processing module is further configured to determine the rotation direction of the first eye and the second eye according to the relative position change of the pupil position information and the light spot position information.

[0037] Furthermore, the eye tracking system further includes:

[0038] Two coupling-in gratings are respectively arranged corresponding to the two waveguide plates, and the waveguide plates are arranged on a side of the two coupling-in gratings close to the first eye portion, and the waveguide plates have a coupling-out surface;

[0039] The first light guide module and the second light guide module are located between the two outcoupling surfaces, and the first light guide module faces one outcoupling surface, and the second light guide module faces the other outcoupling surface. The first reflected light and the second reflected light are incident on the corresponding waveguide plates. The first reflected light is reflected by the incoupling grating and is incident on the first light guide module from the outcoupling surface. The second reflected light is reflected by the incoupling grating and is incident on the second light guide module from the outcoupling surface.

[0040] Furthermore, the image acquisition module is configured such that the first light guide module receives a first reflected light toward the first eye, and the second light guide module receives a second reflected light toward the second eye.

[0041] The eye tracking system and wearable device of the embodiment of the present invention utilize a light source module to illuminate the first eye and the second eye, and utilize a first light guide module and a second light guide module at a certain angle to receive them respectively. Thus, the first light guide module and the second light guide module and the light combining component respectively form a first light path and a second light path, so that the same camera can simultaneously or time-share the first reflected light and the second reflected light in different directions, simplifying the manufacturing cost of the eye tracking system and saving the calibration process in the dual-camera system. On the other hand, the first reflected light and the second reflected light pass through the first light path and the second light path and are incident on the photosensitive area of ​​the same camera, which helps the image processing module to estimate the gaze point information based on the difference and relative position change of the first image information and the second image information. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0043] Figure 1 is a structural diagram of one side of a wearable device provided by the first embodiment of the present invention;

[0044] Figure 2 is a structural diagram of the other side of the wearable device provided by the first embodiment of the present invention;

[0045] Figure 3 is a schematic structural diagram of a wearable device provided by a second embodiment of the present invention;

[0046] Figure 4 is a schematic structural diagram of a wearable device provided by a third embodiment of the present invention;

[0047] Figure 5 is a schematic diagram of the architecture of an eye tracking system according to an embodiment of the present invention;

[0048] Figure 6 is a structural diagram of a light source module according to an embodiment of the present invention;

[0049] Figure 7 Schematic diagram of the architecture of the image acquisition module, the first light guide module, and the second light guide module according to an embodiment of the present invention;

[0050] Figure 8 1 is a schematic structural diagram of an image acquisition module according to an embodiment of the present invention;

[0051] Figure 9is a schematic structural diagram of a photosensitive area in some implementations of an embodiment of the present invention;

[0052] Figure 10 is a schematic structural diagram of the photosensitive area of ​​an embodiment of the present invention in other implementations;

[0053] Figure 11 Schematic diagram of the position of the light spot according to an embodiment of the present invention.

[0054] Description of reference numerals:

[0055] 1-Image acquisition module;

[0056] 11-first light guide module; 12-second light guide module; 13-light combining component; 131-first incident surface; 132-second incident surface; 133-exit surface; 14-camera; 141-photosensitive area; 1411-first photosensitive area; 1412-second photosensitive area; 15-shielding body; 16-photosensitive chip;

[0057] 2-light source module;

[0058] 21 - infrared lamp; 22 - light splitting component; 221 - first irradiation surface; 222 - second irradiation surface; 223 - incident surface; 231 - first lamp group; 232 - second lamp group;

[0059] 3-Image processing module;

[0060] 41-incoupling grating; 42-waveguide plate; 421-outcoupling surface;

[0061] 5-frame;

[0062] 51-nose bridge; 52-glass frame;

[0063] 61-first optical path; 62-second optical path;

[0064] A-first eye; B-second eye; C-light spot; D-pupil. DETAILED DESCRIPTION

[0065] The present invention is described below based on the following embodiments, but the present invention is not limited to these embodiments. In the detailed description of the present invention below, certain specific details are described in detail. Those skilled in the art can fully understand the present invention without these details. To avoid obscuring the essence of the present invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0066] Furthermore, persons of ordinary skill in the art will appreciate that the figures provided herein are for illustration purposes only and are not necessarily drawn to scale.

[0067] Unless the context clearly requires otherwise, words like “include”, “comprising” and the like throughout this application should be interpreted as including rather than exclusive or exhaustive; that is, as meaning “including but not limited to”.

[0068] In the description of the present invention, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood to indicate or imply relative importance. In addition, in the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0069] Unless otherwise specified or limited, the terms "mounted," "connected," "connect," "fixed," and the like should be interpreted broadly. For example, they may refer to fixed connection, detachable connection, or integration; they may refer to direct connection or indirect connection through an intermediate medium; they may refer to internal communication between two components or interaction between two components, unless otherwise specified. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0070] For ease of explanation, spatially relative terms such as "in," "out," "under," "below," "lower," "above," "upper," and the like are used herein to describe the relationship of one element or feature illustrated in the figures to another element or feature. It will be understood that spatially relative terms may be intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over, an element described as being "under" or "beneath" another element or feature would then be positioned "above" the other element or feature. Thus, the example term "under" can encompass both above and below orientations. The device may be oriented in other ways (rotated 90 degrees or in other orientations), and the spatially relative descriptors used herein should be interpreted accordingly.

[0071] Figure 1 and Figure 2 2 is a schematic structural diagram of the wearable device provided in the first embodiment. Figure 3 2 is a schematic diagram of the structure of the wearable device provided by the second embodiment. Figure 4 : is a schematic diagram of the structure of the wearable device provided by the third embodiment. Figure 2 and Figure 4 This is a top view of the wearable device. Figure 1 、 Figure 2 、 Figure 4 The light path of the mid-infrared light is shown by a thick dotted line, and the light paths of the first reflected light and the second reflected light received by the image acquisition module 1 are shown by thin dotted lines.

[0072] In some embodiments, as Figures 1-4As shown, the wearable device in this embodiment can be smart glasses, which can be used to realize functions such as music playback, video playback, and virtual reality. The wearable device includes a frame 5 and an eye tracking system provided on the frame 5. The frame 5 includes a nose bridge 51, two frames 52, and two temples (not shown in the figure), and the two temples are respectively fixed to the two frames 52. The frame 52 is used to fix two waveguides 42. The waveguides 42 can be used to transmit reflected light from the first eye A and the second eye B. The waveguides 42 can also cooperate with the optical display system to display AR or VR images.

[0073] Figure 5 Schematic diagram of the eye tracking system of this embodiment. Figure 6 2 is a schematic structural diagram of the light source module 2 of this embodiment. Figure 7 1 is a schematic diagram of the structure of the image acquisition module 1 , the first light guide module 11 , and the second light guide module 12 of this embodiment. Figure 8 Schematic diagram of the structure of the image acquisition module 1 of this embodiment.

[0074] In some embodiments, as Figure 5 As shown, the eye tracking system includes an image acquisition module 1, a first light guide module 11, a second light guide module 12, a light source module 2 and an image processing module 3. Figure 6 As shown, the light source module 2 is used to illuminate the first eye A and the second eye B to form a first reflected light and a second reflected light respectively. Figure 7 and Figure 8 As shown, the image acquisition module 1 includes a light combining component 13 and a camera 14. The first light guide module 11 and the second light guide module 12 are at a preset angle and are respectively arranged corresponding to the first eye A and the second eye B, and the camera 14 includes a photosensitive area 141. Specifically, the camera 14 includes a photosensitive chip 16, which is communicatively connected to the image processing module 3, and the photosensitive chip 16 includes a photosensitive area 141. The first light guide module 11 and the light combining component 13 form a first light path 61, and the second light guide module 12 and the light combining component 13 form a second light path 62. The photosensitive area 141 receives the first reflected light through the first light path 61 and generates the first image information, and the photosensitive area 141 receives the second reflected light through the second light path 62 and generates the second image information.

[0075] The photosensitive area 141 in this embodiment faces the light combining component 13, and the camera 14 is used to convert the light signal received by the photosensitive area 141 into an image signal to send to the image processing module 3. The setting angle of the first light guide module 11 and the second light guide module 12 can be adjusted according to the distance between the first eye A and the second eye B, or according to the distance between the two waveguide plates 42. At the same time, the light combining component 13 can also be adjusted according to the setting angle of the first light guide module 11 and the second light guide module 12 to control the direction of the light path. The photosensitive chip 16 includes but is not limited to a CMOS image sensor, a CCD image sensor or an infrared camera. The image processing module 3 is configured to determine the gaze point information based on the first image information and the second image information.

[0076] Figure 9 FIG. 1 is a schematic diagram of the structure of the photosensitive area 141 of this embodiment in some implementations. In the figure, the first light path 61 and the second light path 62 project an image onto the same photosensitive area 141 .

[0077] Easy to understand, such as Figure 8-Figure 9 As shown, in this embodiment, the image acquisition module 1 is arranged between the first eye A and the second eye B, so that the first light guide module 11 and the second light guide module 12 respectively shoot the first eye A and the second eye B from different directions. At the same time, the light combining component 13 also makes the positions of the first light path 61 and the second light path 62 projected onto the photosensitive chip 16 have certain differences. As a result, the first image information and the second image information have differences. The image processing module 3 can obtain the user's gaze point information based on the line of sight angle of the pupil D of the first eye A and the second eye B, and by using computer vision algorithms and artificial intelligence models. In addition, when the human eye observes objects at a distance and at a close distance, the visual axis angle will also change. The closer the distance, the larger the visual axis angle, that is, the visual axes of the first eye A and the second eye B will move closer to the nose. The image processing module 3 can obtain the user's gaze point information based on the distance change of the pupil D in the first image information and the second image information (such as Figure 9 The distance L1 in the figure is shown), which can further determine the changes in the distance observed by the wearer.

[0078] Optionally, the photosensitive area 141 simultaneously captures the first image information and the second image information in an overlapping state. The image processing module 3 uses a computer vision algorithm or an artificial intelligence model to parse the first image information and the second image information from the overlapping images.

[0079] In summary, the eye tracking system in this embodiment uses the light source module 2 to illuminate the first eye A and the second eye B, and uses the first light guide module 11 and the second light guide module 12 at a certain angle to receive them respectively. As a result, the first light guide module 11 and the second light guide module 12 and the light combining component 13 form a first light path 61 and a second light path 62 respectively, so that the same camera 14 can collect the first reflected light and the second reflected light in different directions simultaneously or in a time-sharing manner, which simplifies the manufacturing cost of the eye tracking system and saves the calibration process in the dual-camera system. On the other hand, the first reflected light and the second reflected light pass through the first light path 61 and the second light path 62 and are incident on the photosensitive area 141 of the same camera 14, which helps the image processing module 3 estimate the gaze point information based on the difference and relative position change of the first image information and the second image information.

[0080] In some embodiments, as Figures 1-4 As shown, the light source module 2 includes at least one infrared lamp 21. The camera 14 is an infrared camera. That is, the photosensitive chip 16 is configured as an infrared photosensitive chip. The infrared camera in this embodiment is used to receive the infrared light generated by the infrared lamp 21. The power and wavelength of the infrared light of the infrared lamp 21 are configured to be within a safe range to prevent eye damage during use. Furthermore, the frequency of the infrared light is in the non-visible light range, which also prevents any impact on user experience.

[0081] In some embodiments, as Figure 4 and Figure 6 As shown, the light source module 2 further includes a light splitting component 22, which includes an incident surface 223, a first irradiation surface 221, and a second irradiation surface 222. There is only one infrared lamp 21, and the infrared light of the infrared lamp 21 passes through the incident surface 223 and is emitted from the first irradiation surface 221 and the second irradiation surface 222 to the first eye A and the second eye B, respectively.

[0082] Specifically, the light source module 2 in this embodiment is provided with only one infrared lamp 21, which is placed between the two eyes. The light splitting component 22 can split a beam of infrared light into two beams. This further reduces the manufacturing cost of the light source module 2 and also reduces the power consumption of the light source module 2.

[0083] Optionally, the light splitting component 22 can be a beam splitter prism or a flat beam splitter. The flat beam splitter is an optical element that divides one path of light into two paths in proportion through an optical film layer. Taking a cubic beam splitter as an example, Figure 6As shown, the cubic beamsplitter comprises a first right-angle prism, a second right-angle prism, and a beamsplitter coating. Each of the first and second right-angle prisms comprises two right-angled surfaces and an inclined surface. The two inclined surfaces are bonded together, with the beamsplitter coating disposed therebetween. One inclined surface of the first right-angle prism forms an incident surface 223, while the other inclined surface forms a first irradiation surface 221. The inclined surface of the second right-angle prism, facing away from the incident surface 223, forms a second irradiation surface 222. The angle between the first irradiation surface 221 and the second irradiation surface 222 is 90 degrees.

[0084] In addition, in order to ensure that the light from the first irradiation surface 221 and the second irradiation surface 222 can accurately illuminate the first eye A and the second eye B, a reflector can be set in front of the first irradiation surface 221 and the second irradiation surface 222 so that the reflected light emitted by the irradiation surface is projected onto the eyes through the reflector.

[0085] Figure 11 Schematic diagram of the position of the light spot C in this embodiment. State I in the figure shows the position of the light spot C when the first eye A is looking straight ahead, and state II shows the position of the light spot C when the first eye A is turned toward the lower right.

[0086] In some embodiments, as Figure 3 As shown, there are multiple infrared lamps 21. Multiple infrared lamps 21 are arranged at intervals and form a first lamp group 231 and a second lamp group 232. The multiple infrared lamps 21 of the first lamp group 231 and the multiple infrared lamps 21 of the second lamp group 232 are symmetrically arranged relative to the light combining component 13. Figure 11 As shown, the first light group 231 forms a plurality of light spots C on the first eye portion A, and the second light group 232 forms a plurality of light spots C on the second eye portion B.

[0087] In this embodiment, both the first and second image information include pupil position information and light spot position information. The image processing module 3 is further configured to determine the rotational direction of the first eye A and the second eye B based on the relative positional changes between the pupil position information and the light spot position information. In this embodiment, the image processing module 3 not only determines the gaze point information based on the relative positional changes between the first eye A and the second eye B, but also further determines the eyeball's rotational direction based on the relative positional changes between the pupil D and the light spot C, making the image processing module 3's determination of the gaze point information more accurate.

[0088] In some embodiments, as Figure 4 As shown, the image acquisition module 1 is configured such that the first light guide module 11 receives the first reflected light from the first eye A, and the second light guide module 12 receives the second reflected light from the second eye B. In this embodiment, the image processing module 3 directly receives the reflected light from the first eye A and the second eye B, which can simplify the light transmission path.

[0089] In some embodiments, as Figure 1-Figure 2 As shown, the eye tracking system also includes two coupling gratings 41 and two waveguides 42. The two coupling gratings 41 are respectively arranged corresponding to the first eye A and the second eye B. The two waveguides 42 are respectively arranged corresponding to the two coupling gratings 41, and the waveguides 42 are arranged on the side of the two coupling gratings 41 close to the infrared lamp 21. The waveguides 42 have coupling-out surfaces 421. The first light guide module 11 and the second light guide module 12 are located between the two coupling-out surfaces 421, with the first light guide module 11 facing one coupling-out surface 421 and the second light guide module 12 facing the other coupling-out surface 421. The first reflected light and the second reflected light are incident on the corresponding waveguides 42. The first reflected light is reflected by the coupling grating 41 and is incident on the first light guide module 11 from the coupling-out surface 421. The second reflected light is reflected by the coupling grating 41 and is incident on the second light guide module 12 from the coupling-out surface 421.

[0090] Specifically, the two outcoupling surfaces 421 are tilted toward opposite sides and face the first light guide module 11 and the second light guide module 12, respectively. Taking the first reflected light as an example, after the first reflected light enters the waveguide plate 42, it will reach the coupling-in grating 41, and the first reflected light will be coupled into the waveguide plate 42 at the position of the coupling-in grating 41. The first reflected light will continue to refract within the waveguide plate 42 until it is coupled out from the outcoupling surface 421 and enters the first light guide module 11. In this way, the difficulty of arranging the first light guide module 11 and the second light guide module 12 can be reduced, and the impact of environmental changes on image acquisition by the image acquisition module 1 can be reduced.

[0091] Furthermore, if Figure 8 As shown, the light-combining component 13 can be a light-combining prism or a flat beam splitter. Taking a cubic prism as an example, the cubic prism includes a first right-angle prism, a second right-angle prism and a beam splitting film layer, and the first right-angle prism and the second right-angle prism each include two right-angle surfaces and an inclined surface. The two inclined surfaces are bonded together, and a beam splitting film layer is provided between the two. One inclined surface of the above-mentioned first right-angle prism forms a first incident surface 131 facing the first light guide module 11, and the other inclined surface forms an exit surface 133 facing the photosensitive area 141. The inclined surface of the second right-angle prism on the side away from the photosensitive area 141 forms a second incident surface 132. The second incident surface 132 faces the second light guide module 12. The angle between the first incident surface 131 and the second incident surface 132 can be 90 degrees.

[0092] In addition, in order to ensure that the first reflected light and the second reflected light can accurately enter the first incident surface 131 and the second incident surface 132, a reflector can be provided in front of the first incident surface 131 and the second incident surface 132 so that the light from the first light guide module 11 and the second light guide module 12 can accurately enter the first incident surface 131 and the second incident surface 132.

[0093] Figure 10Schematic diagram of the structure of the photosensitive area 141 of this embodiment in other implementations.

[0094] In some embodiments, as Figure 10 As shown, the photosensitive area 141 includes a first photosensitive area 1411 and a second photosensitive area 1412. The first photosensitive area 1411 is used to receive the first reflected light, and the second photosensitive area 1412 is used to receive the second reflected light. In this embodiment, the first photosensitive area 1411 and the second photosensitive area 1412 are spaced apart. That is, the first optical path 61 and the second optical path 62 correspond to different areas of the camera 14, respectively. This prevents overlap between the first image information and the second image information.

[0095] In some embodiments, as Figure 8 As shown, the image acquisition module 1 further includes a blocking body 15, and the image acquisition module 1 is configured to drive the blocking body 15 to be movably disposed in the first optical path 61 or the second optical path 62. The image processing module 3 is configured to capture first image information through the camera 14 in response to the blocking body 15 blocking the second optical path 62, and to capture second image information through the camera 14 in response to the blocking body 15 blocking the first optical path 61.

[0096] Specifically, the shielding body 15 in this embodiment can be configured to swing along the center point of the cubic prism to achieve the effect of alternately blocking the first light path 61 and the second light path 62. Alternatively, the shielding body 15 can be configured as a flip plate, which is driven by a rotary motor and has a first position and a second position in the rotation direction. When the flip plate is in the first position, it blocks the first light path 61, and when the flip plate is in the second position, it blocks the second light path 62.

[0097] Alternatively, as Figure 6 As shown, the shielding member 15 can also be disposed on the light source module 2 so as to movably block the first irradiation light directed toward the first eye A and the second irradiation light directed toward the second eye B. For example, the shielding member 15 can be movably disposed in front of the first irradiation surface 221 and the second irradiation surface 222 to block infrared light. In this embodiment, the image processing module 3 is configured to capture first image information via the camera 14 in response to the shielding member 15 blocking the second light path 62, and to capture second image information via the camera 14 in response to the shielding member 15 blocking the first light path 61.

[0098] In an optional implementation, if Figures 1-4As shown, the eye tracking system in the above embodiment can be applied to a wearable device. The wearable device also includes a frame 5 and an eye tracking system. The frame 5 includes a nose bridge 51 and two frames 52, and the nose bridge 51 is connected between the two frames 52. The eye tracking system includes two waveguides 42, a light source module 2, an image acquisition module 1 and an image processing module 3. The two waveguides 42 are respectively fixed to the two frames 52. The light source module 2 irradiates the first eye A and the second eye B to form a first reflected light and a second reflected light respectively. The image acquisition module 1 is arranged on the nose bridge 51 and includes a first light guide module 11, a second light guide module 12, a light combining component 13 and a camera 14. The first light guide module 11 and the second light guide module 12 are preset. The camera 14 includes a photosensitive area 141, which faces the light combining component 13. The first light guide module 11 and the light combining component 13 form a first optical path 61, and the second light guide module 12 and the light combining component 13 form a second optical path 62. The photosensitive area 141 receives the first reflected light through the first optical path 61 and generates the first image information. The photosensitive area 141 receives the second reflected light through the second optical path 62 and generates the first image information. The image processing module 3 is configured to determine the gaze point information based on the first image information and the second image information.

[0099] Specifically, in this embodiment, the image acquisition module 1 is disposed on the side of the nose bridge 51 close to the face, so that the image acquisition module 1 is symmetrically arranged relative to the first eye A and the second eye B. This improves the consistency of the first image information and the second image information, facilitating the image processing module 3 to parse the first image information and the second image information.

[0100] In summary, the wearable device in this embodiment utilizes the light source module 2 to illuminate the first eye A and the second eye B, and utilizes the first light guide module 11 and the second light guide module 12 at a certain angle to receive them respectively. Thus, the first light guide module 11 and the second light guide module 12 and the light combining component 13 form the first light path 61 and the second light path 62 respectively, so that the same camera 14 can simultaneously or time-share the first reflected light and the second reflected light in different directions, simplifying the manufacturing cost of the eye tracking system and saving the calibration process in the dual-camera system. On the other hand, the first reflected light and the second reflected light pass through the first light path 61 and the second light path 62 and are incident on the photosensitive area 141 of the same camera 14, which helps the image processing module 3 to estimate the gaze point information based on the difference and relative position change of the first image information and the second image information.

[0101] In some embodiments, as Figure 1-Figure 2As shown, the camera 14 is an infrared camera. The light source module 2 is arranged on the bridge of the nose 51 and includes an infrared lamp 21 and a spectroscopic component 22. The spectroscopic component 22 includes an incident surface 223, a first irradiation surface 221, and a second irradiation surface 222. The infrared light of the infrared lamp 21 passes through the incident surface 223 and is emitted from the first irradiation surface 221 and the second irradiation surface 222 to the first eye A and the second eye B respectively. The light source module 2 in this embodiment is also arranged on the bridge of the nose 51 and is located below or above the image acquisition module 1. The distance between the infrared lamp 21 and the first eye A and the second eye B is the same. The layout cost of the infrared lamp 21 is reduced, and the symmetry of the first image information and the second image information is improved.

[0102] In some embodiments, as Figure 3 and Figure 7-Figure 8 As shown, the camera 14 is an infrared camera. The light source module 2 includes a plurality of infrared lamps 21, which are arranged at intervals and form a first lamp group 231 and a second lamp group 232. The plurality of infrared lamps 21 of the first lamp group 231 and the plurality of infrared lamps 21 of the second lamp group 232 are symmetrically arranged relative to the light combining component 13, and the first lamp group 231 is arranged in one frame 52, and the second lamp group 232 is arranged in another frame 52. Figure 11 As shown, the first light assembly 231 forms multiple light spots C on the first eye A, and the second light assembly 232 forms multiple light spots C on the second eye B. Both the first and second image information include pupil position information and light spot position information. The image processing module 3 is further configured to determine the rotation direction of the first and second eyes A and B based on the relative position changes of the pupil position information and the light spot position information. In this embodiment, the image processing module 3 not only determines the gaze point information based on the relative position changes of the first and second eyes A and B, but also further determines the eyeball's rotation mode based on the relative position changes and relationship between the pupil D and the light spot C, making the image processing module 3's determination of the gaze point information more accurate.

[0103] In some embodiments, as Figure 1-Figure 2As shown, the eye tracking system also includes two coupling gratings 41. The two coupling gratings 41 are respectively arranged corresponding to two waveguide plates 42, and the waveguide plates 42 are arranged on the side of the two coupling gratings 41 close to the infrared lamp 21. The waveguide plates 42 have coupling-out surfaces 421. The first light guide module 11 and the second light guide module 12 are located between the two coupling-out surfaces 421, with the first light guide module 11 facing one coupling-out surface 421 and the second light guide module 12 facing the other coupling-out surface 421. The first reflected light and the second reflected light are incident on the corresponding waveguide plates 42. The first reflected light is reflected by the coupling grating 41 and is incident on the first light guide module 11 from the coupling-out surface 421. The second reflected light is reflected by the coupling grating 41 and is incident on the second light guide module 12 from the coupling-out surface 421. As a result, the difficulty of arranging the first light guide module 11 and the second light guide module 12 can be reduced, and the impact of environmental changes on image acquisition by the image acquisition module 1 can be reduced.

[0104] In some embodiments, as Figure 4 As shown, the image acquisition module 1 is configured such that the first light guide module 11 receives the first reflected light from the first eye A. The second light guide module 12 receives the second reflected light from the second eye B. In this embodiment, the image processing module 3 directly receives the reflected light from the first eye A and the second eye B, which can simplify the light transmission path.

[0105] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

[0106] Where the solutions described in this specification and in the examples involve the processing of personal information, such processing will be conducted with a legitimate basis (e.g., with the consent of the personal information subject or as necessary for the performance of a contract) and only within the prescribed or agreed scope. A user's refusal to process personal information other than that required for basic functions will not affect the user's use of these basic functions.

Claims

1. An eye tracking system, characterized in that: The eye tracking system comprises: A first light guide module (11); A second light guide module (12); A light source module (2) irradiates the first eye (A) and the second eye (B) to form a first reflected light and a second reflected light respectively; An image acquisition module (1) comprises a light combining component (13) and a camera (14); the first light guide module (11) and the second light guide module (12) are arranged at a preset angle and correspond to the first eye (A) and the second eye (B), respectively; the camera (14) comprises a photosensitive area (141); the first light guide module (11) and the light combining component (13) form a first light path (61); the second light guide module (12) and the light combining component (13) form a second light path (62); the photosensitive area (141) receives the first reflected light through the first light path (61) and generates first image information; and the photosensitive area (141) receives the second reflected light through the second light path (62) and generates second image information; and The image processing module (3) is configured to determine gaze point information based on the first image information and the second image information.

2. The eye tracking system according to claim 1, wherein: The light source module (2) includes at least one infrared lamp (21); The camera (14) is an infrared camera (14).

3. The eye tracking system according to claim 2, wherein: The light source module (2) further comprises a light splitting component (22), and the light splitting component (22) comprises an incident surface (223), a first irradiation surface (221), and a second irradiation surface (222); The number of the infrared lamp (21) is one, and the infrared light of the infrared lamp (21) passes through the incident surface (223) and is emitted toward the first eye (A) and the second eye (B) from the first irradiation surface (221) and the second irradiation surface (222), respectively.

4. The eye tracking system according to claim 2, wherein: The number of the infrared lamps (21) is plural, and the plural infrared lamps (21) are arranged at intervals to form a first lamp group (231) and a second lamp group (232); the plural infrared lamps (21) of the first lamp group (231) and the plural infrared lamps (21) of the second lamp group (232) are symmetrically arranged relative to the light combining component (13); the first lamp group (231) forms a plurality of light spots (C) on the first eye (A), and the second lamp group (232) forms a plurality of light spots (C) on the second eye (B); The first image information and the second image information both include pupil position information and light spot position information; The image processing module (3) is further configured to determine the rotation direction of the first eye (A) and the second eye (B) according to the relative position change of the pupil position information and the light spot position information.

5. The eye tracking system according to claim 1, wherein: The image acquisition module (1) is configured such that the first light guide module (11) receives a first reflected light toward the first eye (A), and the second light guide module (12) receives a second reflected light toward the second eye (B).

6. The eye tracking system according to claim 2, wherein: The eye tracking system further comprises: Two coupling gratings (41) are respectively arranged corresponding to the first eye (A) and the second eye (B); Two waveguide plates (42) are respectively arranged corresponding to the two coupling gratings (41), and the waveguide plates (42) are arranged on a side of the two coupling gratings (41) close to the infrared lamp (21), and the waveguide plates (42) have a coupling-out surface (421); The first light guide module (11) and the second light guide module (12) are located between the two outcoupling surfaces (421), and the first light guide module (11) faces one outcoupling surface (421), and the second light guide module (12) faces the other outcoupling surface (421). The first reflected light and the second reflected light are incident on the corresponding waveguide plate (42). The first reflected light is reflected by the incoupling grating (41) and is incident on the first light guide module (11) from the outcoupling surface (421). The second reflected light is reflected by the incoupling grating (41) and is incident on the second light guide module (12) from the outcoupling surface (421).

7. The eye tracking system according to claim 1, wherein: The photosensitive area (141) includes a first photosensitive area (1411) and a second photosensitive area (1412), wherein the first photosensitive area (1411) receives the first reflected light, and the second photosensitive area (1412) receives the second reflected light.

8. The eye tracking system according to claim 1, wherein: The image acquisition module (1) further comprises a shielding body (15), and the image acquisition module (1) is configured to drive the shielding body (15) to be movably arranged in the first optical path (61) or the second optical path (62); The image processing module (3) is configured to collect the first image information through the camera (14) in response to the blocking body (15) blocking the second light path (62); And in response to the blocking body (15) blocking the first light path (61), the second image information is collected through the camera (14).

9. A wearable device, characterized in that: The wearable device includes: A glasses frame (5), comprising a nose bridge (51) and two glasses frames (52), wherein the nose bridge (51) is connected between the two glasses frames (52); and An eye tracking system comprises two waveguides (42), a light source module (2), an image acquisition module (1) and an image processing module (3), wherein the two waveguides (42) are respectively fixed to the two frames (52), the light source module (2) irradiates the first eye (A) and the second eye (B) to form a first reflected light and a second reflected light respectively, the image acquisition module (1) is arranged on the bridge of the nose (51) and comprises a first light guide module (11), a second light guide module (12), a light combining component (13) and a camera (14), wherein the first light guide module (11) and the second light guide module (12) are arranged at a preset angle and are separated by a plurality of channels. The camera (14) is respectively arranged corresponding to the first eye (A) and the second eye (B), the camera (14) includes a photosensitive area (141), the photosensitive area (141) faces the light combining component (13), the first light guide module (11) and the light combining component (13) form a first light path (61), the second light guide module (12) and the light combining component (13) form a second light path (62), the photosensitive area (141) receives the first reflected light through the first light path (61) and generates first image information, and the photosensitive area (141) receives the second reflected light through the second light path (62) and generates second image information; The image processing module (3) is configured to determine gaze point information based on the first image information and the second image information.

10. The wearable device according to claim 9, wherein: The camera (14) is an infrared camera (14); The light source module (2) is arranged on the nose bridge (51) and comprises an infrared lamp (21) and a light splitting component (22); the light splitting component (22) comprises an incident surface (223), a first irradiation surface (221) and a second irradiation surface (222); The infrared light of the infrared lamp (21) passes through the incident surface (223) and is emitted toward the first eye (A) and the second eye (B) from the first irradiation surface (221) and the second irradiation surface (222) respectively.

11. The wearable device according to claim 9, wherein: The camera (14) is an infrared camera (14); The light source module (2) comprises a plurality of infrared lamps (21), the plurality of infrared lamps (21) are arranged at intervals and form a first lamp group (231) and a second lamp group (232), the plurality of infrared lamps (21) of the first lamp group (231) and the plurality of infrared lamps (21) of the second lamp group (232) are symmetrically arranged relative to the light combining component (13), and the first lamp group (231) is arranged on one of the lens frames (52), and the second lamp group (232) is arranged on the other of the lens frames (52), the first lamp group (231) forms a plurality of light spots (C) on the first eye portion (A), and the second lamp group (232) forms a plurality of light spots (C) on the second eye portion (B); The first image information and the second image information both include pupil position information and light spot position information; The image processing module (3) is further configured to determine the rotation direction of the first eye (A) and the second eye (B) according to the relative position change of the pupil position information and the light spot position information.

12. The wearable device according to claim 9, wherein: The eye tracking system further comprises: Two coupling-in gratings (41) are respectively arranged corresponding to the two waveguide plates (42), and the waveguide plates (42) are arranged on a side of the two coupling-in gratings (41) close to the first eye (A), and the waveguide plates (42) have a coupling-out surface (421); The first light guide module (11) and the second light guide module (12) are located between the two outcoupling surfaces (421), and the first light guide module (11) faces one outcoupling surface (421), and the second light guide module (12) faces the other outcoupling surface (421). The first reflected light and the second reflected light are incident on the corresponding waveguide plate (42). The first reflected light is reflected by the incoupling grating (41) and is incident on the first light guide module (11) from the outcoupling surface (421). The second reflected light is reflected by the incoupling grating (41) and is incident on the second light guide module (12) from the outcoupling surface (421).

13. The wearable device according to claim 9, wherein: The image acquisition module (1) is configured such that the first light guide module (11) receives a first reflected light toward the first eye (A), and the second light guide module (12) receives a second reflected light toward the second eye (B).

Images