Eye movement tracking structure, display device and head-mounted display equipment

Abstract

The invention relates to the technical field of display, in particular to an eye movement tracking structure, a display device and head-mounted display equipment. The eye movement tracking structure comprises a plurality of light sources, the light sources emit light in the first direction, and the first direction is the direction where the light sources point to eyeballs; the optical structure layer is located on the first direction side of the light source, and the optical structure layer is configured to enable light rays meeting a preset path in the light rays emitted by the light source to penetrate through; the plurality of sensors are arranged corresponding to the plurality of light sources and are configured to detect reflected light rays of the light rays emitted by the corresponding light sources; and the processor is in communication connection with the plurality of sensors, and the processor is configured to judge spatial position parameters of the eyeballs based on detection signals of the plurality of sensors. According to the invention, accurate judgment of eyeball spatial position parameters can be realized by installing the light source in cooperation with the optical structure layer, the sensor and the processor, other optical devices such as an infrared camera with large volume and weight do not need to be additionally arranged, the overall weight and volume can be effectively reduced, and the use experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of display technology, and in particular to an eye tracking structure, a display device, and a head-mounted display device. Background Art

[0002] Head-mounted extended reality (XR) displays are near-eye display devices that integrate virtual reality (VR), augmented reality (AR) and mixed reality (MR) technologies. Such devices usually have high resolution, high contrast, high refresh rate and low latency to provide an immersive visual experience.

[0003] Common head-mounted display devices are large in size and heavy, which affects the user experience. Summary of the invention

[0004] In view of this, the embodiments of the present disclosure provide an eye tracking structure, a display device, and a head-mounted display device, which solve the problems of large size and weight of head-mounted display devices commonly found in related technologies.

[0005] The present disclosure provides an eye tracking structure, comprising: a plurality of light sources, which emit light in a first direction, where the first direction is the direction in which the light sources point to an eyeball; an optical structure layer, located on the first direction side of the light source, and configured to allow light emitted by the light source that conforms to a preset path to pass through; a plurality of sensors, which are arranged corresponding to the plurality of light sources and are configured to detect reflected light of the corresponding light emitted by the light sources; and a processor, which is communicatively connected to the plurality of sensors, and is configured to determine the spatial position parameters of the eyeball based on the detection signals of the plurality of sensors.

[0006] According to one embodiment of the present disclosure, the light source includes: a plurality of first light sources, each of which corresponds to at least one sensor, and the light emitted by the plurality of first light sources propagates in a direction intersecting the first direction after passing through the optical structure layer, and converges at a point on the side of the first direction.

[0007] According to one embodiment of the present disclosure, multiple first light sources are arranged in a ring; preferably, multiple first light sources are arranged along the same circular ring; or, multiple first light sources are arranged along multiple concentric circular rings; preferably, the first light source includes an OLED device and / or an LED device; preferably, the light source also includes a second light source, and the second light source is located at the center of the circular ring or the center of multiple circular rings, and at least one sensor is correspondingly arranged at the second light source, and the light emitted by the second light source propagates along the first direction after passing through the optical structure layer; preferably, the second light source includes an OLED device and / or an LED device.

[0008] According to one embodiment of the present disclosure, a plurality of the first light sources and the second light sources are configured to emit light simultaneously.

[0009] According to one embodiment of the present disclosure, the multiple light sources include: a plurality of third light sources arranged in multiple rows, each row having a plurality of the third light sources arranged at intervals, and light emitted by the plurality of the third light sources propagates along the first direction after passing through the optical structure layer; preferably, the plurality of the third light sources in each row are arranged at equal intervals; preferably, the third light sources include OLED devices and / or LED devices.

[0010] According to one embodiment of the present disclosure, the plurality of third light sources are configured as follows: the plurality of third light sources in each row emit light alternately in sequence according to a preset order; preferably, the duration of each light emission by each third light source is the time from the start of light emission to the time when the corresponding sensor detects the corresponding reflected light.

[0011] According to one embodiment of the present disclosure, the plurality of sensors correspond one-to-one to the plurality of light sources, and the sensors are arranged adjacent to the corresponding light sources; preferably, the sensors include photoelectric sensors; preferably, the reflected light includes the light emitted by the light source that irradiates the eyeball and is reflected by the eyeball.

[0012] The present disclosure also provides a display device, including: a display panel, having a plurality of pixels, wherein the pixels are used to emit visible light toward a first direction side of the display panel; an eye tracking structure of the above embodiment, wherein the plurality of light sources of the eye tracking structure are installed on the display panel, and the optical structure layer of the eye tracking structure is located on the first direction side of the display panel.

[0013] According to one embodiment of the present disclosure, it also includes: the processor is configured to: determine the distance information between the irradiation point of each light source on the eyeball and the light source based on the detection signals of the multiple sensors of the eye tracking structure, and judge the spatial position parameters of the eyeball through the distance information, and / or determine the light intensity information of the reflected light emitted by each light source based on the detection signals of the multiple sensors of the eye tracking structure, and judge the spatial position parameters of the eyeball based on the light intensity information; preferably, the optical structure layer is configured to allow the light emitted by the pixel point to pass through; preferably, the sensor and the corresponding light source are arranged between adjacent pixel points; preferably, the pixel point includes a first sub-pixel and a second sub-pixel, the first sub-pixel is multiplexed as the light source in the eye tracking structure, and the second sub-pixel is used for image display.

[0014] The present disclosure also provides a head-mounted display device, including: a main structure; an eye tracking structure of the above embodiment, the eye tracking structure is installed on the main structure, or a display device of the above embodiment, the display device is installed on the main structure.

[0015] The embodiments of the present disclosure provide an eye tracking structure, a display device, and a head-mounted display device, which can achieve accurate judgment of the spatial position parameters of the eyeball through the cooperation of a light source with an optical structure layer, a sensor, and a processor. There is no need to additionally set up other optical devices such as infrared cameras that are large in size and weight, which can effectively reduce the overall weight and volume and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 Shown is a schematic diagram of an application scenario of an eye tracking structure provided by an embodiment of the present disclosure.

[0018] Figure 2 Shown is a schematic diagram of the arrangement of light sources in an eye tracking structure provided in an embodiment of the present disclosure.

[0019] Figure 3 Shown is a schematic diagram of the arrangement of light sources in an eye tracking structure provided in another embodiment of the present disclosure.

[0020] Figure 4Shown is a schematic diagram of the arrangement of light sources in an eye tracking structure provided in another embodiment of the present disclosure.

[0021] Figure 5 Shown is a schematic diagram of an application scenario of an eye tracking structure provided by another embodiment of the present disclosure.

[0022] Figure 6 Shown is a schematic diagram of an application scenario of time-sharing scanning of an eye tracking structure provided by another embodiment of the present disclosure.

[0023] Figure 7 Shown is a schematic structural diagram of a display device provided in one embodiment of the present disclosure.

[0024] Figure 8 Shown is a cross-sectional view of the relative positions of a light source and pixels in an eye tracking structure provided by an embodiment of the present disclosure.

[0025] Fig. 9 Shown is a cross-sectional view of the relative positions of a light source and pixels in an eye tracking structure provided by another embodiment of the present disclosure.

[0026] Fig.10 Shown is a front view of the relative positions of the light source and the pixel points in the eye tracking structure provided by another embodiment of the present disclosure.

[0027] Fig.11 Shown is a cross-sectional view of the relative positions of a light source and pixels in an eye tracking structure provided in yet another embodiment of the present disclosure.

[0028] Fig.12 Shown is a PPI calculation principle diagram of an eye tracking structure provided by another embodiment of the present disclosure.

[0029] Fig.13 Shown is a schematic structural diagram of a head-mounted display device provided in one embodiment of the present disclosure.

[0030] Reference numerals:

[0031] 100, display panel; 101, pixel; 102, first sub-pixel; 103, second sub-pixel;

[0032] 200, light source; 201, first light source; 202, second light source; 203, third light source;

[0033] 300, optical structure layer; 301, light transmission hole;

[0034] 400, sensor;

[0035] 500, head mounted display device; 510, main structure; 520, eye tracking structure; 530, display device;

[0036] 600. Eyeball. DETAILED DESCRIPTION

[0037] A head-mounted extended reality (XR) display is a wearable device designed for an extended reality experience. It combines and integrates virtual reality (VR) technology, augmented reality (AR) technology, and mixed reality (MR) technology, allowing users to immerse themselves in and interact with a computer-generated environment. This display is usually composed of one or more display screens that are designed to be mounted directly on the user's head to provide a surround visual experience. In addition, head-mounted XR displays usually include a set of complex optical devices (such as cameras for eye tracking), which are the main reason for the bulkiness of glasses and seriously affect the wearing experience. The embodiments of the present invention are intended to solve the above problems and provide a head-mounted display that can be lightweight.

[0038] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0039] Combination Figures 1 to 6 , the eye tracking structure 520 includes: multiple light sources 200, an optical structure layer 300, multiple sensors 400 and a processor. The light source 200 emits light in a first direction X, and the first direction X is the direction in which the light source 200 points to the eyeball 600. The optical structure layer 300 is located on the first direction X side of the light source 200, and the optical structure layer 300 is configured to allow the light emitted by the light source 200 that conforms to the preset path to pass through. The multiple sensors 400 are arranged corresponding to the multiple light sources 200, and are configured to detect the reflected light emitted by the corresponding light sources 200. The processor is in communication connection with the multiple sensors 400 of the eye tracking structure 520, and the processor is configured to determine the spatial position parameters of the eyeball 600 based on the detection signals of the multiple sensors 400.

[0040] Optionally, the reflected light includes the light emitted by the light source 200 irradiating the eyeball 600 and reflected by the eyeball 600. Since the eyeball 600 has a better reflective effect than other parts of the face, the light reflected by the eyeball 600 is easier to detect. In addition, the spatial position parameters of the eyeball 600 can be accurately determined by the light reflected by the eyeball 600.

[0041] Optionally, the optical structure layer 300 may specifically be a grating structure layer or a liquid crystal layer.

[0042] Optionally, the optical structure layer 300 has a light-transmitting hole 301 penetrating the optical structure layer 300 , and the light-transmitting hole 301 is configured to allow light rays that conform to a preset path among the light rays emitted by the light source 200 to pass through.

[0043] Optionally, the spatial position parameters of the eyeball 600 may include time parameters and position parameters of the eyeball 600 , etc., which have been used to determine the gaze point of the eyeball, etc.

[0044] Exemplarily, the optical structure layer 300 includes a shading film layer, which has a shading effect. The shading film layer is imaged to form a light-transmitting hole 301, which corresponds to a plurality of light sources 200, allowing light emitted by each light source 200 that conforms to a preset path to pass through, for example, allowing light emitted by the light source 200 that propagates along a first direction X to pass through, or allowing light emitted by the light source 200 that propagates along a direction intersecting the first direction X to pass through.

[0045] The optical structure layer 300 can avoid interference between multiple light sources 200, which is conducive to ensuring the accuracy of detection. The optical structure layer 300 can limit the light path of the light source 200 and replace the liquid crystal deflection to change the light path, so that the display process such as OLED can be better integrated.

[0046] The light emitted by the multiple light sources 200 can be detected by the corresponding sensors 400 after being reflected by the eyeball 600. Since the reflectivity of the sclera, iris, cornea and retina of the human eye is different, the distances between different positions of the human eye and the light source 200 are different. Therefore, by determining the light intensity of the reflected light of each light source 200 and / or the distance from each light source 200 to the irradiation position through the detection results of the multiple sensors 400 corresponding to the multiple light sources 200, the pupil center line of sight tracking can be achieved, and the spatial position parameters of the eyeball 600 can be determined.

[0047] In some embodiments of the present disclosure, a plurality of sensors 400 correspond to a plurality of light sources 200 one by one, and the sensors 400 are arranged adjacent to the corresponding light sources 200. In other words, the number of sensors 400 is equal to the number of light sources 200, one sensor 400 is arranged corresponding to each light source 200, and each light source 200 is arranged adjacent to the corresponding sensor 400.

[0048] Thus, each light source 200 and its corresponding sensor 400 can be combined into one unit, ensuring that each sensor 400 can accurately detect the reflected light emitted by the corresponding light source 200, avoiding interference between them. In addition, the light source 200 and its corresponding sensor 400 are combined and installed, without the need to additionally involve the installation position of the sensor 400, which can simplify the production process.

[0049] In addition, arranging the sensor 400 adjacent to the corresponding light source 200 can also achieve the purpose of filtering the reflected light by using the optical structure layer 300, that is, the sensor 400 only receives the reflected light passing through the light-transmitting hole 301, and shields the reflected light of the light source 200 other than the light source 200 corresponding to the sensor 400 from interfering with the detection result, thereby effectively improving the accuracy of the detection.

[0050] Combination Figure 1 and Figure 2 In some embodiments of the present disclosure, the light source 200 includes a plurality of first light sources 201, each of which corresponds to at least one sensor 400, and the light emitted by the plurality of first light sources 201 propagates in a direction intersecting the first direction X after passing through the light-transmitting hole 301, and converges at a point on the side of the first direction X.

[0051] The light emitted by the multiple first light sources 201 propagates in a direction intersecting the first direction X after passing through the light-transmitting hole 301, so that the light emitted by the multiple light sources 200 can be concentrated toward the pupil, and the center position of the line of sight can be determined by using the strong mirror reflection of the pupil, which can reduce the detection difficulty of the sensor 400 and improve the detection accuracy.

[0052] In some embodiments of the present disclosure, multiple first light sources 201 are arranged in a ring. The arrangement direction of the multiple first light sources 201 included in the light source 200 can be a circular ring, or an elliptical ring or a ring approximately elliptical according to the shape of the eyeball 600, which should all be within the protection scope of the present disclosure.

[0053] The distance between the convergence point of the light emitted by the multiple first light sources 201 and the first light source 201 is greater than the minimum distance between the eyeball 600 and the first light source 201. For example, the minimum distance between the first light source 201 and the human eye is 12 mm. The distance between the convergence point of the light emitted by the multiple first light sources 201 and the first light source 201 is greater than 12 mm, so that the light emitted by the multiple first light sources 201 can illuminate the eyeball 600 before converging.

[0054] In this embodiment, the circular arrangement of the plurality of first light sources 201 can better adapt to the shape of the human eyeball 600. On the one hand, it is easy to obtain accurate detection data, and on the other hand, there is no need to set up extra light sources 200 and sensors 400, which can reduce production costs and production difficulties.

[0055] Combination Figure 3Preferably, the multiple first light sources 201 are arranged along multiple concentric rings. For example, the multiple first light sources 201 are arranged along three concentric rings, the diameter difference between each two adjacent rings in the multiple rings is equal, and the multiple first light sources 201 in each ring are arranged in sequence at equal intervals. The detection results can be made more comprehensive and accurate by evenly arranging multiple circles of the first light source 201. It is worth noting that the points included in the three concentric rings in the figure only indicate the possible locations of the first light source 201, and show the arrangement of the first light source 201. The specific arrangement quantity and density of the first light source 201 can be flexibly determined according to actual detection requirements.

[0056] Combination Figure 3 Preferably, in one embodiment, the light source 200 further includes a second light source 202, which is located at the center of the plurality of circular rings, and the second light source 202 is provided with at least one sensor 400. The light emitted by the second light source 202 provided at the center of the circle propagates along the first direction X, and can accurately detect the normal view position of the eyeball 600, and can be used as a reference for the spatial position parameters of the eyeball 600, so that the detection data of the spatial position parameters of the eyeball 600 is more comprehensive and accurate.

[0057] Optionally, the plurality of first light sources 201 and the second light source 202 are configured to emit light at the same time. By allowing the plurality of first light sources 201 and the second light source 202 to work at the same time, the time interval generated by the emission of the plurality of light sources 200 one by one is reduced, thereby speeding up the entire detection process and improving the response speed and real-time performance of the system. In this embodiment, since the arrangement of the first light source 201 and the second light source 202 is more matched with the shape of the eyeball 600, the number and density of the light sources 200 and the sensor 400 are relatively small, and therefore, the plurality of first light sources 201 and the second light source 202 work at the same time without crosstalk, thereby ensuring the accuracy of the detection.

[0058] Combination Figure 4 and Figure 5 In some other embodiments of the present disclosure, the multiple light sources 200 include a plurality of third light sources 203 arranged in multiple rows, each row having a plurality of third light sources 203 arranged at intervals, and the optical structure layer 300 is configured to make the light of the plurality of third light sources 203 propagate along the first direction X.

[0059] Optionally, a plurality of third light sources 203 in the same row are arranged in sequence and spaced apart in the horizontal direction.

[0060] Optionally, a plurality of third light sources 203 in the same row are arranged at equal intervals.

[0061] Optionally, the third light sources 203 in adjacent rows are arranged in parallel, for example, are arranged in a horizontal direction.

[0062] Optionally, the third light sources 203 in adjacent rows are respectively aligned in the vertical direction.

[0063] Optionally, the number of third light sources 203 in each row is equal.

[0064] The arrangement of the plurality of third light sources 203 in this embodiment is conducive to reducing the difficulty of production, and achieving more comprehensive detection, thereby improving the accuracy of the detection results.

[0065] Combination Figure 5 and Figure 6 In one embodiment, the plurality of third light sources 203 are configured such that the plurality of third light sources 203 in each row emit light alternately in a preset order. The time-sharing scanning method adopted in this embodiment can effectively avoid the crosstalk problem when the sensor 400 receives the reflected light, and avoid the detection error. Figure 6 , Figure 6 The multiple dotted lines represent multiple scanning paths, wherein the point in each dotted line represents the scanning position of one of the third light sources 203. During time-sharing scanning, the multiple third light sources 203 corresponding to each dotted line and located in the same row emit light in sequence, and the multiple third light sources 203 corresponding to the multiple dotted lines and located in multiple rows emit light row by row. For example, the first third light source 203 in the first row emits light first, and then the second, third, and ... third light sources 203 in the first row emit light in sequence. After all the third light sources 203 in the first row complete the light-emitting process, the multiple third light sources 203 in the second row emit light one by one, and so on, so that the multiple third light sources 203 in each row emit light alternately in sequence according to a preset order.

[0066] As an optional form, the duration of each light emission of each third light source 203 is the time required from the start of light emission to the detection of the corresponding reflected light by the corresponding sensor 400. For example, multiple third light sources 203 in the same row are lit one by one in the order of arrangement starting from one end of the arrangement direction, and the duration of the lighting of the third light source 203 and the time interval between the lighting of adjacent third light sources 203 are both the time required from the start of light emission to the detection of the corresponding reflected light by the corresponding sensor 400. When the reflected light of the light emitted by one of the third light sources 203 is detected by the sensor 400, the third light source 203 stops emitting light, and the adjacent third light source 203 starts emitting light. In this form, the light emission duration of each third light source 203 can be shortened to a minimum, which can effectively improve the detection efficiency and ensure timeliness.

[0067] As another optional form, the duration of each light emission of the plurality of third light sources 203 is equal, and the duration of each light emission of the third light source 203 is not less than the time required from the start of light emission to the detection of the corresponding reflected light by the corresponding sensor 400. In this form, the control difficulty can be reduced and it is easy to implement.

[0068] The first light source 201 , the second light source 202 and the third light source 203 in the above embodiments may all include OLED devices and / or LED devices.

[0069] In some embodiments, the sensor 400 includes a photoelectric sensor, which is a sensor that uses a photoelectric element as a detection element, and can convert the measured change into a change in a light signal, and further convert the light signal into an electrical signal with the help of the photoelectric element. In this embodiment, the photoelectric sensor can receive the reflected light emitted by the light source 200, and convert the light signal into an electrical signal for determining the spatial position parameters of the eyeball 600.

[0070] When the eye tracking structure 520 of the embodiment of the present application determines the spatial position parameters of the eyeball, it can be implemented in any of the following two ways:

[0071] By calculating the time from each light source 200 to the corresponding sensor 400, the relative position of the light source 200 and the eyeball 600 is calculated. The center of the eyeball 600 is closest to the light source 200 (multiple light sources 200 are usually arranged along a plane perpendicular to the first direction X), and eye tracking can be achieved by judging the pupil center through distance.

[0072] By calculating the intensity of the reflected light emitted by each light source 200, due to the different reflectivity of the sclera, iris, cornea and retina of the human eye, the light intensity received by different sensors 400 can be used to identify the center of the eye's line of sight to achieve eye tracking. For example, if the center of the pupil of the human eye is the position where the reflected light is the strongest, then during the detection process, the position of the sensor 400 that receives the strongest reflected light among the different sensors 400 can be determined as the position relative to the center of the pupil of the human eye, and the center of the eye's line of sight can be determined to achieve eye tracking. Figures 7 to 10 The embodiment of the present disclosure further provides a display device 530 , including a display panel 100 and the above-mentioned eye tracking structure 520 .

[0073] The display panel 100 has a plurality of pixels 101, and the pixel 101 is used to emit visible light toward the first direction X side of the display panel 100. The plurality of light sources 200 of the eye tracking structure 520 are installed on the display panel 100, and the optical structure layer 300 of the eye tracking structure 520 is located on the first direction X side of the display panel 100.

[0074] Taking the head-mounted display device 500 as an example, the display panel 100 of the display device 530 is usually at the lens position and is located in front of the human eyeball 600 (the opposite direction of the first direction X) when in use. It can adopt a flat display panel 100 structure or a curved display panel 100 structure that is more adapted to the shape of the human eyeball 600.

[0075] Optionally, the plurality of pixel points 101 of the display panel 100 include a plurality of independently controlled red sub-pixels, green sub-pixels and blue sub-pixels, so that the display panel 100 can display images.

[0076] The optical structure layer 300 is configured to allow the light emitted by the pixel point 101 to pass through. The optical structure layer 300 only blocks the light emitted by the light source 200, and does not block the light emitted by the pixel point 101 of the display panel 100. Exemplarily, the optical structure layer 300 is arranged according to the shape of the light source 200 and the arrangement of multiple light sources 200, so that it only blocks the first direction X side of the light source 200, and allows part of the light from the light source 200 to pass through through the light-transmitting hole 301, while the optical structure layer 300 is not arranged on the first direction X side of the pixel point 101 of the display panel 100, or the optical structure layer 300 is arranged on the first direction X side of the pixel point 101 of the display panel 100. The light-transmitting hole 301 that allows all the light from the pixel point 101 to pass through.

[0077] In an alternative approach, Figure 7 or Figure 8 As shown, a plurality of light sources 200 are respectively arranged between adjacent pixel points 101 of the display panel 100; or Fig. 9 , Fig.10 As shown, a plurality of light sources 200 are arranged on the first direction X side of the gap between adjacent pixel points 101 of the display panel 100; or Fig.11 As shown, the pixel 101 includes a first sub-pixel 102 and a second sub-pixel 103, wherein the first sub-pixel 102 is multiplexed as the light source 200 in the eye tracking structure 520, and the second sub-pixel 103 is used for image display.

[0078] In the two schemes where multiple light sources 200 are respectively arranged between adjacent pixel points 101 of the display panel 100, or are arranged on the side of the first direction X of the gaps between adjacent pixel points 101 of the display panel 100, since the light sources 200 will not reduce the number of original pixel points 101 and will not block the pixel points 101, they will not have any impact on the pixel points 101.

[0079] The pixel 101 includes a first sub-pixel 102 and a second sub-pixel 103. The first sub-pixel 102 is reused as the light source 200 in the eye tracking structure 520, and the second sub-pixel 103 is used for image display. Since the PPI of the light source 200 is relatively small, the replaced pixel 101 is small and will not be recognized by the human eye, and will not affect the display effect. Fig.12Taking VR glasses as an example, the distance m between the glasses and the eyeball 600 is usually 12 mm. Assuming that the tracking accuracy of the eyeball 600 is 1°, the interval a of the light source 200 in the display panel 100 is calculated to be about 0.21 mm by the formula a=tan1°, and the PPI of the light source 200 is calculated to be about 120 by the formula PPI=25.4 / a. Usually, the PPI of the VR display screen is about 2000, so compared with the PPI of the display screen, the PPI of the light source 200 is relatively small and will not affect the display effect.

[0080] Optionally, the light emitted by the light source 200 is a light beam in an invisible light band, so the light emitted by the light source 200 cannot be seen by human eyes, further ensuring that the light source 200 will not affect the display effect.

[0081] Optionally, the multiple sensors 400 of the eye tracking structure 520 are disposed on the display panel 100. For example, the sensors 400 are installed between adjacent pixels 101 of the display panel 100 (see Figure 8 and Fig.11 ), or, arranged at the first direction X side of the gap between adjacent pixel points 101 of the display panel 100 (see Fig. 9 ).

[0082] Preferably, the sensor 400 and the corresponding light source 200 are arranged between adjacent pixel points 101, so as to facilitate the arrangement of the sensor 400 and the light source 200 and facilitate accurate reception of the reflected light emitted by the corresponding light source 200 by the sensor 400.

[0083] In an optional form, the processor is configured to: determine the distance information between the illumination point of each light source 200 on the eyeball 600 and the light source 200 based on the detection signals of multiple sensors 400, and judge the spatial position parameters of the eyeball 600 through the distance information.

[0084] Since the distances from different positions of the eyeball 600 to the display panel 100 vary, when the eyeball 600 rotates and the spatial position parameters change, the irradiation position of the light source 200 on the eyeball 600 changes, and the distance between at least part of the irradiation point of the light source 200 on the eyeball 600 and the light source 200 also changes. Therefore, the processor can accurately determine the spatial position parameters of the eyeball 600 through the distance information.

[0085] Optionally, the processor is configured to determine the distance information between the irradiation point of the light source 200 on the eyeball 600 and the light source 200 according to the time required for the light source 200 to start emitting light and for the sensor 400 corresponding to the light source 200 to receive the reflected light.

[0086] In another optional form, the processor is configured to: determine the light intensity information of the reflected light emitted by each light source 200 based on the detection signals of multiple sensors 400, and judge the spatial position parameters of the eyeball 600 based on the light intensity information.

[0087] Since different parts of the eyeball 600 (sclera, iris, cornea and retina, etc.) have different reflectivities, the intensities of light reflected by the corresponding light source 200 received by sensors 400 at different positions are different. For example, multiple light sources 200 are configured to emit light of the same intensity, but there are differences in the reflected light received by multiple sensors 400. Based on this difference, the position of the illumination point of the light source 200 on the eyeball 600 can be determined, thereby accurately determining the spatial position parameters of the eyeball 600.

[0088] In another optional form, the processor is configured to: determine the distance information between the irradiation point of each light source 200 on the eyeball 600 and the light source 200 and the light intensity information of the reflected light emitted by each light source 200, respectively, based on the detection signals of the multiple sensors 400, and judge the spatial position parameters of the eyeball 600 based on the distance information and the light intensity information. The form of judging the spatial position parameters of the eyeball 600 based on the distance information and the light intensity information can more accurately judge the spatial position parameters of the eyeball 600, and when one of the judgment methods cannot determine the spatial position parameters of the eyeball 600, it can be determined by another judgment method.

[0089] Optionally, the processor is in communication connection with the display panel 100, and the processor is configured to generate a target image based on the spatial position parameters of the eyeball 600, and send a signal of the target image to the drive circuit of the display panel 100, so that the drive circuit of the display panel 100 controls the pixel point 101 or the light-emitting element on the display panel 100 to display the target image information. For example, the processor determines the gaze point of the eyeball 600 based on the spatial position parameters of the eyeball 600, and generates an enlarged image of the image at the position corresponding to the gaze point, so that the display panel 100 displays the enlarged image information.

[0090] Combination Fig.13 This embodiment also provides a head-mounted display device 500, including a main structure 510, and also including the above-mentioned eye tracking structure 520 or the above-mentioned display device 530, wherein the eye tracking structure 520 or the display device 530 is installed on the main structure 510.

[0091] Exemplarily, the main structure 510 of this embodiment includes a frame, which is used to install the above-mentioned eye tracking structure 520 or display device 530. When the head-mounted display device 500 of this embodiment is worn, the frame is located in front of the human face. Further, the main structure 510 also includes a wearing piece, which can be a temple, a nose pad, and a strap, etc., which is used to be hung on the ears of the human body or wrapped around the head of the human body, so that the display panel 100 in the eye tracking structure 520 or the display device 530 of this embodiment is maintained at a target distance (e.g., 12 mm) in front of the eyeball 600.

[0092] The phrases "one embodiment", "an embodiment", etc. mentioned in the specification indicate that the embodiment described may include a specific feature, structure or characteristic, but not every embodiment may include the specific feature, structure or characteristic. In addition, such phrases do not necessarily refer to the same embodiment. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, it is within the knowledge of those skilled in the art to implement such feature, structure or characteristic in conjunction with other embodiments, whether explicitly or not explicitly described.

[0093] It should be understood that “on,” “above,” and “over” in this disclosure should be interpreted in the broadest manner, so that “on” not only means “directly on something,” but also includes the meaning of “on something” with intervening features or layers therebetween, and “above” or “over” not only includes the meaning of “above” or “over,” but also may include the meaning of “above” or “over something” with no intervening features or layers therebetween (i.e., directly on something).

[0094] Additionally, spatially relative terms, such as "below," "below," "beneath," "above," "above," etc., may be used herein for ease of description to describe the relationship of one component or feature to other components or features as shown in the figures. The spatially relative terms are intended to encompass different orientations of the component in use or operation in addition to the orientation depicted in the figures. The device may be otherwise oriented (rotated 90 degrees or at other orientations), and the spatially relative descriptors used herein may likewise be interpreted accordingly.

[0095] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0096] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. An eye tracking structure, characterized in that: include: A plurality of light sources, each of which emits light in a first direction, wherein the first direction is a direction in which the light source points to an eyeball; an optical structure layer, located at the first direction side of the light source, the optical structure layer being configured to allow light rays emitted by the light source that conform to a preset path to pass through; a plurality of sensors, arranged corresponding to the plurality of light sources, and configured to detect reflected light from the corresponding light sources; A processor is communicatively connected to the plurality of sensors, and the processor is configured to determine the spatial position parameters of the eyeball based on the detection signals of the plurality of sensors.

2. The eye tracking structure according to claim 1, characterized in that: The light source comprises: A plurality of first light sources, each of which corresponds to at least one of the sensors, and the light emitted by the plurality of first light sources propagates in a direction intersecting the first direction after passing through the optical structure layer, and converges at a point on the first direction side.

3. The eye tracking structure according to claim 2, characterized in that: A plurality of the first light sources are arranged in a ring shape; Preferably, a plurality of the first light sources are arranged along the same circular ring; Alternatively, a plurality of the first light sources are arranged along a plurality of concentric rings; Preferably, the first light source comprises an OLED device and / or an LED device; Preferably, the light source further includes a second light source, the second light source is located at the center of the circular ring or at the center of a plurality of the circular rings, the second light source is correspondingly provided with at least one of the sensors, and the light emitted by the second light source propagates in the first direction after passing through the optical structure layer; Preferably, the second light source comprises an OLED device and / or an LED device.

4. The eye tracking structure according to claim 3, characterized in that: The plurality of first light sources and the second light source are configured to emit light simultaneously.

5. The eye tracking structure according to claim 1, characterized in that: The plurality of light sources include: A plurality of third light sources are arranged in a plurality of rows, each row having a plurality of the third light sources arranged at intervals, and light emitted by the plurality of the third light sources propagates along the first direction after passing through the optical structure layer; Preferably, the plurality of third light sources in each row are arranged at equal intervals; Preferably, the third light source includes an OLED device and / or an LED device.

6. The eye tracking structure according to claim 5, characterized in that: The plurality of third light sources are configured as follows: the plurality of third light sources in each row emit light alternately in sequence according to a preset order; Preferably, the duration of each light emission of each third light source is the time required from the start of light emission to the detection of corresponding reflected light by the corresponding sensor.

7. The eye tracking structure according to any one of claims 1 to 6, characterized in that: The plurality of sensors correspond to the plurality of light sources one by one, and the sensors are arranged adjacent to the corresponding light sources; Preferably, the sensor comprises a photoelectric sensor; Preferably, the reflected light includes light emitted by the light source irradiating the eyeball and reflected by the eyeball.

8. A display device, characterized in that: include: A display panel having a plurality of pixels, wherein the pixels are used to emit visible light toward a first direction side of the display panel; The eye tracking structure according to any one of claims 1 to 7, wherein the plurality of light sources of the eye tracking structure are mounted on the display panel, and the optical structure layer of the eye tracking structure is located on the first direction side of the display panel.

9. The display device according to claim 8, characterized in that: Also includes: The processor is configured to: determine the distance information between the illumination point of each light source on the eyeball and the light source based on the detection signals of the multiple sensors of the eye tracking structure, and judge the spatial position parameter of the eyeball according to the distance information, and / or determine the light intensity information of the reflected light emitted by each light source based on the detection signals of the multiple sensors of the eye tracking structure, and judge the spatial position parameter of the eyeball based on the light intensity information; Preferably, the optical structure layer is configured to allow light emitted by the pixel points to pass through; Preferably, the sensor and the corresponding light source are arranged between adjacent pixel points; Preferably, the pixel point includes a first sub-pixel and a second sub-pixel, the first sub-pixel is multiplexed as the light source in the eye tracking structure, and the second sub-pixel is used for image display.

10. A head mounted display device, characterized in that: include: Main structure; The eye tracking structure according to any one of claims 1 to 7, wherein the eye tracking structure is mounted on the main structure; or the display device according to claim 8 or 9, wherein the display device is mounted on the main structure.

Images