Wearable display device and method for determining position of gaze point

Abstract

A wearable display device and a method for determining the position of a gaze point in the field of virtual reality technology are disclosed. The wearable display device can process the electrical signals transmitted from each photoelectric sensing assembly efficiently, so that the wearable display device can quickly determine the position of the gaze point of the user's eyes on the display panel based on the electrical signals transmitted from each photoelectric sensing assembly, thereby improving the efficiency of image display on the display panel and achieving a high update rate of the display panel.

Description

[Technical Field]

[0001] The present application relates to the field of virtual reality technology, and in particular to a wearable display device and a method for determining the position of a gaze point. [Background technology]

[0002] A virtual reality (VR) device is a device that can create a virtual environment from displayed images and immerse the user in that virtual environment. Summary of the Invention [Means for solving the problem]

[0003] This application provides a wearable display device and a method for determining the position of a gaze point, and the technical solution is as follows:

[0004] In one aspect, a display panel having a display area and a peripheral area surrounding the display area;

[0005] a plurality of light-emitting elements that emit light to be irradiated onto the user's eyes;

[0006] a plurality of photoelectric sensing assemblies disposed in the peripheral region, each of which is used to receive optical signals of the plurality of light-emitting elements reflected by the user's eyes and convert the optical signals into electrical signals, the electrical signals being used to determine the position of the user's eye gaze point on the display panel;

[0007] an optical structure disposed on a side of the photoelectric sensing assembly away from the display panel, the orthogonal projection of which onto the display panel is located in the peripheral region, the optical structure having a light-shielding region and a plurality of light-transmitting regions, the plurality of light-transmitting regions corresponding one-to-one to the plurality of photoelectric sensing assemblies, each of the light-transmitting regions being used to transmit the optical signal to at least one corresponding photoelectric sensing assembly;

[0008] Including, This relates to a wearable display device.

[0009] As one option, the peripheral region includes a first region extending along a first direction and a second region extending along a second direction, the first direction intersecting the second direction, and the plurality of photoelectric sensing assemblies includes a plurality of first photoelectric sensing assemblies and a plurality of second photoelectric sensing assemblies;

[0010] Here, the plurality of first photoelectric sensing assemblies are arranged in the first region and along the first direction, and the plurality of second photoelectric sensing assemblies are arranged in the second region and along the second direction.

[0011] As one option, the optical structure includes a first branched structure extending along the first direction and a second branched structure extending along the second direction;

[0012] the first branch structure has a plurality of first light-transmitting areas corresponding one-to-one to the plurality of first photoelectric sensing assemblies, and an orthogonal projection of each of the first light-transmitting areas onto the display panel at least partially overlaps with an orthogonal projection of a corresponding one of the first photoelectric sensing assemblies onto the display panel;

[0013] The second branch structure has a plurality of second light-transmitting areas that correspond one-to-one to the plurality of second photoelectric sensing assemblies, and the orthogonal projection of each of the second light-transmitting areas onto the display panel at least partially overlaps with the orthogonal projection of a corresponding one of the second photoelectric sensing assemblies onto the display panel.

[0014] As an alternative, the first direction is perpendicular to the second direction, and the peripheral region includes two first regions and two second regions;

[0015] Here, the two first regions are arranged along the second direction and are respectively arranged on both sides of the display region, and the two second regions are arranged along the first direction and are respectively arranged on both sides of the display region.

[0016] As an option, the optical structure includes two first branch structures corresponding one-to-one to the two first regions and two second branch structures corresponding one-to-one to the two second regions.

[0017] Optionally, a detection width of the photoelectric sensing assembly along the target direction and a width of the light-transmitting region along the target direction are positively correlated;

[0018] a detection width of the photoelectric sensing assembly along the target direction and a width of the photoelectric sensing assembly along the target direction are positively correlated;

[0019] Here, the target direction is the first direction or the second direction.

[0020] Optionally, the detection width h of the photoelectric sensing assembly along the target direction is:

[0021]

number

[0022] Fulfilling

[0023] Here, u is the distance between the user's eye and the optical structure, v is the distance between the optical structure and the multiple photoelectric sensing assemblies, n1 is the refractive index of the medium between the optical structure and the multiple photoelectric sensing assemblies, n2 is the refractive index of the medium between the user's eye and the optical structure, p is the width of the photoelectric sensing assembly along the target direction, and s is the width of the light-transmitting region along the target direction.

[0024] Optionally, each of the light-transmitting regions is a circular through-hole;

[0025] Alternatively, each of the light-transmitting regions is a rectangular through-hole.

[0026] As one option, a first side of each of the rectangular through holes among the plurality of rectangular through holes is parallel to a first direction and a second side of each of the rectangular through holes is parallel to a second direction, and the plurality of rectangular through holes includes a plurality of first rectangular through holes and a plurality of second rectangular through holes;

[0027] the first rectangular through hole is located in a first region extending along the first direction in the peripheral region, and a length of a first side of the first rectangular through hole is smaller than a length of a second side of the first rectangular through hole;

[0028] The second rectangular through hole is located in a second region extending along the second direction in the peripheral region, and a length of a first side of the second rectangular through hole is greater than a length of a second side.

[0029] Optionally, the wearable display device further comprises a light-transmitting layer;

[0030] The light-transmitting layer is disposed between the plurality of photoelectric sensing assemblies and the optical structure.

[0031] Optionally, the wearable display device further comprises a filter;

[0032] The filter is positioned on a side of the plurality of photoelectric sensing assemblies away from the display panel, and the orthogonal projection of the filter onto the display panel covers the orthogonal projection of the plurality of photoelectric sensing assemblies onto the display panel, and the filter is for transmitting infrared light and absorbing visible light.

[0033] As an option, the wearable display device further includes a lens and a lens frame;

[0034] The lens is disposed on the display side of the display panel, and the lens frame is disposed on the edge of the lens.

[0035] As one option, the plurality of light emitting elements are arranged on a side of the lens frame away from the display panel and are connected to the lens frame.

[0036] As one option, the plurality of light emitting elements are uniformly arranged on a side of the lens frame away from the display panel.

[0037] Optionally, the light emitting element is an infrared light emitting diode.

[0038] Optionally, the wearable display device further includes a first polarizing layer and a second polarizing layer;

[0039] The first polarizing layer is disposed on the light-emitting side of the light-emitting element, and the second polarizing layer is disposed on the side of the plurality of photoelectric sensing assemblies away from the display panel, and the polarization direction of the second polarizing layer intersects with the polarization direction of the first polarizing layer.

[0040] As an option, the polarization direction of the second polarizing layer is perpendicular to the polarization direction of the first polarizing layer.

[0041] In another aspect, the present invention is applied to the wearable display device described in the above aspect,

[0042] receiving light signals from the plurality of light emitting elements reflected by the user's eyes;

[0043] converting the optical signal into an electrical signal;

[0044] determining a position of a user's eye gaze point on the display panel based on a signal value of the electrical signal and a position of the at least one photoelectric sensing assembly; Including, This relates to a method for determining the position of the point of interest.

[0045] As an option, determining a position of a user's eye gaze point on a display panel based on the signal value of the electrical signal and the position of at least one photoelectric sensing assembly includes:

[0046] determining at least one target first photoelectric sensing assembly from a plurality of first photoelectric sensing assemblies arranged along a first direction;

[0047] determining at least one target second photoelectric sensing assembly from a plurality of second photoelectric sensing assemblies arranged along a second direction;

[0048] determining a position of a user's eye gaze point on the display panel based on the position of each of the target first photoelectric sensing assemblies and the position of each of the target second photoelectric sensing assemblies; Including,

[0049] Here, the signal value of the electrical signal transmitted from the target first photoelectric sensing assembly is less than or equal to a first threshold, and the signal value of the electrical signal transmitted from the target second photoelectric sensing assembly is less than or equal to a second threshold.

[0050] In yet another aspect, the present invention relates to a computer-readable storage medium having instructions stored thereon, the instructions being executed by a display device to implement the determination method described in the previous aspect.

[0051] In yet another aspect, a computer program product is provided that includes instructions that, when executed on the computer, cause the computer to perform the method for determining the position of a point of regard described in the previous aspect.

[0052] In order to more clearly explain the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments will be briefly described below. The drawings in the following description are only some embodiments of the present disclosure, and it is obvious to those skilled in the art that other drawings can be obtained based on these drawings without any creative work. [Brief explanation of the drawings]

[0053] [Figure 1] 1 is a structural schematic diagram of a wearable display device according to an embodiment of the present application; [Figure 2] FIG. 1 is a plan view of a display panel according to an embodiment of the present application. [Figure 3] 1 is a schematic diagram of a display panel and a photoelectric sensing assembly according to an embodiment of the present application; [Figure 4] 1 is a structural schematic diagram of an optical structure according to an embodiment of the present application; [Figure 5] 1 is a schematic diagram of signal values ​​of optical signals reflected at different regions of a user's eye received by a plurality of first photoelectric sensing assemblies according to an embodiment of the present application; [Figure 6] 10 is a schematic diagram of optical signal values ​​reflected at different regions of a user's eye received by a plurality of second photoelectric sensing assemblies according to an embodiment of the present application. [Figure 7] FIG. 1 is a schematic diagram of another display panel and photoelectric sensing assembly according to an embodiment of the present application. [Figure 8] FIG. 1 is a structural schematic diagram of another optical structure according to an embodiment of the present application. [Figure 9] 1 is a schematic diagram of a detection width along a target direction of a photoelectric sensing assembly according to an embodiment of the present application; [Figure 10] FIG. 10 is a schematic diagram of a detection width along a target direction of another photoelectric sensing assembly according to an embodiment of the present application. [Figure 11] FIG. 10 is a schematic diagram of a detection width along a target direction of yet another photoelectric sensing assembly according to an embodiment of the present application. [Figure 12]FIG. 2 is a structural schematic diagram of another wearable display device according to an embodiment of the present application. [Figure 13] FIG. 10 is a structural schematic diagram of yet another optical structure according to an embodiment of the present application. [Figure 14] FIG. 10 is a structural schematic diagram of yet another wearable display device according to an embodiment of the present application. [Figure 15] 1 is a flowchart of a method for determining the position of a gaze point according to an embodiment of the present application; [Figure 16] 1 is a flowchart of another method for determining the position of a gaze point according to an embodiment of the present application; [Figure 17] 4 is a graph of an electrical signal of a first photoelectric sensing assembly according to an embodiment of the present application. [Figure 18] 10 is a graph of an electrical signal of another first photoelectric sensing assembly according to an embodiment of the present application. [Figure 19] FIG. 10 is a schematic diagram of yet another display panel and photoelectric sensing assembly according to an embodiment of the present application. [Figure 20] 10 is a graph of an electrical signal of a second photoelectric sensing assembly according to an embodiment of the present application. [Figure 21] 10 is a graph of an electrical signal of another second photoelectric sensing assembly according to an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0054] In order to make the objectives, technical solutions and advantages of the present application clearer, the following describes the embodiments of the present application in more detail with reference to the drawings.

[0055] In related technology, a VR device includes a display panel, a camera, a processor, and a driving circuit. The camera is used to capture an image of a user's eyes. The processor determines the position of the user's gaze point on the display panel from the eye image and partially renders the display image that needs to be displayed based on the position of the gaze point. The driving circuit receives the partially rendered display image sent from the processor and drives the display panel to display it. Since the processor can partially render only the area of ​​the display image where the gaze point is located and does not need to render the display image globally, it not only reduces the processor load but also ensures the display effect of the display panel.

[0056] However, in the related art, the processor has low efficiency in determining the position of the gaze point based on the image of the eye captured by the camera, which reduces the display efficiency of the display panel.

[0057] The terms used in some of the embodiments of this application are not intended to limit this application and are used only to describe the examples of this application. Unless otherwise defined, technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by those of ordinary skill in the art to which this application pertains. The terms "first," "second," "third," and similar terms used in the specification and claims of this application do not denote any order, quantity, or importance, but are used merely to distinguish between different components. Similarly, similar terms such as "one" or "1" do not denote a numerical limitation but rather mean the presence of at least one. Similar terms such as "comprise" or "contain" mean that the element or object appearing before this term covers the elements or objects listed thereafter and their equivalents, without excluding other elements or objects. Similar terms such as "couple" or "connect" are not limited to physical or mechanical connections, but can include direct or indirect electrical connections. "Top," "bottom," "left," "right," etc., refer only to relative positional relationships; if the absolute position of the described object changes, the relative positional relationships may change accordingly.

[0058] 1 is a structural schematic diagram of a wearable display device according to an embodiment of the present application. Referring to FIG. 1, the wearable display device 10 may include a display panel 101, a plurality of light-emitting elements 102, a plurality of photoelectric sensing assemblies 103, and an optical structure 104.

[0059] Here, the light emitted from the plurality of light-emitting elements 102 is used to illuminate the user's eyes, and the user's eyes can reflect the light emitted from the plurality of light-emitting elements 102. Thus, each photoelectric sensing assembly 103 is used to receive the optical signal of the light-emitting elements 102 reflected by the user's eyes and convert the optical signal into an electrical signal, which is used to determine the position of the user's eye gaze point on the display panel 101.

[0060] Typically, the data amount of the electrical signal is small and the data amount of the image is large, so the efficiency of processing the electrical signal by the wearable display device 10 is high relative to the efficiency of processing the image by the wearable display device 10. In the embodiment of the present application, the wearable display device 10 has high efficiency of processing the electrical signal transmitted from each photoelectric sensing assembly 103, which can quickly determine the position of the user's eye gaze point on the display panel 101, thereby improving the efficiency of image display on the display panel 101 and achieving a high update rate of the display panel 101.

[0061] 2 is a plan view of a display panel according to an embodiment of the present application. Referring to FIG. 2, the display panel 101 has a display area 101a and a peripheral area 101b surrounding the display area 101a. A plurality of photoelectric sensing assemblies 103 are arranged in the peripheral area 101b. The plurality of photoelectric sensing assemblies 103 do not affect the normal display of the display panel 101, and the display effect of the display panel 101 is good.

[0062] 1 and 2, the optical structure 104 is disposed on the side of the photoelectric sensing assembly 103 away from the display panel 101. The orthogonal projection of the optical structure 104 onto the display panel 101 is located in the peripheral region 101b. The optical structure 104 has a light-blocking region 104a and a plurality of light-transmitting regions 104b, which correspond one-to-one to the plurality of photoelectric sensing assemblies 103, and each light-transmitting region 104b is used to transmit an optical signal to at least one corresponding photoelectric sensing assembly 103.

[0063] Alternatively, each light-transmitting region 104b may transmit an optical signal only to one corresponding photoelectric sensing assembly 103. Alternatively, each light-transmitting region 104b may transmit an optical signal not only to one corresponding photoelectric sensing assembly 103 but also to photoelectric sensing assemblies 103 adjacent to the photoelectric sensing assembly 103 corresponding to the light-transmitting region 104b. In this case, the optical signal transmitted by the light-transmitting region 104b to the corresponding one photoelectric sensing assembly 103 may be strong, and the optical signal transmitted to the adjacent photoelectric sensing assemblies 103 may be weak.

[0064] Because the orthogonal projection of the optical structure 104 onto the display panel 101 is located in the peripheral region 101b, the optical structure 104 does not affect the normal display of the display region 101a of the display panel 101. Furthermore, the optical structure 104 is divided into a light-blocking region 104a and a plurality of light-transmitting regions 104b, and each light-transmitting region 104b is used to transmit an optical signal to at least a corresponding photoelectric sensing assembly 103. Therefore, the optical signals received by different photoelectric sensing assemblies 103 correspond to different regions of the user's eye, so that the wearable display device 10 can determine the position of the gaze point based on electrical signals converted from the optical signals reflected by different regions of the user's eye.

[0065] As described above, an embodiment of the present application provides a wearable display device, and the wearable display device has high processing efficiency for the electrical signals transmitted from each photoelectric sensing assembly, so that the wearable display device can quickly determine the position of the user's eye gaze point on the display panel based on the electrical signals transmitted from each photoelectric sensing assembly, thereby improving the efficiency of image display on the display panel and achieving a high update rate for the display panel.

[0066] Alternatively, the light emitting element 102 may be an infrared light emitting diode. Because there is a large difference in the reflectance of the pupil, sclera, and iris of a user's eye to infrared light (for example, the reflectance of the pupil is 3% to 5%, the reflectance of the sclera is 70% to 80%, and the reflectance of the iris is 10% to 20%), the light emitting element 102 is designed to be an infrared light emitting diode, so that there is a large difference in the optical signals of the infrared light reflected by the pupil, the optical signals of the infrared light reflected by the sclera, and the optical signals of the infrared light reflected by the iris received by the photoelectric sensing assembly 103, and the wearable display device 10 This is useful for the processor to determine the position of the gaze point of the user's eye (pupil) on the display panel 101. Exemplarily, the wavelength range of the light emitted from the light emitting element 102 may be 850 nm (nanometers) to 940 nm.

[0067] 2, the peripheral region 101b of the display panel 101 includes a first region 101b1 extending along a first direction X and a second region 101b2 extending along a second direction Y. Here, the first direction X intersects with the second direction Y.

[0068] 2 and 3, the plurality of photoelectric sensing assemblies 103 may include a plurality of first photoelectric sensing assemblies 103a and a plurality of second photoelectric sensing assemblies 103b. The plurality of first photoelectric sensing assemblies 103a are arranged in a first region 101b1 and are arranged along a first direction X. The plurality of second photoelectric sensing assemblies 103b are arranged in a second region 101b2 and are arranged along a second direction Y.

[0069] As an option, the plurality of first photoelectric sensing assemblies 103a are uniformly arranged along the first direction X, and the plurality of second photoelectric sensing assemblies 103b are uniformly arranged along the second direction Y.

[0070] 2 and 4, the optical structure 104 may include a first branch structure 1041 extending along a first direction X and a second branch structure 1042 extending along a second direction Y.

[0071] The first branch structure 1041 has a plurality of first light-transmitting areas 104b1 corresponding one-to-one to the plurality of first photoelectric sensing assemblies 103a. The orthogonal projection of each first light-transmitting area 104b1 onto the display panel 101 at least partially overlaps with the orthogonal projection of the corresponding one of the first photoelectric sensing assemblies 103a onto the display panel 101. This allows the first light-transmitting area 104b1 to transmit an optical signal to the corresponding one of the first photoelectric sensing assemblies 103a.

[0072] The second branch structure 1042 has a plurality of second light-transmitting areas 104b2 corresponding one-to-one to the plurality of second photoelectric sensing assemblies 103b. The orthogonal projection of each second light-transmitting area 104b2 onto the display panel 101 at least partially overlaps with the orthogonal projection of the corresponding one of the second photoelectric sensing assemblies 103b onto the display panel 101. This allows the second light-transmitting area 104b2 to transmit an optical signal to the corresponding one of the second photoelectric sensing assemblies 103b.

[0073] In an embodiment of the present application, a processor included in wearable display device 10 can receive electrical signals transmitted from each of the plurality of first photoelectric sensing assemblies 103 a and determine at least one target first photoelectric sensing assembly from the plurality of first photoelectric sensing assemblies 103 a. The processor can also receive electrical signals transmitted from each of the plurality of second photoelectric sensing assemblies 103 b and determine at least one target second photoelectric sensing assembly from the plurality of second photoelectric sensing assemblies 103 b. Finally, the processor can determine the position of the user's eye gaze point on display panel 101 based on the position of the at least one target first photoelectric sensing assembly and the position of the at least one target second photoelectric sensing assembly.

[0074] Here, the signal value of the electrical signal transmitted from the target first photoelectric sensing assembly may be equal to or less than a first threshold, and the signal value of the electrical signal transmitted from the target second photoelectric sensing assembly may be equal to or less than a second threshold, where the first threshold and the second threshold may be equal or unequal, and the embodiment of the present application is not limited thereto.

[0075] Because the reflectivity of the light emitted from the light-emitting element 102 in different regions of the user's eye is different, the optical signals collected by the first photoelectric sensing assembly 103a at different positions and the second photoelectric sensing assembly 103b at different positions are different. Because the pupil has the darkest color, the signal value of the optical signal reflected by the pupil is the smallest. Furthermore, the signal value of the electrical signal converted from the optical signal reflected by the pupil is the smallest. Here, the signal value of the optical signal is intended to represent the light intensity.

[0076] As an option, the processor may pre-store the positions of each of the first photoelectric sensing assemblies 103 a and each of the second photoelectric sensing assemblies 103 b. For either the first photoelectric sensing assembly 103 a or the second photoelectric sensing assemblies 103 b, the position of the photoelectric sensing assembly 103 stored in the processor may refer to the coordinates of the photoelectric sensing assembly 103 in a two-dimensional coordinate system. The two-dimensional coordinate system may refer to a coordinate system based on a plane on which the display panel 101 is located.

[0077] For example, FIG. 5 shows signal values ​​of optical signals reflected from different regions of the user's eye collected by each first photoelectric sensing assembly 103a. FIG. 6 shows signal values ​​of optical signals reflected from different regions of the user's eye collected by each second photoelectric sensing assembly 103b. In FIGS. 5 and 6, the abscissa x0 indicates the location of each photoelectric sensing assembly 103, and the ordinate y0 indicates the signal value of the received electrical signal. Referring to FIGS. 5 and 6, the pupil has the darkest color, so the optical signal reflected from the pupil is smallest. Correspondingly, the signal value of the electrical signal obtained by converting the optical signal reflected from the pupil is smallest. Therefore, in an embodiment of the present application, the processor can reliably determine the position of the user's eye's gaze point on the display panel 101 based on the position of the target first photoelectric sensing assembly with the smallest signal value of the transmitted electrical signal and the position of the target second photoelectric sensing assembly with the smallest signal value of the transmitted electrical signal.

[0078] In an embodiment of the present application, the first threshold and the second threshold may be fixed values ​​pre-stored in the processor. Alternatively, the first threshold may be determined by the processor based on signal values ​​of the received electrical signals from the plurality of first photoelectric sensing assemblies 103a. The second threshold may be determined by the processor based on signal values ​​of the received electrical signals from the plurality of second photoelectric sensing assemblies 103b.

[0079] For example, the processor may sort the signal values ​​of the N electrical signals transmitted from the N first photoelectric sensing assemblies 103a in ascending order and determine the signal value of the nth bit as a first threshold, where N is an integer greater than 1 and n is an integer greater than 1 and less than N / 2. The processor may sort the signal values ​​of the M electrical signals transmitted from the M second photoelectric sensing assemblies 103b in ascending order and determine the signal value of the mth bit as a second threshold, where M is an integer greater than 1 and m is an integer greater than 1 and less than M / 2.

[0080] Alternatively, the processor may receive the plurality of first photoelectric sensing assemblies 103a Sent from The smallest signal value of the received electrical signal is determined as the first threshold value, and the received second photoelectric sensing assemblies 103b Sent from The smallest signal value of the electrical signal is determined as the second threshold value.

[0081] In an embodiment of the present application, the processor can determine a first coordinate value of a target first photoelectric sensing assembly having the smallest signal value of the electrical signals transmitted from the plurality of first photoelectric sensing assemblies 103 a, and can determine a second coordinate value of a target second photoelectric sensing assembly having the smallest signal value of the electrical signals transmitted from the plurality of second photoelectric sensing assemblies 103 b. The processor can determine the position of the user's eye gaze point on the display panel 101 based on the first coordinate value and the second coordinate value.

[0082] 2, a first direction X is perpendicular to a second direction Y. The first direction X may be a pixel row direction of the display panel 101, and the second direction Y may be a pixel column direction of the display panel 101.

[0083] Referring to FIG. 2, the peripheral region 101b may include two first regions 101b1 and two second regions 101b2. The two first regions 101b1 are arranged along the second direction Y and are respectively disposed on both sides of the display region 101a. The two second regions 101b2 are arranged along the first direction X and are respectively disposed on both sides of the display region 101a. Correspondingly, referring to FIG. 7, for the plurality of first photoelectric sensing assemblies 103a included in the plurality of photoelectric sensing assemblies 103, some are arranged in one first region 101b1 and other parts are arranged in the other first region 101b1. For the plurality of second photoelectric sensing assemblies 103b included in the plurality of photoelectric sensing assemblies 103, some are arranged in one second region 101b2 and other parts are arranged in the other second region 101b2. Also, referring to Figure 8, the optical structure 104 includes two first branch structures 1041 that correspond one-to-one to the two first regions 101b1 and two second branch structures 1042 that correspond one-to-one to the two second regions 101b2.

[0084] This allows the processor to determine the position of the user's eye gaze point on the display panel 101 based on the first photoelectric sensing assembly 103a in the two first areas 101b1 and the second photoelectric sensing assembly 103b in the two second areas 101b2, thereby improving the accuracy of the determined gaze point position.

[0085] In the embodiment of the present application, the detection width of the photoelectric sensing assembly 103 along the target direction and the width of the light-transmitting region 104b along the target direction are positively correlated, that is, the larger the width of the light-transmitting region 104b along the target direction, the larger the detection width of the photoelectric sensing assembly 103 along the target direction, and the smaller the width of the light-transmitting region 104b along the target direction, the smaller the detection width of the photoelectric sensing assembly 103 along the target direction.

[0086] Furthermore, there is a positive correlation between the detection width along the target direction of the photoelectric sensing assembly 103 and the width along the target direction of the photoelectric sensing assembly 103. That is, the larger the detection width along the target direction of the photoelectric sensing assembly 103, the larger the detection width along the target direction of the photoelectric sensing assembly 103, and the smaller the width along the target direction of the photoelectric sensing assembly 103, the smaller the detection width along the target direction of the photoelectric sensing assembly 103.

[0087] Here, the detection width along the target direction of the photoelectric sensing assembly 103 may refer to the width of the area of ​​the user's eye corresponding to the optical signal detectable by the photoelectric sensing assembly 103, that is, the optical signal reflected in the area of ​​the user's eye whose width along the target direction is the detection width can be detected by the photoelectric sensing assembly 103. The target direction may be the first direction X or the second direction Y.

[0088] As an option, the detection width h of the photoelectric sensing assembly 103 along the target direction is:

[0089]

number

[0090] Meet the following.

[0091] Now, referring to FIG. 9, u is the distance between the user's eye and the optical structure 104, v is the distance between the optical structure 104 and the multiple photoelectric sensing assemblies 103, n1 is the refractive index of the medium between the optical structure 104 and the multiple photoelectric sensing assemblies 103, n2 is the refractive index of the medium between the user's eye and the optical structure 104, p is the width of the photoelectric sensing assembly 103 along the target direction, and s is the width of the light-transmitting region 104b along the target direction.

[0092] In the embodiment of the present application, the above formula (1) can be derived by the following process.

[0093] 9, the angle between the two first connecting lines respectively connecting the opposite sides of the detection area of ​​the photoelectric sensing assembly 103 to the light-transmitting area 104b of the optical structure 104 is α (the angle α can be referred to as the light-receiving angle). The angle between the two second connecting lines respectively connecting the opposite sides of the light-transmitting area 104b of the optical structure 104 to the opposite sides of the photoelectric sensing assembly 103 is β. Here, the two second connecting lines may be refracted light lines of the light incident from the light-transmitting area 104b where the two first connecting lines are located.

[0094] Assuming that the distance between the intersection point of the two first connecting lines and the photoelectric sensing assembly 103 is x, and referring to FIG. 9, the following equation is obtained:

[0095]

number

[0096] According to the above formula (2),

[0097]

number

[0098] is obtained.

[0099] According to the law of refraction,

[0100]

number

[0101] becomes.

[0102] Assuming that the distance between the intersection point of the two second connecting lines and the photoelectric sensing assembly 103 is y, and referring to FIG. 9, the following equation is obtained:

[0103]

number

[0104] According to the triangle similarity theorem,

number

[0105] is obtained.

[0106] According to the above equations (5) and (6),

[0107]

number

[0108] is obtained.

[0109] As can be seen from Figure 9,

[0110]

number

[0111] becomes.

[0112] Substituting equation (7) into equation (4) gives α / 2, which is then used to calculate the

[0113]

number

[0114] After this, we substitute equation (8) into equation (3).

[0115]

number

[0116] This gives us the above formula (1).

[0117] As one option, the detection width h of the photoelectric sensing assembly 103 along the target direction relates to the accuracy with which the wearable display device 10 determines the position of the gaze point. The larger the detection width h of the photoelectric sensing assembly 103 along the target direction, the less accurate the wearable display device 10 will be at determining the position of the gaze point, and the smaller the detection width h of the photoelectric sensing assembly 103 along the target direction, the more accurate the wearable display device 10 will be at determining the position of the gaze point.

[0118] Before designing the wearable display device 10, by determining an appropriate width p of the photoelectric sensing assembly 103 along the target direction and a width s of the light-transmitting area 104b along the target direction, the detection width h of the photoelectric sensing assembly 103 along the target direction can be obtained, and the accuracy of the position determination of the gaze point by the wearable display device 10 can meet the accuracy requirements.

[0119] For example, referring to Figures 10 and 11, Figure 11 has a larger width s of the light-transmitting region 104b along the target direction compared to Figure 10, and in order to make the detection width h of the photoelectric sensing assembly 103 along the target direction of Figure 11 equal to the detection width h of the photoelectric sensing assembly 103 along the target direction of Figure 10, the width p of the photoelectric sensing assembly 103 along the target direction of Figure 11 can be smaller than the width p of the photoelectric sensing assembly 103 along the target direction of Figure 10.

[0120] In the embodiment of the present application, referring to Figures 4 and 8, each light-transmitting region 104b is a circular through-hole, or referring to Figures 12 and 13, each light-transmitting region 104b is a rectangular through-hole, where the rectangular through-hole may be a slit.

[0121] 13 , a first side of each of the plurality of rectangular through holes 104b is parallel to the first direction X, and a second side of each of the plurality of rectangular through holes 104b is parallel to the second direction Y. The plurality of rectangular through holes 104b includes a plurality of first rectangular through holes 104b1 and a plurality of second rectangular through holes 104b2. The first rectangular through hole 104b1 is located in a first region 101b1 extending along the first direction X in the peripheral region 101b, and the length of the first side of the first rectangular through hole 104b1 is smaller than the length of the second side. The second rectangular through hole 104b2 is located in a second region 101b2 extending along the second direction Y in the peripheral region 101b, and the length of the first side of the second rectangular through hole 104b2 is larger than the length of the second side.

[0122] The slits limit the light collection of the circular through-holes only in the target direction. For example, the first rectangular through-hole 104b1 located in the first region 101b1 limits the light collection in the first direction X (the target direction is the first direction X). The second rectangular through-hole 104b2 located in the second region 101b2 limits the light collection in the second direction Y (the target direction is the second direction Y).

[0123] The slit limits light collection only in the target direction and does not limit light reception in other directions perpendicular to the target direction, so that a large amount of optical signal passes through the circular through-hole, allowing the photoelectric sensing assembly 103 to receive a strong optical signal and reducing the requirements for the photoelectric sensing assembly 103's optical signal reception capability.

[0124] 14 is a structural schematic diagram of yet another wearable display device according to an embodiment of the present application. Referring to FIG. 14, the wearable display device 10 may further include a light-transmitting layer 105. The light-transmitting layer 105 is disposed between the multiple photoelectric sensing assemblies 103 and the optical structure 104.

[0125] By disposing the light-transmitting layer 105 between the photoelectric sensing assembly 103 and the optical structure 104, the distance between the photoelectric sensing assembly 103 and the optical structure 104 can be increased, and the area of ​​the user's eye corresponding to the optical signal that each photoelectric sensing assembly 103 can receive can be reduced, and the photoelectric sensing assembly 103 This can improve the accuracy of the received optical signal and further improve the accuracy of the determined position of the gaze point.

[0126] 14 , the wearable display device 10 may further include a filter 106. The filter 106 is disposed on a side of the plurality of photoelectric sensing assemblies 103 away from the display panel 101. The orthogonal projection of the filter 106 onto the display panel 101 covers the orthogonal projection of the plurality of photoelectric sensing assemblies 103 onto the display panel 101. Here, the filter 106 is for transmitting infrared light and absorbing visible light.

[0127] A filter is attached to the side of the photoelectric sensing assembly 103 away from the display panel 101. 106 is arranged to filter visible light, thereby preventing the light emitted from the display panel 101 from affecting the optical signal received by the photoelectric sensing assembly 103, and ensuring the accuracy of the determined position of the gaze point.

[0128] 1, 12, and 14, the wearable display device 10 may further include a lens 107 and a lens frame 108. The lens 107 may be disposed on the display side of the display panel 101, and a user can view an image displayed on the display panel 101 through the lens 107. The lens frame 108 is disposed on the edge of the lens 107 to support and fix the lens 107.

[0129] 1, 12, and 14, the plurality of light-emitting elements 102 may be arranged on the side of the lens frame 108 away from the display panel 101 and connected to the lens frame 108. In other words, the plurality of light-emitting elements 102 are fixed to the side of the lens frame 108 away from the display panel 101.

[0130] As an option, the plurality of light-emitting elements 102 may be uniformly arranged on the side of the lens frame 108 away from the display panel 101, thereby improving the uniformity of the optical signals received in each region of the user's eye and ensuring the accuracy of the determined position of the gaze point.

[0131] 14 , in an embodiment of the present application, the wearable display device 10 may further include a first polarizing layer 109 and a second polarizing layer 110. The first polarizing layer 109 may be disposed on the light-emitting side of the light-emitting element 102, and the second polarizing layer 110 may be disposed on the side of the multiple photoelectric sensing assemblies 103 away from the display panel 101. Here, the polarization direction of the second polarizing layer 110 may intersect with the polarization direction of the first polarizing layer 109.

[0132] The light emitted from the light-emitting element 102 first passes through the first polarizing layer 109 and can then be directed to the user's eye. The light reflected from the user's eye first passes through the second polarizing layer 110 and can then be directed to the photoelectric sensing assembly 103.

[0133] Here, the light emitted from the light emitting element 102 is converted into polarized light after passing through the first polarizing layer 109, and the polarized light is irradiated to the user's eye, where it is specularly reflected and diffusely reflected. The specularly reflected light and diffusely reflected light from the user's eye can be irradiated to the second polarizing layer 110.

[0134] The specularly reflected light is emitted parallel to one direction after being reflected, and the polarization direction of the second polarizing layer 110 crosses the polarization direction of the first polarizing layer 109, so the specularly reflected light at the user's eye cannot be transmitted through the second polarizing layer 110. The diffusely reflected light is emitted along each direction after being reflected, so the diffusely reflected light at the user's eye can be transmitted through the second polarizing layer 110, even though the polarization direction of the second polarizing layer 110 crosses the polarization direction of the first polarizing layer 109.

[0135] Therefore, the photoelectric sensing assembly 103 can only receive the light that is diffusely reflected by the user's eye, without receiving the light that is specularly reflected by the user's eye, that is, in the embodiment of the present application, the optical signal of the light-emitting element 102 that is reflected by the user's eye and received by the photoelectric sensing assembly 103 is the optical signal of the light-emitting element 102 that is diffusely reflected by the user's eye.

[0136] The photoelectric sensing assembly 103 converts the diffusely reflected optical signal of the light emitting element 102 into an electrical signal, and displays the electrical signal on the wearable display device. By Based on the electrical signal, the position of the gaze point of the user's eye on the display panel 101 is determined. The method according to the embodiment of the present application suppresses the specular reflection of the light emitted from the light emitting element 102 by the user's eye, thereby avoiding the influence of the specularly reflected light on determining the gaze point position and ensuring the accuracy of the determined gaze point position.

[0137] As an option, the polarization direction of the second polarizing layer 110 is perpendicular to the polarization direction of the first polarizing layer 109. Designing the polarization direction of the second polarizing layer 110 to be perpendicular to the polarization direction of the first polarizing layer 109 can further ensure that the light from the light-emitting element 102 specularly reflected by the user's eye does not pass through the second polarizing layer 110, and can prevent the photoelectric sensing assembly 103 from receiving the specularly reflected light and affecting the determination of the position of the gaze point.

[0138] As described above, the embodiments of the present application provide a wearable display device, and the wearable display device has high processing efficiency for the electrical signals transmitted from each photoelectric sensing assembly, so that the wearable display device can quickly determine the position of the user's eye gaze point on the display panel based on the electrical signals transmitted from each photoelectric sensing assembly, thereby improving the efficiency of image display on the display panel and achieving a high update rate for the display panel.

[0139] 15 is a flowchart of a method for determining the position of a gaze point according to an embodiment of the present application, which is used in a wearable display device according to the embodiment. 10 15, the method may include the following steps.

[0140] In step 201, optical signals of a plurality of light emitting elements reflected by a user's eye are received.

[0141] In an embodiment of the present application, the wearable display device 10 includes a display panel 101, a plurality of light-emitting elements 102, and a plurality of photoelectric sensing assemblies 103. Light emitted from the plurality of light-emitting elements 102 is used to illuminate a user's eyes. The display panel 101 has a display area 101a and a peripheral area 101b surrounding the display area 101a. A plurality of photoelectric sensing assemblies 103 may be disposed in the peripheral area 101b, and each photoelectric sensing assembly 103 can receive optical signals from the plurality of light-emitting elements 102 reflected by the user's eyes.

[0142] In step 202, the optical signal is converted to an electrical signal.

[0143] In the embodiment of the present application, each photoelectric sensing assembly 103 can receive an optical signal reflected by a user's eye, and then convert the received optical signal into an electrical signal.

[0144] In step 203, the position of the gaze point of the user's eye on the display panel is determined based on the magnitude of the signal value of the electrical signal and the position of the at least one photoelectric sensing assembly.

[0145] In an embodiment of the present application, the wearable display device 10 further includes a processor, which is connected to each photoelectric sensing assembly 103 and can receive the electrical signal transmitted from each photoelectric sensing assembly 103. After receiving the electrical signal transmitted from each photoelectric sensing assembly 103, the processor can determine the position of the gaze point of the user's eye on the display panel 101 based on the magnitude of the signal value of the electrical signal transmitted from each photoelectric sensing assembly 103 and the position of at least one photoelectric sensing assembly 103.

[0146] In an embodiment of the present application, the position of each photoelectric sensing assembly 103 may be pre-stored in the processor. Because different areas of the eye have different reflectivities to light (e.g., infrared light), the signal values ​​of the optical signals reflected by different areas of the eye received by the photoelectric sensing assemblies 103 are different. The magnitudes of the signal values ​​of the electrical signals converted by the photoelectric sensing assemblies 103 based on the different optical signals are different. Thus, the processor can determine the position of the user's eye gaze point on the display panel 101 based on the magnitudes of the signal values ​​of the electrical signals and the positions of the photoelectric sensing assemblies 103.

[0147] Generally, the amount of data of the electrical signal is small and the amount of data of the image is large, so that the processor can process the electrical signal with higher efficiency than the processor can process the image. In the embodiment of the present application, the processor can process the electrical signal transmitted from each photoelectric sensing assembly 103 with higher efficiency, and can quickly determine the position of the user's eye gaze point on the display panel 101, thereby improving the efficiency of image display on the display panel 101 and achieving a high update rate of the display panel 101.

[0148] As described above, the embodiments of the present application provide a method for determining the position of the gaze point, and the wearable display device has high processing efficiency for the electrical signals transmitted from each photoelectric sensing assembly, so that the wearable display device can quickly determine the position of the gaze point of the user's eyes on the display panel based on the electrical signals transmitted from each photoelectric sensing assembly, thereby improving the efficiency of image display on the display panel and increasing the update rate of the display panel.

[0149] 16 is a flowchart of another method for determining the position of a gaze point according to an embodiment of the present application. The method may be applied to the wearable display device 10 according to the above embodiment. Referring to FIG. 16, the method may include the following steps.

[0150] In step 301, a plurality of first photoelectric sensing assemblies and a plurality of second photoelectric sensing assemblies receive optical signals reflected by a user's eyes.

[0151] In an embodiment of the present application, the wearable display device 10 includes a display panel 101 and a plurality of photoelectric sensing assemblies 103. The display panel 101 has a display area 101a and a peripheral area 101b surrounding the display area 101a. A user typically sits on the display side of the display panel 101 to view an image displayed on the display panel 101. The plurality of photoelectric sensing assemblies 103 may be disposed on the display side of the display panel 101 and in the peripheral area 101b.

[0152] The display side of the display panel 101 is provided with a plurality of light-emitting elements 102, and light emitted from the plurality of light-emitting elements 102 can be irradiated onto a user's eyes. The user's eyes can reflect the light emitted from the plurality of light-emitting elements 102. Then, the light emitted from the plurality of light-emitting elements 102 can be reflected by the user's eyes and then irradiated onto a plurality of photoelectric sensing assemblies 103, so that the plurality of photoelectric sensing assemblies 103 can receive optical signals reflected by the user's eyes.

[0153] As an option, the plurality of photoelectric sensing assemblies 103 includes a plurality of first photoelectric sensing assemblies 103a arranged along a first direction X and a plurality of second photoelectric sensing assemblies 103b arranged along a second direction Y, where the plurality of first photoelectric sensing assemblies 103a and the plurality of second photoelectric sensing assemblies 103b can both receive optical signals diffusely reflected by a user's eye.

[0154] In step 302, each photoelectric sensing assembly of the plurality of first photoelectric sensing assemblies and the plurality of second photoelectric sensing assemblies converts a received optical signal into an electrical signal.

[0155] In the embodiment of the present application, after the plurality of first photoelectric sensing assemblies 103a and the plurality of second photoelectric sensing assemblies 103b receive optical signals, each photoelectric sensing assembly 103 can convert the received optical signals into electrical signals.

[0156] There is a positive correlation between the signal value of the electrical signal converted by the photoelectric sensing assembly 103 and the signal value of the optical signal received by the photoelectric sensing assembly 103. That is, the larger the signal value of the optical signal received by the photoelectric sensing assembly 103, the larger the signal value of the electrical signal obtained by converting the optical signal received by the photoelectric sensing assembly 103, and the smaller the signal value of the optical signal received by the photoelectric sensing assembly 103, the smaller the signal value of the electrical signal obtained by converting the optical signal received by the photoelectric sensing assembly 103. Here, the signal value of the optical signal represents the intensity of light.

[0157] In step 303, each photoelectric sensing assembly sends an electrical signal to a processor.

[0158] In an embodiment of the present application, a processor in the wearable display device 10 can be connected to each photoelectric sensing assembly 103. Each photoelectric sensing assembly 103 can send an electrical signal to the processor.

[0159] In step 304, the processor determines at least one target first photoelectric sensing assembly from the plurality of first photoelectric sensing assemblies.

[0160] In an embodiment of the present application, after receiving the electrical signals transmitted from the plurality of first photoelectric sensing assemblies 103a, the processor can determine at least one target first photoelectric sensing assembly from the plurality of first photoelectric sensing assemblies 103a, and can also determine the position of each target first photoelectric sensing assembly, for example, determine the coordinate value of each target first photoelectric sensing assembly.

[0161] Here, the signal value of the electrical signal sent from the target first photoelectric sensing assembly to the processor is equal to or less than a first threshold value. The first threshold value may be a fixed value pre-stored in the processor, or may be determined based on the signal values ​​of the electrical signals from the plurality of first photoelectric sensing assemblies 103a received by the processor.

[0162] For example, the processor may sort the signal values ​​of the N electrical signals transmitted from the N first photoelectric sensing assemblies 103a in ascending order and determine the signal value of the n-th bit as the first threshold value, where N is an integer greater than 1 and n is an integer greater than 1 and less than N / 2. Alternatively, the processor may sort the signal values ​​of the N electrical signals transmitted from the N first photoelectric sensing assemblies 103a in ascending order and determine the signal value of the n-th bit as the first threshold value, where N is an integer greater than 1 and less than N / 2. Sent from The smallest signal value of the electrical signal can be determined as the first threshold value.

[0163] If the first threshold value is the smallest signal value of the electrical signals transmitted from the plurality of first photoelectric sensing assemblies 103a, the processor can determine one target first photoelectric sensing assembly from the plurality of first photoelectric sensing assemblies 103a, thereby determining a first coordinate value of the target first photoelectric sensing assembly having the smallest signal value of the electrical signals transmitted from the plurality of first photoelectric sensing assemblies 103a.

[0164] As one option, the first coordinate value can be expressed as (first abscissa value, first ordinate value). The first abscissa value may be the coordinate value of the target first photoelectric sensing assembly in the first direction X, and the first ordinate value may be the coordinate value of the target first photoelectric sensing assembly in the second direction Y. Here, since multiple first photoelectric sensing assemblies 103a are arranged along the first direction X, the coordinate value of each first photoelectric sensing assembly 103a in the second direction Y may be 0. That is, the first ordinate value of the target first photoelectric sensing assembly may be 0.

[0165] FIG. 17 is a graph of an electrical signal of a first photoelectric sensing assembly according to an embodiment of the present application. FIG. 18 is a graph of an electrical signal of another first photoelectric sensing assembly according to an embodiment of the present application. Referring to FIGS. 17 and 18, the positions of each first photoelectric sensing assembly 103a from left to right are −4, −3, −2, −1, 0, 1, 2, 3, and 4, respectively. The abscissa in FIGS. 17 and 18 represents the position of each first photoelectric sensing assembly 103a, and the ordinate represents the signal value of the electrical signal. Ek in FIGS. 17 and 18 represents 10 to the minus kth power, for example, 5E−8 represents 5 multiplied by 10 to the minus eighth power.

[0166] Assuming that one target first photoelectric sensing assembly is determined from the plurality of first photoelectric sensing assemblies 103a by the processor, the signal value of the electrical signal transmitted from the first photoelectric sensing assembly 103a whose arrangement position is 0 in FIG. 17 is the smallest, and therefore, it can be determined that the first photoelectric sensing assembly 103a whose arrangement position is 0 is the target first photoelectric sensing assembly. Alternatively, the signal value of the electrical signal transmitted from the first photoelectric sensing assembly 103a whose arrangement position is -1 in FIG. 18 is the smallest, and therefore, it can be determined that the first photoelectric sensing assembly 103a whose arrangement position is -1 is the target first photoelectric sensing assembly. Now, referring to FIG. 19, nine first photoelectric sensing assemblies 103a are disposed in the first region 101b1 of the peripheral region 101b of the display panel 101.

[0167] In step 305, the processor determines at least one target second photoelectric sensing assembly from the plurality of second photoelectric sensing assemblies.

[0168] In an embodiment of the present application, after receiving the electrical signals transmitted from the plurality of second photoelectric sensing assemblies 103b, the processor can determine at least one target second photoelectric sensing assembly from the plurality of second photoelectric sensing assemblies 103b, and can also determine the position of each target second photoelectric sensing assembly, for example, determine the coordinate value of each target second photoelectric sensing assembly.

[0169] Here, the signal value of the electrical signal sent from the target second photoelectric sensing assembly to the processor is equal to or less than a second threshold value. The second threshold value may be a fixed value pre-stored in the processor, or may be determined based on the signal values ​​of the electrical signals from the plurality of second photoelectric sensing assemblies 103b received by the processor.

[0170] For example, the processor may sort the signal values ​​of the M electrical signals transmitted from the M second photoelectric sensing assemblies 103b in ascending order and determine the signal value of the m-th bit as the second threshold, where M is an integer greater than 1 and m is an integer greater than 1 and less than M / 2. Alternatively, the processor may sort the signal values ​​of the M electrical signals transmitted from the M second photoelectric sensing assemblies 103b in ascending order and determine the signal value of the m-th bit as the second threshold, where M is an integer greater than 1 and m is an integer greater than 1 and less than M / 2. Sent from The smallest signal value of the electrical signal can be determined as the second threshold value.

[0171] If the second threshold value is the smallest signal value of the electrical signal transmitted from the second photoelectric sensing assembly 103b, the processor can determine one target second photoelectric sensing assembly from the plurality of second photoelectric sensing assemblies 103b, thereby determining second coordinate values ​​of the target second photoelectric sensing assembly having the smallest signal value of the electrical signal transmitted from the plurality of second photoelectric sensing assemblies 103b.

[0172] As one option, the second coordinate value can be expressed as (second abscissa value, second ordinate value). The second abscissa value may be the coordinate value of the target second photoelectric sensing assembly in the first direction X, and the second ordinate value may be the coordinate value of the target second photoelectric sensing assembly in the second direction Y. Here, since multiple second photoelectric sensing assemblies 103b are arranged along the second direction Y, the coordinate value of each second photoelectric sensing assembly 103b in the first direction X may be 0. That is, the second abscissa value of the target second photoelectric sensing assembly may be 0.

[0173] FIG. 20 is a graph of an electrical signal transmitted from a second photoelectric sensing assembly according to an embodiment of the present application. FIG. 21 is a graph of an electrical signal transmitted from another second photoelectric sensing assembly according to an embodiment of the present application. Referring to FIGS. 20 and 21, the positions of each second photoelectric sensing assembly 103b are −4, −3, −2, −1, 0, 1, 2, 3, and 4, respectively, from top to bottom. The abscissa in FIGS. 20 and 21 represents the position of each second photoelectric sensing assembly 103b, and the ordinate represents the signal value of the electrical signal. Ek in FIGS. 20 and 21 represents 10 to the minus kth power, for example, 5E−8 represents 5 multiplied by 10 to the minus eighth power.

[0174] Assuming that one target second photoelectric sensing assembly is determined from the plurality of second photoelectric sensing assemblies 103b by the processor, the signal value of the electrical signal transmitted from the second photoelectric sensing assembly 103b whose arrangement position is -2 in FIG. 20 is the smallest, and therefore, it can be determined that the second photoelectric sensing assembly 103b whose arrangement position is -2 is the target second photoelectric sensing assembly. Alternatively, the signal value of the electrical signal transmitted from the second photoelectric sensing assembly 103b whose arrangement position is -3 in FIG. 21 is the smallest, and therefore, it can be determined that the second photoelectric sensing assembly 103b whose arrangement position is -3 is the target second photoelectric sensing assembly. Here, referring to FIG. 19, nine second photoelectric sensing assemblies 103b are disposed in the second region 101b2 of the peripheral region 101b of the display panel 101.

[0175] Note that the number of first photoelectric sensing assemblies 103a arranged in the first region 101b1 of the peripheral region 101b of the display panel 101 may be the same as the number of second photoelectric sensing assemblies 103b arranged in the second region 101b2. Of course, the number of first photoelectric sensing assemblies 103a arranged in the first region 101b1 may be different from the number of second photoelectric sensing assemblies 103b arranged in the second region 101b2. For example, since the length of the display panel 101 in the first direction X is usually greater than the length in the second direction Y, the number of first photoelectric sensing assemblies 103a arranged in the first region 101b1 may be greater than the number of second photoelectric sensing assemblies 103b arranged in the second region 101b2.

[0176] In step 306, the processor determines the position of the user's eye gaze point on the display panel based on the position of each target first photoelectric sensing assembly and the position of each target second photoelectric sensing assembly.

[0177] In an embodiment of the present application, after determining the position of each target first photoelectric sensing assembly and the position of each target second photoelectric sensing assembly, the processor can determine the position of the user's eye gaze point on the display panel 101 based on the position of each target first photoelectric sensing assembly and the position of each target second photoelectric sensing assembly.

[0178] In one possible embodiment, assuming that a plurality of target first photoelectric sensing assemblies have been determined by the processor, the processor can determine a first coordinate value of each of the plurality of target first photoelectric sensing assemblies. Here, the first coordinate value of each of the target first photoelectric sensing assemblies can both be expressed as (first abscissa value, first ordinate value). Then, the processor can determine a first horizontal average value of the first abscissa values ​​of the plurality of target first photoelectric sensing assemblies and a first vertical average value of the first ordinate values ​​of the plurality of target first photoelectric sensing assemblies.

[0179] Since the first ordinate value of each target first photoelectric sensing assembly is zero, the first ordinate average value of the first ordinates of the plurality of target first photoelectric sensing assemblies is also zero.

[0180] Accordingly, assuming that the processor has determined a plurality of target second photoelectric sensing assemblies, the processor can determine a second coordinate value of each target second photoelectric sensing assembly among the plurality of target second photoelectric sensing assemblies. Here, the second coordinate value of each target second photoelectric sensing assembly can be expressed as (second abscissa value, second ordinate value). Then, the processor can determine a second horizontal average value of the second abscissa values ​​of the plurality of target second photoelectric sensing assemblies and a second vertical average value of the second ordinates of the plurality of target second photoelectric sensing assemblies.

[0181] Since the second abscissa value of each target second photoelectric sensing assembly is zero, the second average value of the second abscissas of the plurality of target second photoelectric sensing assemblies is also zero.

[0182] Then, based on the first horizontal average value and the second vertical average value, the processor can determine the position of the gaze point of the user's eyes on the display panel 101. For example, the position of the gaze point can be represented by the coordinates of the gaze point (first horizontal average value, second vertical average value).

[0183] In another possible embodiment, assuming that one target first photoelectric sensing assembly is determined by the processor, the processor can determine a first coordinate value of the target first photoelectric sensing assembly, where the first coordinate value of the target first photoelectric sensing assembly can both be expressed as (first abscissa value, first ordinate value).

[0184] Correspondingly, assuming that one target second photoelectric sensing assembly is determined by the processor, the processor can determine the second coordinate value of the target second photoelectric sensing assembly, where the second coordinate value of the target second photoelectric sensing assembly can both be expressed as (second abscissa value, second ordinate value).

[0185] Then, based on the first abscissa value and the second ordinate value, the processor can determine the position of the gaze point of the user's eyes on the display panel 101. For example, the position of the gaze point can be represented by the coordinates of the gaze point (first abscissa value, second ordinate value).

[0186] For example, referring to Figures 17 and 20, the position a of the gaze point may be represented by the coordinates (0, -2) of the gaze point, or referring to Figures 18 and 21, the position b of the gaze point may be represented by the coordinates (-1, -3) of the gaze point.

[0187] After the processor determines the position of the gaze point, it renders an image to be displayed on the display panel 101 based on the position of the gaze point, and transmits the rendered image to be displayed to a drive circuit of the display panel 101, so that the drive circuit can display the rendered image to be displayed. of Alternatively, after determining the position of the gaze point, the processor can transmit the position of the gaze point to another processor. The other processor renders an image to be displayed on the display panel 101 based on the position of the gaze point, and transmits the rendered image to be displayed to a drive circuit of the display panel 101, so that the drive circuit can display the rendered image. of The display panel 101 is driven to display the image.

[0188] Here, rendering the image to be displayed may be rendering a portion of the area where the point of gaze is located in the image to be displayed. Here, the area where the point of gaze is located may refer to a target area centered on the point of gaze. The shape of the target area may be circular or rectangular, and the size of the target area may be a size pre-stored in the processor.

[0189] It should be noted that the priority of steps in the method for determining the position of the gaze point according to the embodiment of the present application can be appropriately adjusted, and steps can be increased or decreased according to circumstances. For example, step 305 can be performed synchronously with step 304, or step 305 can be performed before step 304. Methods that can be easily conceived by any person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application and will not be further described.

[0190] As described above, the embodiments of the present application provide a method for determining the position of the gaze point, and the wearable display device has high processing efficiency for the electrical signals transmitted from each photoelectric sensing assembly, so that the wearable display device can quickly determine the position of the gaze point of the user's eyes on the display panel based on the electrical signals transmitted from each photoelectric sensing assembly, thereby improving the efficiency of image display on the display panel and increasing the update rate of the display panel.

[0191] In an embodiment of the present application, a computer-readable storage medium is provided, and instructions are stored in the computer-readable storage medium, and the instructions are executed by a wearable display device to realize a method for determining the position of a gaze point according to the above method embodiment.

[0192] In an embodiment of the present application, a computer program product including instructions is provided, which, when executed on a computer, causes the computer to perform the method for determining the position of a gaze point according to the above method embodiment.

[0193] The above are only optional examples of the present application, and do not limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included within the protection scope of the present application.

Claims

1. a display panel having a display area and a peripheral area surrounding the display area; a plurality of light-emitting elements that emit light to be irradiated onto the user's eyes; a plurality of photoelectric sensing assemblies disposed in the peripheral region, each of which is used to receive optical signals of the plurality of light-emitting elements reflected by the user's eyes and convert the optical signals into electrical signals, the electrical signals being used to determine the position of the user's eye gaze point on the display panel; an optical structure disposed on a side of the photoelectric sensing assembly away from the display panel, the orthogonal projection of which onto the display panel is located in the peripheral region, the optical structure having a light-shielding region and a plurality of light-transmitting regions, the plurality of light-transmitting regions corresponding one-to-one to the plurality of photoelectric sensing assemblies, each of the light-transmitting regions being used to transmit the optical signal to at least one corresponding one of the photoelectric sensing assemblies; Including, the peripheral region includes a first region extending along a first direction and a second region extending along a second direction, the first direction intersecting the second direction, and the plurality of photoelectric sensing assemblies includes a plurality of first photoelectric sensing assemblies and a plurality of second photoelectric sensing assemblies; wherein the plurality of first photoelectric sensing assemblies are arranged in the first region and along the first direction, and the plurality of second photoelectric sensing assemblies are arranged in the second region and along the second direction; A wearable display device comprising:

2. the optical structure includes a first branch structure extending along the first direction and a second branch structure extending along the second direction; the first branch structure has a plurality of first light-transmitting areas corresponding one-to-one to the plurality of first photoelectric sensing assemblies, and an orthogonal projection of each of the first light-transmitting areas onto the display panel at least partially overlaps with an orthogonal projection of a corresponding one of the first photoelectric sensing assemblies onto the display panel; the second branch structure has a plurality of second light-transmitting areas corresponding one-to-one to the plurality of second photoelectric sensing assemblies, and an orthogonal projection of each of the second light-transmitting areas onto the display panel at least partially overlaps with an orthogonal projection of a corresponding one of the second photoelectric sensing assemblies onto the display panel; The wearable display device of claim 1 .

3. the first direction is perpendicular to the second direction, and the peripheral region includes two first regions and two second regions; wherein the two first regions are arranged along the second direction and are disposed on both sides of the display region, and the two second regions are arranged along the first direction and are disposed on both sides of the display region. The wearable display device of claim 1 .

4. The wearable display device of claim 3, wherein the optical structure includes two first branch structures that correspond one-to-one to the two first regions and two second branch structures that correspond one-to-one to the two second regions.

5. The detection width of the photoelectric sensing assembly along the target direction and the width of the light-transmitting region along the target direction are positively correlated; a detection width of the photoelectric sensing assembly along the target direction and a width of the photoelectric sensing assembly along the target direction are positively correlated; wherein the target direction is the first direction or the second direction. The wearable display device of claim 1 .

6. The detection width h of the photoelectric sensing assembly along the target direction is: [Equation 1] Fulfilling where u is the distance between the user's eye and the optical structure, v is the distance between the optical structure and the plurality of photoelectric sensing assemblies, n1 is the refractive index of the medium between the optical structure and the plurality of photoelectric sensing assemblies, n2 is the refractive index of the medium between the user's eye and the optical structure, p is the width of the photoelectric sensing assembly along the target direction, and s is the width of the light-transmitting region along the target direction. The wearable display device according to claim 5 .

7. Each of the light-transmitting regions is a circular through-hole; Alternatively, each of the light-transmitting regions is a rectangular through-hole. The wearable display device of claim 1 .

8. a first side of each of the rectangular through holes among the plurality of rectangular through holes is parallel to a first direction and a second side of each of the rectangular through holes is parallel to a second direction, and the plurality of rectangular through holes includes a plurality of first rectangular through holes and a plurality of second rectangular through holes; the first rectangular through hole is located in a first region extending along the first direction in the peripheral region, and a length of a first side of the first rectangular through hole is smaller than a length of a second side of the first rectangular through hole; the second rectangular through hole is located in a second region extending along the second direction in the peripheral region, and a length of a first side of the second rectangular through hole is greater than a length of a second side of the second rectangular through hole; The wearable display device of claim 7 .

9. further comprising a light-transmitting layer; the light-transmitting layer is disposed between the plurality of photoelectric sensing assemblies and the optical structure; The wearable display device of claim 1 .

10. Further comprising a filter, the filter is disposed on a side of the plurality of photoelectric sensing assemblies away from the display panel, and an orthogonal projection of the filter onto the display panel obscures an orthogonal projection of the plurality of photoelectric sensing assemblies onto the display panel, and the filter is for transmitting infrared light and absorbing visible light. The wearable display device of claim 1 .

11. Further comprising a lens and a lens frame, the lens is disposed on a display side of the display panel, and the lens frame is disposed on an edge of the lens. The wearable display device of claim 1 .

12. The wearable display device according to claim 11 , wherein the plurality of light-emitting elements are arranged on a side of the lens frame away from the display panel and connected to the lens frame.

13. further comprising a first polarizing layer and a second polarizing layer; The first polarizing layer is disposed on a light-emitting side of the light-emitting element, and the second polarizing layer is disposed on a side of the plurality of photoelectric sensing assemblies away from the display panel, and the polarization direction of the second polarizing layer intersects with the polarization direction of the first polarizing layer. The wearable display device of claim 1 .

14. The wearable display device of claim 13 , wherein the polarization direction of the second polarizing layer is perpendicular to the polarization direction of the first polarizing layer.

15. The wearable display device according to any one of claims 1 to 14, receiving light signals from the plurality of light emitting elements reflected by the user's eyes; converting the optical signal into an electrical signal; determining a position of a user's eye gaze point on the display panel based on a signal value of the electrical signal and a position of the at least one photoelectric sensing assembly; Including, A method for determining the position of a gaze point, characterized by:

16. Determining a position of a gaze point of a user's eye on a display panel based on the signal value of the electrical signal and the position of at least one photoelectric sensing assembly includes: determining at least one target first photoelectric sensing assembly from a plurality of first photoelectric sensing assemblies arranged along a first direction; determining at least one target second photoelectric sensing assembly from a plurality of second photoelectric sensing assemblies arranged along a second direction; determining a position of a user's eye gaze point on the display panel based on the position of each of the target first photoelectric sensing assemblies and the position of each of the target second photoelectric sensing assemblies; Including, wherein the signal value of the electrical signal transmitted from the target first photoelectric sensing assembly is equal to a first threshold value, the first threshold value being the smallest signal value among the signal values ​​of the electrical signals transmitted from the plurality of first photoelectric sensing assemblies; and the signal value of the electrical signal transmitted from the target second photoelectric sensing assembly is equal to a second threshold value, the second threshold value being the smallest signal value among the signal values ​​of the electrical signals transmitted from the plurality of second photoelectric sensing assemblies.

16. The method of claim 15.

17. 16. A computer-readable storage medium having instructions stored thereon, the instructions being executed by a display device to implement the determination method of claim 15.

Images