Electronic device and method for gaze tracking

Abstract

An electronic device and a method for gaze tracking are provided. The electronic device according to an embodiment disclosed herein generates a plurality of light beams through a polarization filter and a polarization changing element or via a plurality of light sources, and obtains an encoded image through a superlens that refracts the generated plurality of light beams into different paths. The electronic device may acquire line-of-sight information about a user based on the encoded image and track a line-of-sight of the user.

Description

Technical Field

[0001] This disclosure relates to an electronic device for eye tracking and a method of operating the same. More specifically, this disclosure relates to an electronic device for tracking a gaze based on an eye image distorted by transmission through a multiplexed-metalens, and a method of operating the electronic device, wherein the multiplexed-metalens refract incident light into different paths according to optical properties. Background Technology

[0002] Augmented reality is a technology that overlays virtual images onto physical environments or real-world objects to display them together, and augmented reality devices (such as smart glasses) that use augmented reality technology are used in everyday life, for example, for information retrieval, providing directions, and taking pictures.

[0003] In particular, with the development of image sensors and AI-based image processing technologies, the use of cameras has expanded from simple shooting to object recognition, biometrics, and information gathering. While camera pixel resolution continues to increase for various applications, there are physical limitations in reducing the size and cost of camera modules.

[0004] Recently, lensless imaging technology, which uses thin phase masks instead of lenses, has attracted attention. A phase mask is a very thin optical element used to modulate the phase of incident light, and lensless imaging technology is a scheme to photograph a subject by using a thin phase mask that modulates light instead of a lens. Lensless imaging technology can reduce the thickness and focal length of lens modules, and therefore can create ultra-thin cameras that exceed the physical limitations of existing cameras.

[0005] In lensless imaging technology, the most crucial factor determining camera performance is the structure of the phase mask. Intricate curved structures can be designed for different phase masks. In metalens, a type of phase mask, intricate curved structures can be designed by arranging relatively small nanostructures in a two-dimensional (2D) manner. In particular, multiplexed metalens can be used to provide various focal lengths or process light of various wavelengths within a single lens, and such devices can help miniaturize camera modules and improve their durability. Summary of the Invention

[0006] Solution to the problem According to one aspect of this disclosure, an electronic device includes: a polarizing filter configured to receive incident light and change the polarization of the incident light; a polarization changing element configured to receive light from the polarizing filter; a superlens configured to refract light received from the polarization changing element into different paths according to the polarization direction of the light received from the polarization changing element; an image sensor configured to receive refracted light from the superlens; at least one processor; and at least one memory storing at least one instruction, wherein the at least one instruction, when executed by the at least one processor, causes the electronic device to: cause the polarizing filter to change the incident light into a first polarized light in a first direction and transmit the first polarized light; obtain a first coded image by the image sensor based on the first polarized light refracted into a first path by the superlens; and, based on the identification that user gaze information cannot be obtained from the first coded image: cause the polarization changing element to change the first polarized light into a second polarized light in a second direction and transmit the second polarized light; obtain a second coded image by the image sensor based on the second polarized light refracted into a second path by the superlens; and obtain the user's gaze information from the second coded image.

[0007] According to one aspect of this disclosure, an electronic device includes: a first light source configured to output first light; a second light source configured to output second light; a superlens configured to refract incident light into different paths according to the wavelength of the incident light; an image sensor configured to receive light refracted by the superlens; at least one processor; and at least one memory storing at least one instruction, wherein the at least one instruction, when executed by the at least one processor, causes the electronic device to: acquire a first coded image through the image sensor, wherein the first coded image is based on the first light refracted into a first path by the superlens; and based on identifying that user gaze information cannot be obtained from the first coded image, acquire the user's gaze information from a second coded image acquired through the image sensor, wherein the second coded image is based on the second light refracted into a second path by the superlens.

[0008] According to one aspect of this disclosure, a method includes: obtaining a first coded image by receiving first polarized light refracted into a first path by a superlens, wherein the superlens is configured to refract incident light into different paths according to the polarization direction of the incident light; identifying whether a user's gaze information can be obtained from the first coded image; based on the identification that the user's gaze information cannot be obtained from the first coded image: obtaining a second coded image by receiving second polarized light refracted into a second path by the superlens; and obtaining the user's gaze information from the second coded image. Attached Figure Description

[0009] This disclosure can be readily understood by referring to the following detailed description and accompanying drawings, wherein reference numerals denote structural elements.

[0010] Figure 1 This is a conceptual diagram illustrating an operation performed by an electronic device according to an embodiment of the present disclosure, based on tracking a line of sight by receiving an coded image obtained by receiving light transmitted by a multiplexed superlens; Figure 2 This is a conceptual diagram illustrating gaze tracking based on encoded images performed by an electronic device according to an embodiment of the present disclosure; Figure 3 This is a flowchart of an operation method of an electronic device according to an embodiment of the present disclosure; Figure 4 This is a flowchart of an operation method of an electronic device according to an embodiment of the present disclosure; Figure 5 This is a flowchart of an operation method of an electronic device according to an embodiment of the present disclosure; Figure 6 This is a conceptual diagram illustrating an operation performed by an electronic device according to an embodiment of the present disclosure, based on tracking a gaze by receiving an coded image obtained using light generated by a light source and a multiplexed superlens; Figure 7 This is a conceptual diagram illustrating an operation performed by an electronic device according to an embodiment of the present disclosure, based on tracking a line of sight by receiving an coded image obtained by receiving at least one of a plurality of lights generated using a plurality of light sources and a plurality of multiplexed superlenses; Figure 8 This is a flowchart of a method for tracking a line of sight based on using multiple light sources and multiple lights generated by multiplexed superlenses, executed by an electronic device according to an embodiment of the present disclosure; Figure 9 This is a conceptual diagram illustrating an operation performed by an electronic device according to an embodiment of the present disclosure, based on an coded image obtained by receiving light output from multiple light sources to track a gaze. Figure 10a This is a conceptual diagram used to explain the operation of focusing a first wavelength of light output from a light source onto an image sensor; Figure 10b This is a conceptual diagram used to explain the operation of focusing a second wavelength of light output from a light source onto an image sensor; Figure 11 This is a flowchart of a method for tracking a gaze by extracting feature points based on an coded image, performed by an electronic device according to an embodiment of the present disclosure; Figure 12 This is a flowchart of a method for tracking a gaze by reconstructing an image based on an encoded image, performed by an electronic device according to embodiments of the present disclosure; and Figure 13 This is a block diagram of the structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0011] While widely used, generic terms have been chosen to describe this disclosure in consideration of its functionality, these generic terms may vary depending on the intent of those skilled in the art, precedents, the emergence of new technologies, etc. Terms arbitrarily chosen by the applicant of this disclosure may also be used in specific circumstances. In such cases, their meanings need to be given in the detailed description of the embodiments of this disclosure. Therefore, terms must be defined based on their meanings and the entirety of the specification, rather than by simply stating the terms.

[0012] Unless a singular expression has a distinct meaning in the context, it may contain plural expressions. Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0013] The terms “comprising” and / or “including” or “containing” and / or “containing” as used herein specify the presence of a stated element, but do not exclude the presence or addition of one or more other elements. When used in this specification, the terms “unit,” “device,” and “module” refer to a unit that performs at least one function or operation and can be implemented as hardware, software, or a combination of hardware and software.

[0014] Depending on the context, the expression "configured as (or set to)" may be used interchangeably with, for example, "suitable for," "capable of," "designed for," "suitable for," "manufactured as," or "capable of." The expression "configured as (or set to)" does not necessarily refer to something "specifically designed for" in terms of hardware. Rather, in some cases, the expression "the system is configured as" may refer to a situation where the system, together with another device or component, is "capable of." For example, the phrase "processors configured (or set) to perform A, B, and C" may refer to a dedicated processor (such as an embedded processor) for performing the respective operations, or a general-purpose processor (such as a central processing unit (CPU) or application processor (AP)) that performs the respective operations by executing one or more software programs stored in memory.

[0015] When an element (e.g., a first element) is "bonded" or "connected" to another element (e.g., a second element), the first element may be directly bonded to or connected to the second element, or, unless otherwise stated, a third element may be present between the first element and the second element.

[0016] This document describes embodiments of the present disclosure in detail with reference to the accompanying drawings, so that those skilled in the art to which this disclosure pertains can easily implement it. However, this disclosure may be implemented in many different forms and should not be construed as limited to the examples set forth herein.

[0017] Embodiments of this disclosure will now be described more fully below with reference to the accompanying drawings.

[0018] Figure 1 This is a conceptual diagram illustrating an operation performed by an electronic device according to an embodiment of the present disclosure, based on tracking a line of sight by receiving an coded image obtained by receiving light transmitted through a multiplexed superlens.

[0019] Reference Figure 1 The electronic device may include a polarizing filter 110, a polarization changing element 120, a multiplexed superlens 130, an image sensor 140, and a processor 150.

[0020] According to embodiments of this disclosure, polarizing filter 110 may transmit only a first polarized light PL1 with a specific polarization in the external incident light IL. Polarizing filter 110 may include a polarization axis X1 for polarizing the external incident light IL in a specific direction. Polarizing filter 110 may transmit light components in the external incident light IL that vibrate in a direction along the polarization axis X1, and may block light components that vibrate in a direction different from the direction along the polarization axis X1.

[0021] For example, polarizing filter 110 may transmit only vertically polarized light from the external incident light IL or only horizontally polarized light from the external incident light IL. The vertically polarized light and the horizontally polarized light may each be light polarized in a direction perpendicular to the direction of travel of the external incident light IL.

[0022] According to the disclosed embodiments, the external incident light IL can be light vibrating in all directions perpendicular to the direction of travel of the external incident light IL, and can be, for example, natural light. The external incident light IL passes through a polarizing filter 110, which generates a first horizontally polarized light PL1 by polarizing the incident light IL in the horizontal direction. Although Figure 1 The first polarized light PL1, which has only horizontal polarization, is shown to be transmitted through the polarization filter 110, but this disclosure is not limited thereto.

[0023] According to embodiments of this disclosure, the external incident light IL can be light reflected from object 10. For example, polarization filter 110 can transmit only vertically polarized light within the incident light IL reflected from object 10. For example, object 10 can be as follows: Figure 1 The image shows an eye, and the processor 150 can track the user's gaze based on the light reflected from the eye.

[0024] According to embodiments of this disclosure, the polarization changing element 120 can be configured to transmit first polarized light PL1 in a first direction transmitted by the polarization filter 110, or to change the first polarized light PL1 in the first direction to second polarized light PL2 in a second direction. When linearly polarized light is incident perpendicularly onto a plane, the polarization changing element 120 can change the phase of the traveling wave differently. For example, the polarization changing element 120 can generate second polarized light PL2 with vertical polarization from the first polarized light PL1 with horizontal polarization. The polarization changing element 120 can change the polarization form of the incident light. The polarization changing element 120 can transmit the first polarized light PL1 with horizontal polarization from the first polarized light PL1 with horizontal polarization without making a change.

[0025] When linearly polarized light in one direction is incident perpendicularly on the front surface of the polarization changing element 120, the polarization changing element 120 can selectively generate a first polarized light PL1 linearly polarized in the first direction or a second polarized light PL2 linearly polarized in the second direction. According to embodiments of this disclosure, the first and second directions can be perpendicular to each other.

[0026] However, the number of polarization directions that the polarization changing element 120 can select for polarized light is merely an example, and this disclosure is not limited thereto. For example, the processor 150 can control the polarization changing element 120 to selectively generate linearly polarized light in one of the directions from a first direction to a third direction.

[0027] According to embodiments of this disclosure, the polarization changing element 120 can be an element that changes the direction of polarization of incident light in response to external influences (such as mechanical forces or electromagnetic fields). According to embodiments of this disclosure, the polarization changing element 120 may include a polarization cell and a polarization direction controller. For example, the polarization cell can be configured such that light incident on the polarization changing element 120 is transmitted and the polarization direction of the incident light is changed. The polarization cell can be a single-layer film or a rectangular shaped box. For example, the polarization direction controller can apply pressure to the polarization cell to control the polarization direction changed by the polarization cell with respect to incident light. The pressure can be a mechanical force, an electromagnetic field, etc.

[0028] According to embodiments of this disclosure, the second polarized light PL2 generated by the polarization changing element 120 can be incident toward the multiplexed superlens 130. The incident light IL modified by the polarization filter 110 and the polarization changing element 120, as well as the second polarized light PL2 generated by the polarization filter 110 and the polarization changing element 120, can include information about the object 10.

[0029] According to embodiments of this disclosure, the multiplexed superlens 130 can be a lens in which nanostructures are arranged in a two-dimensional (2D) manner. The multiplexed superlens 130 can be configured to modulate the phase of transmitted light. The multiplexed superlens 130 can alter the path of light by modulating the phase of transmitted light. The detailed arrangement shape of the nanostructures included in the multiplexed superlens 130 can vary depending on the desired optical performance of the multiplexed superlens 130.

[0030] According to embodiments of this disclosure, the shape of each of the nanostructures included in the multiplexed superlens 130 can have a relatively strong response to incident light having specific optical properties. The nanostructures included in the multiplexed superlens 130 can alter the path or phase of transmitted light by influencing the vibrational mode of the transmitted light. However, the principle of altering the path or phase of light does not limit this disclosure.

[0031] For example, a nanostructure of a certain shape can strongly respond to incident light with a wavelength of 940 nm, and can modify the incident light with a wavelength of 940 nm so that the incident light can pass through the multiplexed superlens 130 and propagate in a specific direction. Optionally, a nanostructure of a certain shape can modulate the incident light with a wavelength of 940 nm so that the incident light can pass through the multiplexed superlens 130 and propagate in a specific phase. Simultaneously, a nanostructure of a certain shape can strongly respond to incident light with a wavelength of 850 nm, and can modify the incident light with a wavelength of 850 nm so that the incident light can pass through the multiplexed superlens 130 and propagate in another direction.

[0032] According to embodiments of the present disclosure, the multiplexed superlens 130 can be a lens that alters transmitted light to travel along different paths based on optical properties. When the multiplexed superlens 130 alters transmitted light to travel along different paths, an electronic device according to embodiments of the present disclosure can provide a camera module that achieves various focal lengths based on transmitted light by using the multiplexed superlens 130.

[0033] For example, light with a first wavelength can be incident on the multiplexing superlens 130, and the multiplexing superlens 130 can guide the light with the first wavelength to focus at a first point on the imaging surface of the image sensor 140. Light with a second wavelength can be incident on the multiplexing superlens 130, and the multiplexing superlens 130 can guide the light with the second wavelength to focus at a second point on the imaging surface of the image sensor 140. The first point may be different from the second point. In other words, the multiplexing superlens 130 can change the incident light to travel along different paths according to the wavelength of the incident light, and can provide different focal lengths according to the incident light.

[0034] As another example, vertically polarized light PL1 may be incident toward the multiplexing superlens 130, and the multiplexing superlens 130 may guide the vertically polarized light PL1 to focus at a first point on the imaging surface of the image sensor 140. Horizontally polarized light may be incident toward the multiplexing superlens 130, and the multiplexing superlens 130 may guide the horizontally polarized light to focus at a second point on the imaging surface of the image sensor 140. The first point may be different from the second point. In other words, the multiplexing superlens 130 may change the incident light to travel along different paths depending on the polarization direction of the incident light, and may provide different focal lengths depending on the incident light.

[0035] However, the multiplexing superlens 130 can alter the incident light to travel along different paths depending on the optical properties of the incident light, and the wavelength and polarization direction described as examples of optical properties are merely examples and do not limit the present disclosure. According to embodiments of the present disclosure, the multiplexing superlens 130 may include a surface formed of nanostructures disposed on a substrate, and the path of transmitted light may vary depending on the arrangement of the nanostructures of the multiplexing superlens 130 and the shape of each of the nanostructures. For example, the characteristics of the multiplexing superlens 130 may vary depending on the height, shape, arrangement, and properties of the nanostructures formed on the multiplexing superlens 130.

[0036] The nanostructures included in the multiplexing superlens 130 can guide polarized light incident on the multiplexing superlens 130 to any focal point according to the arrangement shape of the nanostructures. "Arrangement shape" refers to at least one of the following: the size of each nanostructure, the shape of each nanostructure, the spacing between the nanostructures, a size distribution according to the position of the nanostructures relative to the region where the multiplexing superlens 130 is located, a shape distribution according to the position of the nanostructures, or a spacing distribution according to the position of the nanostructures. The detailed arrangement shape of the nanostructures included in the multiplexing superlens 130 can vary according to the desired optical performance of the multiplexing superlens 130. For example, the arrangement shape of the nanostructures can vary according to the wavelength range, focal length, etc., of the light expected to be collected by the multiplexing superlens 130.

[0037] According to embodiments of this disclosure, an electronic device can obtain an encoded image by receiving second polarized light PL2 transmitted by a multiplexed superlens 130 using an image sensor 140. The electronic device can obtain an encoded image 20 corresponding to the object 10 based on the distribution of the second polarized light PL2 transmitted by the multiplexed superlens 130.

[0038] Light transmitted by the multiplexed superlens 130 is received by the image sensor 140. For example, the image sensor 140 may be a 2D sensor assembled in an array, comprising a plurality of pixels arranged in a matrix, and each of the pixels may include at least one photoelectric conversion element. The image sensor 140 can sense light using the photoelectric conversion elements and output an image signal as an electrical signal based on the sensed light. Electronic devices can obtain an encoded image 20 by converting the light received via the image sensor 140 into an electrical signal.

[0039] According to embodiments of this disclosure, the multiplexed superlens 130 is configured to refract incident light via different paths based on optical properties. For example, vertically polarized light can be guided to be transmitted through the multiplexed superlens 130 and focused at a first point on the imaging surface of the image sensor 140. As another example, horizontally polarized light can be guided to be transmitted through the multiplexed superlens 130 and focused at a second point on the imaging surface of the image sensor 140. An electronic device can obtain an encoded image 20 by converting light received via each point on the imaging surface of the image sensor 140 into an electrical signal. Figure 2 This function of reusing the superlens 130 is easy to understand.

[0040] The encoded image 20 may vary depending on at least one of the object 10 reflecting the light, the arrangement shape of the nanostructures included in the multiplexed superlens 130, or the optical properties of the light refracted by the multiplexed superlens 130. Typically, the object represented in the encoded image 20, which is phase-modulated by the multiplexed superlens 130, may be difficult to identify with the naked eye.

[0041] According to embodiments of this disclosure, an electronic device can obtain a user's gaze information from an encoded image 20 using a processor 150. The electronic device can then track the user's gaze direction based on this gaze information.

[0042] According to embodiments of this disclosure, an electronic device can obtain feature points from an encoded image 20 using a processor 150. The processor 150 may include an artificial intelligence (AI) algorithm or AI network for obtaining the feature points. The AI ​​algorithm included in the processor 150 may be an algorithm trained to extract feature points from a modulated image (i.e., the encoded image 20). The electronic device can track the user's gaze direction based on the feature points extracted using the AI ​​algorithm.

[0043] For example, the feature point may include at least one of a pupil feature point or an eye reflection feature point. The electronic device may extract at least one of the pupil feature point or eye reflection feature point from the coded image 20. The electronic device may track the user's gaze direction based on at least one of the pupil feature point or eye reflection feature point extracted using an AI algorithm.

[0044] According to embodiments of this disclosure, an electronic device can recover an image from an encoded image 20 using a processor 150. The processor 150 may include an AI algorithm or AI network for image recovery. The AI ​​algorithm included in the processor 150 may be an algorithm trained to recover an image from a modulated image (i.e., from the encoded image 20). The electronic device can track the user's gaze direction based on the image recovered using the AI ​​algorithm.

[0045] Figure 2 This is a conceptual diagram illustrating gaze-tracking processing based on encoded images performed by an electronic device according to an embodiment of the present disclosure.

[0046] For ease of explanation, the references above will be briefly given or omitted. Figure 1 A repetitive description of the described items. Figure 2 The first polarized light PL1 shown can be reflected by an object (e.g., an eye) and by... Figure 1 The polarization filter 110 polarizes the light.

[0047] For reference only. Figure 2 The superlenses 131 and 132 shown may have the same characteristics as those used above. Figure 1 The superlens 130 described has the same configuration. Figure 2 The image sensors 141 and 142 shown may have the same features as those used above. Figure 1 The image sensor 140 described has the same configuration.

[0048] According to embodiments of this disclosure, an electronic device can generate a second polarized light PL2 with horizontal polarization from a first polarized light PL1 with vertical polarization using a polarization changing element 121. The generated second polarized light PL2 can be incident towards a multiplexing superlens 131. The electronic device can obtain a first coded image 21 by receiving the light transmitted by the multiplexing superlens 131 using an image sensor 141. However, according to embodiments of this disclosure, when the light transmitted by the multiplexing superlens 131 is focused outside the imaging surface 145 of the image sensor 141 according to the area of ​​the image sensor 141, the obtained first coded image 21 may not include information about an object (e.g., an eye). Therefore, the electronic device may not be able to obtain gaze information based on the obtained first coded image 21.

[0049] According to embodiments of this disclosure, the electronic device can determine whether user gaze information can be obtained from the acquired first coded image 21. In operation S10, when user gaze information cannot be obtained from the acquired first coded image 21, the electronic device can control the polarization changing element 122 to transmit vertically polarized light PL1 without modification.

[0050] According to embodiments of this disclosure, an electronic device can control polarization changing element 122 to transmit a first polarized light PL1 having vertical polarization without alteration. For example, polarization changing elements 121 and 122 can generate polarized light PL1 and PL2, each having at least two optical properties, from the first polarized light PL1 incident via an on / off operation.

[0051] The transmitted first polarized light PL1 may be incident toward the multiplexing superlens 132. An electronic device can obtain a second coded image 22 by receiving the light transmitted by the multiplexing superlens 132 using an image sensor 142. However, according to embodiments of this disclosure, when the light transmitted by the multiplexing superlens 132 is focused onto the imaging surface 146 of the image sensor 142 according to a region of the image sensor 142, the obtained second coded image 22 may include information about an object (e.g., an eye). Therefore, the electronic device can obtain gaze information based on the obtained second coded image 22.

[0052] The electronic device can obtain the user's gaze information from the second coded image 22. The electronic device can track the user's gaze based on the second coded image 22.

[0053] Figure 3 This is a flowchart of an operation method of an electronic device according to an embodiment of the present disclosure.

[0054] In operation S310, the electronic device can obtain a first coded image by receiving first polarized light refracted in a first path by a multiplexed superlens.

[0055] According to embodiments of this disclosure, an electronic device can generate first polarized light vibrating in one direction from external incident light. The external incident light can be light reflected from an object (e.g., from an eye). The external incident light can be light vibrating in all directions perpendicular to the direction of light travel. The electronic device can generate first polarized light in one direction from external incident light by allowing the external incident light to pass through a polarizing filter. The polarizing filter can be configured to transmit the first polarized light in one direction from the incident light.

[0056] According to embodiments of this disclosure, the first polarized light in one direction can be light vibrating in a direction perpendicular to the direction of travel. For example, the first polarized light in one direction can be a polarization component with horizontal polarization or a polarization component with vertical polarization.

[0057] According to embodiments of this disclosure, first polarized light in one direction generated by a polarizing filter can be transmitted through a polarization-changing element. An electronic device can change the polarization form of the first polarized light in one direction by using the polarization-changing element. The polarization-changing element can be configured to change the polarization form of the first polarized light transmitted by the polarizing filter.

[0058] For example, when linearly polarized light is incident perpendicularly onto a plane, the polarization-changing element can modulate the phase of the traveling wave. The polarization-changing element can generate a second polarized light with perpendicular polarization from a first polarized light in one direction. Optionally, the polarization-changing element can transmit the first polarized light in one direction without altering its polarization.

[0059] According to embodiments of this disclosure, a multiplexing superlens can be configured to refract incident light into different paths based on optical properties. For example, a multiplexing superlens can be configured to refract incident light into different paths based on the polarization direction of the incident light. A multiplexing superlens can refract first polarized light into a first path. A multiplexing superlens can refract second polarized light into a second path.

[0060] Multiplexed superlenses can be lenses composed of nanostructures arranged in a 2D manner. Multiplexed superlenses can be configured to modulate the phase of transmitted light. Multiplexed superlenses can alter the path of light by modulating the phase of transmitted light.

[0061] In particular, the shape of each nanostructure included in the multiplexed superlens can exhibit a relatively strong response to incident light with specific optical properties. By influencing the vibrational mode of the transmitted light, the nanostructures included in the multiplexed superlens can alter the path or phase of the transmitted light according to different optical properties.

[0062] According to embodiments of this disclosure, an electronic device may use an image sensor to receive first polarized light transmitted by a multiplexed superlens and traveling along a first path. The electronic device may obtain a first coded image by receiving the first polarized light. The electronic device may obtain a first coded image corresponding to an object based on the distribution of the first polarized light transmitted by the multiplexed superlens.

[0063] The first coded image may vary depending on at least one of the object reflecting the light, the arrangement shape of the nanostructures included in the multiplexed superlens, or the optical properties of the light refracted by the multiplexed superlens. Depending on the irregular arrangement shape of the nanostructures in the multiplexed superlens, the first coded image obtained by receiving the first polarized light transmitted by the multiplexed superlens may be difficult to recognize with the naked eye.

[0064] In operation S320, the electronic device can determine whether the user's gaze information can be obtained from the first coded image.

[0065] For example, the first polarized light transmitted by the multiplexed superlens can form a portion of an image for the eye within the imaging plane of the image sensor. An electronic device can obtain a first coded image by receiving the first polarized light, and the electronic device can determine, based on the first coded image where the eye image is normally formed, that it can acquire information about the user's gaze.

[0066] As another example, the first polarized light transmitted by the multiplexed superlens can form a portion of an image of the eye outside the imaging plane of the image sensor. The electronic device may fail to receive the first polarized light using the image sensor, and the first coded image may not include the information about the eye contained in the first polarized light. The electronic device may determine that it can obtain information about the user's gaze based on the first coded image where the eye image is not properly formed.

[0067] In operation S330, when the electronic device cannot obtain the user's gaze information based on the determination of whether the user's gaze information can be obtained, the electronic device can change the first polarized light transmitted by the polarization filter into the second polarized light.

[0068] When linearly polarized light is incident perpendicularly onto a plane, the polarization-changing element can modulate the phase of the traveling wave. The polarization-changing element can generate second polarized light with perpendicular polarization from first polarized light in one direction. Optionally, the polarization-changing element can transmit first polarized light in one direction without alteration. When the user's line of sight is not available, the electronic device can use the polarization-changing element to generate second polarized light from the first polarized light for use.

[0069] Although the use of first polarized light in operation S310 and the generation of second polarized light in operation S330 of the electronic device have been described above, the order of use of the first polarized light and the second polarized light does not limit this disclosure. For example, in operation S310, the electronic device may use the second polarized light, and when the electronic device cannot obtain the user's line-of-sight information by using the second polarized light, the electronic device may use the first polarized light as a lower priority.

[0070] In operation S340, the electronic device can obtain a second coded image by receiving second polarized light refracted in a second path by the multiplexed superlens. A brief, or omitted, repetition of the descriptions above using operation S310 will be given.

[0071] According to embodiments of this disclosure, an electronic device may use an image sensor to receive second polarized light transmitted by a multiplexed superlens. The electronic device can obtain a second coded image by receiving the second polarized light. The electronic device can obtain a second coded image corresponding to an object based on the distribution of the second polarized light transmitted by the multiplexed superlens.

[0072] When operating S350, the electronic device can obtain the user's gaze information from the second coded image.

[0073] According to embodiments of this disclosure, the first polarized light and the second polarized light transmitted by the multiplexed superlens can travel along different paths. The first polarized light and the second polarized light can be focused at different positions on the imaging plane of the image sensor. An electronic device can obtain a first coded image by receiving the first polarized light using the image sensor, and can obtain a second coded image by receiving the second polarized light.

[0074] According to embodiments of this disclosure, the first polarized light and the second polarized light can be focused at different locations inside and outside the imaging plane of the image sensor. For example, the first polarized light can be focused at a point outside the imaging plane of the image sensor. The second polarized light can be focused at a point inside the imaging plane of the image sensor. The electronic device can obtain a first coded image by receiving the first polarized light, but may not obtain the user's gaze information based on the obtained first coded image. The electronic device can obtain a second coded image by receiving the second polarized light, and can obtain the user's gaze information based on the obtained second coded image.

[0075] Figure 4 This is a flowchart of an operation method of an electronic device according to an embodiment of the present disclosure.

[0076] For ease of explanation, the above references will be briefly given or omitted. Figure 3 A repetitive description of the described items.

[0077] Reference Figure 4 , can Figure 3 After operation S320, operation S410 is executed. According to an embodiment of this disclosure, in operation S320, the electronic device can determine whether it is possible to obtain the user's gaze information. In operation S410, when the electronic device is able to obtain the user's gaze information based on the determination that it is possible to obtain the user's gaze information, the electronic device can obtain the user's gaze information from the first coded image.

[0078] When the user's gaze information is available, in operation S410, the electronic device can obtain the user's gaze information from the first coded image.

[0079] When user gaze information cannot be obtained, the electronic device may generate second polarized light in operation S330. The electronic device may further execute operations S340 and S350 to obtain user gaze information from a second coded image obtained by receiving the second polarized light.

[0080] Figure 5 This is a flowchart of an operation method of an electronic device according to an embodiment of the present disclosure.

[0081] For ease of explanation, the above references will be briefly given or omitted. Figure 3 A repetitive description of the described items.

[0082] Reference Figure 5 , can Figure 3 Operation S510 is executed after operation S340. In operation S510, the electronic device can further determine whether the user's gaze information can be obtained from the second coded image. Because operation S510 overlaps with the content explained using operation S320, operation S510 is omitted.

[0083] In operation S520, if the electronic device cannot obtain the user's gaze information based on a re-determination of whether the user's gaze information can be obtained, the electronic device can change the first polarized light transmitted by the polarizing filter into third polarized light. The electronic device can generate third polarized light. Because operation S520 overlaps with the content explained using operation S330, operation S520 is briefly explained or omitted.

[0084] According to embodiments of this disclosure, an electronic device can generate first polarized light from externally incident light by allowing externally incident light to pass through a polarizing filter. The electronic device can change the polarization form of the first polarized light by using a polarization-changing element. The electronic device can also transmit the first polarized light in a first direction without alteration, or generate second polarized light in a second direction or third polarized light in a third direction from the first polarized light by using the polarization-changing element.

[0085] The third polarized light may differ from the first and second polarized light. The vibration direction of the third polarized light may differ from each of the vibration directions of the first and second polarized light.

[0086] In operation S530, the electronic device can obtain a third coded image by receiving third polarized light refracted in a third path by the multiplexed superlens. Because operation S530 overlaps with the content explained using operations S310 and S340, operation S530 is briefly explained or omitted.

[0087] According to embodiments of this disclosure, an electronic device may use an image sensor to receive third polarized light transmitted by a multiplexed superlens. The electronic device can obtain a third coded image by receiving the third polarized light. The electronic device can obtain a third coded image corresponding to an object based on the distribution of the third polarized light transmitted by the multiplexed superlens.

[0088] In operation S540, the electronic device can obtain the user's gaze information from the third coded image. Because operation S540 overlaps with the content explained using operation S350, operation S540 is briefly explained or omitted.

[0089] According to embodiments of this disclosure, the first, second, and third polarized light transmitted by the multiplexed superlens can travel along different paths. The first, second, and third polarized light can be focused at different positions on the imaging plane of the image sensor. An electronic device can obtain a first coded image by receiving the first polarized light using the image sensor, a second coded image by receiving the second polarized light, and a third coded image by receiving the third polarized light.

[0090] According to embodiments of this disclosure, the first polarized light, the second polarized light, and the third polarized light can be focused at different locations inside and outside the imaging plane of the image sensor. For example, the first polarized light and the second polarized light can be focused at a point outside the imaging plane of the image sensor. The third polarized light can be focused at a point inside the imaging plane of the image sensor.

[0091] An electronic device can obtain a first coded image by receiving first polarized light, but may not obtain the user's gaze information based on the obtained first coded image. An electronic device can obtain a second coded image by receiving second polarized light, but may not obtain the user's gaze information based on the obtained second coded image. An electronic device can obtain a third coded image by receiving third polarized light, and may obtain the user's gaze information based on the obtained third coded image.

[0092] Figure 6 This is a conceptual diagram illustrating an operation performed by an electronic device according to an embodiment of the present disclosure, based on tracking a gaze by receiving an coded image obtained using light generated by a light source and a multiplexed superlens.

[0093] For ease of explanation, the references above will be briefly given or omitted. Figure 1 A repetitive description of the described items.

[0094] Reference Figure 6 According to embodiments of the present disclosure, the electronic device may further include a light source 102 for outputting light.

[0095] According to embodiments of this disclosure, light source 102 can output light. Light source 102 can output light toward object 10. The light output by light source 102 can be reflected by object 10 located in front of a light modulation module, wherein the light modulation module includes a polarizing filter 110, a polarization changing element 120, and a multiplexing superlens 130. The light reflected by object 10 can be incident toward polarizing filter 110, and the polarization form of the incident light can be modified by polarizing filter 110. The incident light can be polarized by polarizing filter 110, the polarization form can be changed by polarization changing element 120, and the light path can be changed by multiplexing superlens 130 according to optical properties. For example, the light path can be changed according to the wavelength or polarization direction of the light.

[0096] According to embodiments of this disclosure, the light source 102 can output light according to a preset wavelength region. For example, the light source 102 can be an infrared light source, and the light source 102 can output light in the infrared band. As another example, the light source 102 can be an ultraviolet light source, and the light source 102 can output light in the ultraviolet band.

[0097] According to embodiments of the present disclosure, the light source 102 can output light according to a wavelength region including wavelengths of 850 nm or 940 nm.

[0098] According to embodiments of this disclosure, light output from light source 102 can be reflected by object 10. Incident light IL directed toward polarizing filter 110 can be light output from light source 102 and reflected by object 10. External incident light IL passes through polarizing filter 110, which can generate horizontally polarized light PL1 by polarizing the incident light IL in the horizontal direction. Although Figure 1 The first polarized light PL1, which has only horizontal polarization, is shown to be transmitted through the polarization filter 110, but the polarization direction is not limited to this disclosure.

[0099] According to embodiments of this disclosure, a first polarized light PL1 having horizontal polarization can be transmitted through a polarization-changing element 120. An electronic device can generate a second polarized light PL2 having vertical polarization from the first polarized light PL1 having horizontal polarization by using the polarization-changing element 120. The second polarized light PL2 generated by the polarization-changing element 120 can be incident toward the multiplexing superlens 130.

[0100] According to embodiments of this disclosure, the second polarized light PL2 can be transmitted by the multiplexing superlens 130. The electronic device can use the multiplexing superlens 130 to refract the transmitted second polarized light PL2 into different paths based on its optical properties. The electronic device can change the path of the second polarized light PL2 by using the multiplexing superlens 130.

[0101] According to embodiments of this disclosure, an electronic device can obtain an encoded image by receiving second polarized light PL2 transmitted by a multiplexed superlens 130 using an image sensor 140. The electronic device can obtain an encoded image 20 corresponding to the object 10 based on the distribution of the second polarized light PL2 transmitted by the multiplexed superlens 130.

[0102] According to embodiments of this disclosure, an electronic device can obtain a user's gaze information from an encoded image 20 using a processor 150. The electronic device can then track the user's gaze direction based on this gaze information.

[0103] Figure 7This is a conceptual diagram illustrating an operation performed by an electronic device according to an embodiment of the present disclosure, based on tracking a line of sight by receiving an coded image obtained using at least one of a plurality of light sources and a plurality of multiplexed superlenses.

[0104] For ease of explanation, the above references will be briefly given or omitted. Figure 6 A repetitive description of the described items.

[0105] Reference Figure 7 The electronic device may include multiple light sources. The electronic device may include a first light source 102 and a second light source 104. The first light source 102 may output a first light. The second light source 104 may output a second light. The wavelength regions of the first light and the second light may be different from each other.

[0106] According to embodiments of this disclosure, a first light source 102 can output first light. The first light source 102 can output the first light toward an object 10. The first light output by the first light source 102 can be reflected by the object 10 located in front of the light modulation module, wherein the light modulation module includes a polarization filter 110, a polarization changing element 120, and a multiplexed superlens 130.

[0107] According to embodiments of this disclosure, the first light reflected by object 10 can be incident toward polarizing filter 110. Figure 7 In this process, the first light incident toward the polarizing filter 110 can be a first incident light IL1. The first light can be transmitted through the polarizing filter 110, and the electronic device can generate horizontal polarization by using the polarizing filter 110 to polarize the first light in the horizontal direction. The first light polarized in the horizontal direction can be a first polarized light PL1.

[0108] According to embodiments of this disclosure, first polarized light PL1 can be transmitted by polarization changing element 120. An electronic device can generate second polarized light PL2 with vertical polarization from the first polarized light PL1 using polarization changing element 120. The second polarized light PL2 generated by polarization changing element 120 can be incident towards multiplexed superlens 130.

[0109] The polarization changing element 120 can transmit the first polarized light PL1 without alteration, or it can change the first polarized light PL1 into the second polarized light PL2. Although the above description is based on the changed second polarized light PL2... Figure 7 However, an encoded image can be obtained based on the first polarized light PL1 transmitted without any change.

[0110] When linearly polarized light in one direction is incident perpendicularly on the front surface of the polarization changing element 120, the polarization changing element 120 can selectively generate linearly polarized light in either the first or second direction. Figure 7In this process, a second polarized light PL2 with vertical polarization is generated. However, the second polarized light PL2 may have horizontal polarization, and the electronic device can control the polarization changing element 120 to transmit the first polarized light PL1 with vertical or horizontal polarization without making any changes.

[0111] According to embodiments of this disclosure, the second polarized light PL2 can be transmitted by the multiplexing superlens 130. The electronic device can use the multiplexing superlens 130 to refract the transmitted second polarized light PL2 into different paths based on its optical properties. The electronic device can change the path of the second polarized light PL2 by using the multiplexing superlens 130.

[0112] According to embodiments of this disclosure, an electronic device can obtain a first coded image by receiving second polarized light PL2 transmitted by a multiplexed superlens 130 using an image sensor 140. The electronic device can obtain a first coded image 21 corresponding to the object 10 based on the distribution of the second polarized light PL2 transmitted by the multiplexed superlens 130.

[0113] According to embodiments of this disclosure, the electronic device can obtain the user's gaze information from the first coded image 21 using the processor 150. The electronic device can then track the user's gaze direction based on the gaze information.

[0114] According to embodiments of this disclosure, the second light source 104 can output second light. The second light source 104 can output second light toward the object 10. An electronic device can obtain a second coded image 22 by receiving the second light transmitted by the polarization filter 110, the polarization changing element 120, and the multiplexing superlens 130. The method performed by the electronic device to obtain the second coded image 22 by using the second light is the same as the method to obtain the first coded image 21 by using the first light, and therefore its description will be omitted.

[0115] According to embodiments of this disclosure, the electronic device can obtain the user's gaze information from the second coded image 22 using the processor 150. The electronic device can then track the user's gaze direction based on the gaze information.

[0116] According to embodiments of this disclosure, an electronic device can obtain a user's gaze information from a first encoded image 21 or a second encoded image 22 using a processor 150. The electronic device can then track the user's gaze direction based on this gaze information.

[0117] For example, the electronic device can determine whether it can obtain the user's gaze information from the first coded image 21. When the electronic device can obtain the user's gaze information from the first coded image 21, it can track the user's gaze direction using the first coded image 21. When the electronic device cannot obtain the user's gaze information from the first coded image 21, it can output second light using the second light source 104 and obtain a second coded image 22 based on the output second light. The electronic device can then track the user's gaze direction from the second coded image 22.

[0118] The electronic device can then determine whether it can obtain the user's gaze information from the second coded image 22. When the electronic device can obtain the user's gaze information from the second coded image 22, it can track the user's gaze direction using the second coded image 22. When the electronic device cannot obtain the user's gaze information from the second coded image 22, it can output a third light using another light source and obtain a third coded image based on the output third light. The electronic device can then track the user's gaze direction from the third coded image.

[0119] The electronic device may include three or more light sources and may repeatedly generate multiple coded images based on multiple lights, such that coded images capable of obtaining information about the user's gaze can be obtained. The number of light sources classified by polarization-changing elements and the degree of light delay do not limit this disclosure.

[0120] Figure 8 This is a flowchart of a method for tracking a line of sight performed by an electronic device according to an embodiment of the present disclosure, based on the use of multiple light sources and multiple lights generated by a multiplexed superlens.

[0121] For ease of explanation, the above references will be briefly given or omitted. Figure 3 A repetitive description of the described items.

[0122] In operation S810, polarization changing elements can be used to generate first to fourth polarized light. The electronic device can transmit the first polarized light by allowing the first light to pass through a polarization filter. The electronic device can transmit the transmitted first polarized light without modification, or it can modify the transmitted first polarized light into second polarized light. The electronic device can transmit a third polarized light by allowing the second light to pass through a polarization filter. The electronic device can transmit the transmitted third polarized light without modification, or it can modify the transmitted third polarized light into fourth polarized light.

[0123] According to embodiments of this disclosure, an electronic device can change the polarization form of a first polarized light or a third polarized light by using a polarization-changing element. Based on the first polarized light, the electronic device can transmit first polarized light having a first polarization direction without modification, or it can generate second polarized light having a second polarization direction. The electronic device can selectively generate first polarized light or second polarized light by controlling the polarization-changing element.

[0124] According to embodiments of this disclosure, the electronic device may include a first light source and a second light source. The first light source may output a first light. The second light source may output a second light.

[0125] The first light and the second light can be different from each other. The wavelength regions of the first light and the second light can also be different from each other. For example, an electronic device can output a first light with a wavelength of 850 nm by using a first light source, and can output a second light with a wavelength of 940 nm by using a second light source.

[0126] According to embodiments of this disclosure, an electronic device can generate third polarized light polarized in one direction by allowing second light to pass through a polarizing filter.

[0127] According to embodiments of this disclosure, an electronic device can transmit third-polarized light with a third polarization direction without modification, or can generate fourth-polarized light with a fourth polarization direction. The electronic device can selectively generate third-polarized or fourth-polarized light by controlling a polarization-changing element. For example, the third polarization direction may be the same as the first direction, and the fourth direction may be the same as the second direction.

[0128] In operation S820, the electronic device can obtain a third coded image by receiving one of the first to fourth polarized light using an image sensor. In operation S830, the electronic device can obtain the user's gaze information from the third coded image.

[0129] According to embodiments of this disclosure, an electronic device can alter the path of one of a first to a fourth polarized light beam by using a multiplexed superlens. The electronic device can obtain an encoded image by receiving one of the altered paths of the first to fourth polarized light beams. The electronic device can obtain user gaze information from the obtained encoded image. The electronic device can track the user's gaze based on the encoded image.

[0130] According to embodiments of this disclosure, an electronic device can acquire an coded image based on one of a first to a fourth polarized light. The electronic device can determine whether user gaze information can be obtained from the acquired coded image. When the electronic device can obtain the user's gaze information, it can track the user's gaze from the acquired coded image. However, when the electronic device cannot obtain the user's gaze information, it can acquire another coded image based on another of the first to fourth polarized light. The electronic device can then acquire the user's gaze information based on this newly acquired coded image.

[0131] The electronic device can repeatedly generate multiple coded images based on the first to fourth polarized light, so that coded images that can obtain the user's gaze information can be obtained.

[0132] According to embodiments of this disclosure, the encoded image may have different shapes depending on the optical properties of the first to fourth polarized light. The optical properties of the first to fourth polarized light may be different from each other, and the encoded images obtained based on the first to fourth polarized light may be different from each other.

[0133] For example, the first and second polarized light are light transformed from light with a wavelength of 940 nm output from the first light source, and can be light with a wavelength of 940 nm. The third and fourth polarized light are light transformed from light with a wavelength of 850 nm output from the second light source, and can be light with a wavelength of 850 nm.

[0134] Because the first and second polarized light are generated from the same light source, they can be light of the same wavelength. However, their polarization directions can be different depending on the polarization changing element. Similarly, because the third and fourth polarized light are generated from the same light source, they can also be light of the same wavelength.

[0135] Figure 9 This is a conceptual diagram illustrating an operation performed by an electronic device according to an embodiment of the present disclosure, based on an coded image obtained by receiving light emitted from multiple light sources to track a gaze. For ease of explanation, the references above will be briefly given or omitted. Figure 1 A repetitive description of the described items.

[0136] Reference Figure 9 The electronic device may include a first light source 102, a second light source 104, a multiplexed superlens 230, an image sensor 240, and a processor 250.

[0137] According to embodiments of this disclosure, the first light source 102 can output a first light L1. The first light source 102 can output the first light L1 toward the object 10. The first light L1 output by the first light source 102 can be reflected by the object 10 located in front of the multiplexed superlens 230.

[0138] According to embodiments of this disclosure, the first light L1 reflected by object 10 can be transmitted through multiplexed superlens 230. The electronic device can use multiplexed superlens 230 to refract the transmitted first light L1 into different paths based on its optical properties. The electronic device can change the path of the first light L1 by using multiplexed superlens 230.

[0139] According to embodiments of this disclosure, an electronic device can obtain a first coded image 21 by using an image sensor 240 to receive first light L11 transmitted by a multiplexed superlens 230. The electronic device can obtain the first coded image 21 corresponding to the object 10 based on the distribution of the first light L11 transmitted by the multiplexed superlens 230.

[0140] According to embodiments of this disclosure, the electronic device can obtain the user's gaze information from the first coded image 21 using the processor 250. The electronic device can then track the user's gaze direction based on the gaze information.

[0141] According to embodiments of this disclosure, the second light source 104 can output a second light L2. The second light source 104 can output the second light L2 toward the object 10. The second light L2 output by the second light source 104 can be reflected by the object 10 located in front of the multiplexing superlens 230.

[0142] The method of obtaining the second coded image 22 by using the second light L2, performed by an electronic device, is the same as the method of obtaining the first coded image 21 by using the first light L1, and therefore its description will be omitted.

[0143] According to embodiments of this disclosure, the electronic device can obtain the user's gaze information from the second coded image 22 using the processor 250. The electronic device can then track the user's gaze direction based on the gaze information.

[0144] According to embodiments of this disclosure, an electronic device can obtain a user's gaze information from a first encoded image 21 or a second encoded image 22 using a processor 250. The electronic device can then track the user's gaze direction based on this gaze information.

[0145] For example, the electronic device can determine whether it can obtain the user's gaze information from the first coded image 21. When the electronic device can obtain the user's gaze information from the first coded image 21, it can track the user's gaze direction using the first coded image 21. When the electronic device cannot obtain the user's gaze information from the first coded image 21, it can output a second light L2 using the second light source 104, and obtain a second coded image 22 based on the output second light L2. The electronic device can then track the user's gaze direction from the second coded image 22.

[0146] The electronic device can then determine whether it can obtain the user's gaze information from the second coded image 22. When the electronic device can obtain the user's gaze information from the second coded image 22, it can track the user's gaze direction using the second coded image 22. When the electronic device cannot obtain the user's gaze information from the second coded image 22, it can output a third light using another light source and obtain a third coded image based on the output third light. The electronic device can then track the user's gaze direction from the third coded image.

[0147] The electronic device may include three or more light sources and may repeatedly generate multiple coded images based on multiple lights, thereby enabling the acquisition of coded images containing user gaze information. The number of light sources is not limited in this disclosure.

[0148] Figure 10a It is a conceptual diagram used to explain the operation of focusing a first wavelength of light output from a light source onto an image sensor.

[0149] According to embodiments of this disclosure, an electronic device can control a first light source to output a first light 61 having a first wavelength λ1. The first light 61 can be reflected by an object and transmitted through a multiplexed superlens 230. The electronic device can alter the path of the first light 61 by allowing it to pass through the multiplexed superlens 230. After the first light 61 is transmitted through the multiplexed superlens 230, the electronic device can alter the path of the first light 61 such that it is focused at a first point P1 on the imaging surface 241 of the image sensor 240. The first light 61, with its altered path, can reach the first point P1 within the imaging surface 241 of the image sensor 240 and can be focused at the first point P1.

[0150] For example, the nanostructure of the multiplexed superlens 230 can partially interact with the region of the first wavelength λ1 of the first light 61 to change the path of the light, so that the first light 61 is focused at the first point P1.

[0151] Figure 10b This is a conceptual diagram used to explain the operation of focusing a second wavelength of light output from a light source onto an image sensor.

[0152] According to embodiments of this disclosure, an electronic device can control a second light source to output a second light 62 having a second wavelength λ2. The second light 62 can be reflected by an object and transmitted through a multiplexed superlens 230. The electronic device can alter the path of the second light 62 by allowing it to pass through the multiplexed superlens 230. After the second light 62 is transmitted through the multiplexed superlens 230, the electronic device can alter the path of the second light 62 such that the second light 62 forms at a second point P2 on the imaging surface 241 of the image sensor 240. The second light 62, with its altered path, can reach the second point P2 within the imaging surface 241 of the image sensor 240 and can be focused at the second point P2.

[0153] For example, the nanostructure of the multiplexed superlens 230 can partially interact with the region of the second wavelength λ2 of the second light 62 to change the path of the light, so that the second light 62 is focused at the second point P2.

[0154] According to embodiments of this disclosure, the multiplexed superlens 230 can change the path of transmitted light to different paths depending on the wavelength of the light. Therefore, electronic devices can use the multiplexed superlens 230 to change the path of light to... Figure 10a The first ray of light is focused at the first point P1, and the path of the light is changed so that... Figure 10b The second light beam is focused at the second point P2.

[0155] According to embodiments of this disclosure, with Figure 10a and Figure 10b Unlike other devices, the electronic device allows light to pass through the multiplexed superlens 230, thereby altering the light path so that the light is focused outside the imaging surface 241 of the image sensor 240. When the light is focused outside the imaging surface 241, the electronic device can obtain an encoded image based on the light received by the image sensor 240, but it can be determined that the user's gaze information cannot be obtained from the obtained encoded image. Based on this determination, the electronic device can obtain an encoded image based on other light, and the user's gaze information can be obtained from the obtained encoded image.

[0156] Figure 11 This is a flowchart of a method for tracking a gaze by extracting feature points based on an coded image, performed by an electronic device according to an embodiment of the present disclosure. Figure 3 Operation S350 may include operation S1110 and operation S1120.

[0157] In operation of S1110, the electronic device can obtain feature points about the eye from the encoded image.

[0158] According to this disclosure, a feature point can refer to a point in an image that represents a major feature or a point of interest. For example, the angle at which two or more edges intersect each other, detected based on changes in pixel values, and the point within an image with the largest or smallest pixel value can be a feature point.

[0159] Feature points may include information about at least one of the image's position coordinates or shape. For example, feature points may include at least one of pupil feature points or eye reflection feature points.

[0160] According to embodiments of this disclosure, an electronic device can obtain feature points from an coded image. The electronic device can obtain feature points by inputting the obtained coded image into an AI model. The AI ​​model can be a model trained to extract feature points. The AI ​​model can be a model trained to extract feature points from an coded image, wherein the coded image is obtained based on light passing through a multiplexed superlens.

[0161] According to embodiments of this disclosure, an electronic device can obtain feature points by inputting a first coded image or a second coded image into an AI model. The coded image input for obtaining feature points is not limited to this disclosure.

[0162] When operating S1120, the electronic device can obtain the user's gaze information based on feature points.

[0163] According to embodiments of this disclosure, an electronic device can obtain a user's gaze information based on feature points. The electronic device can track the user's gaze direction based on these feature points. For example, the electronic device can determine the user's gaze direction by considering factors such as the shape of the pupil and the position of reflective coordinates.

[0164] Figure 12 This is a flowchart of a method for tracking a gaze by reconstructing an image based on an encoded image, performed by an electronic device according to an embodiment of the present disclosure. Figure 3 Operation S350 may include operation S1210 and operation S1220.

[0165] In operation S1210, the electronic device can obtain a restored image of the eye from the encoded image.

[0166] Encoded images can vary depending on at least one of the object reflecting the light or the pattern shape of the multiplexed superlens. Typically, objects represented in coded images phase-modulated by multiplexed superlenses may be difficult to identify with the naked eye.

[0167] According to embodiments of this disclosure, an electronic device can obtain a reconstructed image from an encoded image. The electronic device can obtain the reconstructed image by inputting the obtained encoded image into an AI model. The AI ​​model can be a model trained to reconstruct an image. The AI ​​model can also be a model trained to obtain a reconstructed image by reconstructing an encoded image, wherein the encoded image is obtained based on light passing through a multiplexed superlens.

[0168] In operation S1220, the electronic device can obtain the user's gaze information based on the recovered image.

[0169] According to embodiments of this disclosure, an electronic device can obtain a user's gaze information based on a recovered image. The electronic device can then track the user's gaze direction based on the recovered image. For example, the electronic device can determine the user's gaze direction by considering, for example, the shape of the pupil within the recovered image and the gaze direction within the recovered image.

[0170] Figure 13 This is a block diagram illustrating the structure of an electronic device 1300 according to an embodiment of the present disclosure.

[0171] For ease of explanation, the references above will be briefly given or omitted. Figure 1 and Figure 2 A repetitive description of the described items.

[0172] Electronic device 1300 can be a means for acquiring an image by receiving light transmitted by multiplexed superlens 1320 via image sensor 1330. Electronic device 1300 can be implemented as any device, such as a mobile device, smartphone, laptop computer, desktop computer, tablet PC, wearable device, e-book terminal, digital broadcasting terminal, personal digital assistant (PDA), portable multimedia player (PMP), navigation device, MP3 player, or portable camera. According to embodiments of this disclosure, electronic device 1300 can be an augmented reality device. An augmented reality device is a device capable of realizing "augmented reality" and may include not only augmented reality glasses worn on a user's face but also head-mounted displays (HMDs) or augmented reality helmets worn on a user's head.

[0173] Reference Figure 13 The electronic device 1300 may include an optical controller 1310, a multiplexed superlens 1320, an image sensor 1330, a processor 1340, and a memory 1350. The optical controller 1310, the processor 1340, and the memory 1350 may be electrically and / or physically connected to each other.

[0174] Figure 13 The components shown are merely examples of embodiments according to this disclosure, and the components included in the electronic device 1300 are not limited to those described above. Figure 13 The components shown. Electronic device 1300 may not include... Figure 13 Some of the components shown, and may also include Figure 13 Components not shown. For example, electronic device 1300 may also include a power source (e.g., a battery) that supplies driving power to optical controller 1310, processor 1340 and memory 1350.

[0175] According to embodiments of this disclosure, the image sensor 1330 can be used to track eye gaze. In this case, the image sensor 1330 typically uses an infrared light source. In order to track eye gaze using the image sensor 1330, infrared light output from the infrared light source can be reflected from the eye and transmitted through the multiplexed superlens 1320. The light reflected from the eye and transmitted through the multiplexed superlens 1320 can be received by the image sensor 1330.

[0176] The optical controller 1310 can be configured to selectively generate multiple lights with different optical properties.

[0177] According to embodiments of this disclosure, the optical controller 1310 may include a polarization filter and a polarization changing element.

[0178] Polarizing filters can transmit only polarized light (PL) with a specific polarization from external incident light. When linearly polarized light is incident perpendicularly onto a plane, a polarization-changing element can alter the phase of the traveling wave.

[0179] Electronic device 1300 can selectively generate one of a plurality of lights with distinct optical properties based on externally incident light using optical controller 1310. The externally incident light can be polarized in one direction by a polarizing filter.

[0180] The phase of polarized light can be modulated by a polarization-changing element. Electronic device 1300 can modulate the phase of polarized light by controlling the polarization-changing element, and can selectively generate one of a plurality of lights with different phases depending on the degree to which the phase of the polarized light is modulated.

[0181] According to embodiments of this disclosure, the optical controller 1310 may include a first light source and a second light source.

[0182] The first light source can output light with a first wavelength. The second light source can output light with a second wavelength. The first wavelength and the second wavelength can be different from each other. For example, the first wavelength can be 850 nm and the second wavelength can be 940 nm.

[0183] The electronic device 1300 can selectively generate one of two lights with different wavelengths by controlling the first light source and the second light source.

[0184] According to embodiments of the present disclosure, the optical controller 1310 may include a first light source, a second light source, a polarization filter, and a polarization changing element.

[0185] The electronic device 1300 can selectively generate one of a first light having a first wavelength and a second light having a second wavelength by controlling a first light source and a second light source.

[0186] When the first light is incident, the electronic device can modulate the phase of the polarized first light by controlling the polarization filter and the polarization changing element, and can selectively generate one of a plurality of first lights with different phases according to the degree to which the phase of the polarized first light is modulated.

[0187] When the second light is incident, the electronic device 1300 can modulate the phase of the polarized second light by controlling the polarization filter and the polarization changing element, and can selectively generate one of a plurality of second lights with different phases according to the degree to which the phase of the polarized second light is modulated.

[0188] The multiplexed superlens 1320 can be a lens in which relatively small-scale nanostructures are arranged in a 2D manner. The multiplexed superlens 1320 may include a surface on which nanostructures are formed and arranged on a substrate, and the phase of the transmitted light can be modulated according to the arrangement of the nanostructures of the multiplexed superlens 1320 and the shape of each nanostructure.

[0189] The multiplexed superlens 1320 can be configured to refract incident light in different paths according to its optical properties. The electronic device 1300 can adjust the focusing position of the incident light by using the multiplexed superlens 1320 to refract the incident light in each path.

[0190] For example, even when the first light refracted by the multiplexed superlens 1320 is not focused within the imaging surface of the image sensor 1330, the electronic device 1300 can still obtain an encoded image of the object by using the second light refracted by the multiplexed superlens 1320. When at least one light refracted by the multiplexed superlens 1320 forms part of an image within the imaging surface of the image sensor 1330, the electronic device 1300 can obtain an encoded image of the object by using multiple lights.

[0191] Therefore, when at least one of the multiple light beams refracted by the multiplexed superlens 1320 forms part of an image within the imaging surface of the image sensor 1330, the electronic device 1300 can obtain an coded image of an object using multiple light beams and can track the user's gaze based on the coded image. Because the electronic device 1300 can utilize multiple light beams, the area of ​​the image sensor 1330 can be minimized, and when an image of at least one light beam is formed within the imaging surface of the image sensor 1330 with the minimized area, the electronic device can track the user's gaze.

[0192] For example, the horizontal and vertical lengths of the image sensor 1330 can be reduced, and the width of the image sensor 1330 can also be reduced while having the need for fewer internal components. Similarly, the electronic device 1300 including the image sensor 1330 (i.e., the eye-tracking camera system) can be miniaturized. Specifically, when the volume of a conventional eye-tracking camera is 3.2 × 3.2 × 3.9 (mm)... 3 When using the eye-tracking camera of the electronic device 1300 according to an embodiment of the present disclosure, the volume can be 1.6 × 1.6 × 1.5 (mm). 3 ).

[0193] As the image sensor 1330 becomes smaller, the power consumption of the electronic device 1300 can be reduced. For example, when the power consumption of a conventional eye-tracking camera is 85 mW, the power consumption of the eye-tracking camera using the electronic device 1300 according to embodiments of the present disclosure can be 42.5 mW. According to embodiments of the present disclosure, the power consumption of the eye-tracking camera can be reduced by 50%.

[0194] However, this is merely an example, and the miniaturization of the image sensor 1330 and the power consumption of the electronic device 1300 are determined by the performance required for the situation and do not limit this disclosure.

[0195] Image sensor 1330 is an imaging element configured to acquire an coded image by receiving light transmitted by multiplexed superlens 1320, converting the brightness or intensity of the received light into an electrical signal, and imaging the electrical signal. Image sensor 1330 may be implemented as, for example, a charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS), but is not limited thereto.

[0196] Processor 1340 may execute one or more instructions or program code stored in memory 1350, and may perform functions and / or operations corresponding to the instructions or program code. Processor 1340 may include hardware components that perform arithmetic, logic, input / output operations, and signal processing. Processor 1340 may include, but is not limited to, at least one of a central processing unit, microprocessor, graphics processing unit, application processor (AP), application-specific integrated circuit (ASIC), digital signal processor (DSP), digital signal processing device (DSPD), programmable logic device (PLD), or field-programmable gate array (fPGA).

[0197] Processor 1340 in Figure 13 The processor 1340 is shown as a single element, but embodiments of this disclosure are not limited thereto. According to embodiments, the processor 1340 may be configured as one or more.

[0198] According to embodiments of this disclosure, processor 1340 may be configured as a dedicated hardware chip for performing artificial intelligence (AI) learning.

[0199] Instructions and program code readable by the processor 1340 may be stored in the memory 1350. The memory 1350 may include at least one of the following: flash memory, hard disk memory, multimedia card micro / card-type memory (e.g., Secure Digital (SD) or Extreme Digital (XD) memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), programmable ROM (PROM), mask ROM, flash ROM, hard disk drive (HDD), or solid-state drive (SSD).

[0200] The memory 1350 may store instructions or program code for performing the functions or operations of the electronic device 1300. According to embodiments of this disclosure, at least one of the following—instructions, algorithms, data structures, program code, or applications readable by the processor 1340—may be stored in the memory 1350. The instructions, algorithms, data structures, and program code stored in the memory 1350 may be implemented using, for example, programming or scripting languages ​​(such as C, C++, Java, assembler, etc.).

[0201] Memory 1350 may store instructions, algorithms, data structures, or program code related to an AI model used to obtain user gaze information from an encoded image. For example, memory 1350 may store program code for acquiring feature points or reconstructing an image from an encoded image and tracking the user's gaze using the feature points or reconstructed image. Modules included in memory 1350 represent units that process functions or operations performed by processor 1340 and may be implemented as software, such as instructions, algorithms, data structures, or program code.

[0202] Processor 1340 can track a user's gaze from an encoded image. Processor 1340 can track a user's gaze by using an AI algorithm, wherein the AI ​​algorithm is trained to obtain the user's gaze information from an encoded image whose phase has been modulated by a specific pattern of a multiplexed superlens 1320.

[0203] According to the following embodiments, the processor 1340 can be implemented by running instructions or program code stored in the memory 1350.

[0204] According to embodiments of this disclosure, processor 1340 can obtain a first coded image by receiving first polarized light refracted in a first path by multiplexed superlens 1320. Processor 1340 can determine whether user gaze information can be obtained from the first coded image. When, based on this determination, processor 1340 cannot obtain user gaze information, processor 1340 can change the first polarized light transmitted by a polarizing filter to second polarized light. Processor 1340 can obtain a second coded image by receiving second polarized light refracted in a second path by multiplexed superlens 1320. Processor 1340 can obtain user gaze information from the second coded image.

[0205] According to an embodiment of this disclosure, when the processor 1340 is able to obtain the user's gaze information based on this determination, the processor 1340 can obtain the user's gaze information from the first encoded image.

[0206] According to embodiments of this disclosure, processor 1340 may further determine whether user gaze information can be obtained from the second coded image. If, based on this re-determination, processor 1340 cannot obtain user gaze information, processor 1340 may change the first polarized light transmitted by the polarizing filter to third polarized light. Processor 1340 may obtain a third coded image by receiving the third polarized light refracted in a third path by the multiplexed superlens 1320. Processor 1340 may obtain user gaze information from the third coded image.

[0207] According to embodiments of this disclosure, first polarized light can be transmitted by allowing first light to pass through a polarization filter, and the first polarized light can be transmitted by a polarization-changing element without change, or the first polarized light transmitted by the polarization-changing element can be changed to second polarized light. Third polarized light can be transmitted by allowing second light to pass through a polarization filter, and the third polarized light can be transmitted by the polarization-changing element without change, or the third polarized light transmitted by the polarization-changing element can be changed to fourth polarized light. Processor 1340 can obtain a third coded image by receiving the first polarized light, second polarized light, third polarized light, or fourth polarized light using image sensor 1330. Processor 1340 can obtain user gaze information from the third coded image.

[0208] According to embodiments of this disclosure, processor 1340 can obtain feature points about the eyes from a second encoded image. Processor 1340 can obtain user gaze information based on the feature points.

[0209] According to embodiments of this disclosure, processor 1340 can obtain a restored image of the eye from a second encoded image. Processor 1340 can then obtain user gaze information based on the restored image.

[0210] According to embodiments of this disclosure, processor 1340 can obtain a first coded image by using image sensor 1330 to receive first light refracted in a first path by multiplexed superlens 1320. Processor 1340 can determine whether user gaze information can be obtained from the first coded image. When, based on this determination, processor 1340 cannot obtain user gaze information, processor 1340 can obtain a second coded image by using image sensor 1330 to receive second light refracted in a second path by multiplexed superlens 1320. Processor 1340 can obtain user gaze information from the second coded image.

[0211] According to an embodiment of this disclosure, when the processor 1340 is able to obtain the user's gaze information based on this determination, the processor 1340 can obtain the user's gaze information from the first encoded image.

[0212] According to embodiments of this disclosure, processor 1340 may further determine whether user gaze information can be obtained from the second coded image. If, based on this further determination, processor 1340 cannot obtain user gaze information, processor 1340 may obtain a third coded image by using image sensor 1330 to receive third light refracted in a third path by multiplexed superlens 1320. Processor 1340 may then obtain user gaze information from the third coded image.

[0213] According to embodiments of this disclosure, processor 1340 can obtain feature points about the eyes from a second encoded image. Processor 1340 can obtain user gaze information based on the feature points.

[0214] According to embodiments of this disclosure, processor 1340 can obtain a restored image of the eye from a second encoded image. Processor 1340 can then obtain user gaze information based on the restored image.

[0215] The processor 1340 of the electronic device 1300 according to embodiments of the present disclosure may use an AI model trained to obtain user gaze information from an encoded image. According to embodiments of the present disclosure, the AI ​​model may be a deep neural network model trained according to supervised learning, wherein the training according to supervised learning applies an encoded image obtained based on a simulated light distribution based on a multiplexed superlens 1320 as input data, and applies the user's gaze corresponding to the encoded image as the output ground truth. "Training" may refer to training a neural network such that the neural network discovers or learns methods for analyzing multiple input data for the neural network, methods for classifying multiple input data, and / or methods for extracting features required to produce result data from multiple input data. Specifically, through training, the deep neural network model can optimize the weight values ​​in the neural network by learning training data (e.g., multiple original images and multiple feature points). The deep neural network model outputs a target result by processing multiple input data via a neural network with optimized weight values.

[0216] The type of AI model is not limited to this disclosure, and the AI ​​model can be implemented as any of the following: Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Bidirectional Recurrent Deep Neural Network (BRDNN), and Deep Q-Network. AI models can be further subdivided. CNNs can be subdivided into Deep CNNs (DCNN) or CapsNet neural networks.

[0217] Processor 1340 can obtain user gaze information from an encoded image using a pre-trained AI model. According to embodiments of this disclosure, processor 1340 can input an encoded image obtained by image sensor 1330 into the AI ​​model, and can obtain user gaze information corresponding to the encoded image by performing inference using the AI ​​model. For example, the pre-trained AI model can obtain feature points or reconstructed images corresponding to the encoded image, and can obtain user gaze information based on the feature points or reconstructed images.

[0218] The AI ​​model can be stored in the memory 1350 of the electronic device 1300. However, this is merely an example and does not limit the present disclosure. For example, the AI ​​model can be stored in an external server. In this case, the electronic device 1300 may also include a communication interface capable of performing data communication with the external server, and can receive the AI ​​model or the result data (e.g., feature points) inferred by the AI ​​model from the external server through the communication interface. Typically, the memory storage capacity, throughput speed, and ability to collect training datasets of the electronic device 1300 may be limited compared to a server. Therefore, the server can perform the storage of large amounts of data and operations requiring high throughput, and then send the required data and / or AI model to the electronic device 1300 via a communication network. In this case, by receiving and using the AI ​​model or the inference data of the AI ​​model by the server, the electronic device 1300 can have the ability to perform the required operations quickly and easily without using large-capacity memory and a processor with high operating capabilities.

[0219] Machine-readable storage media may be provided as non-transitory storage media. "Non-transitory storage media" is a tangible device and simply means that it does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is stored semi-permanently in the storage medium and cases where data is temporarily stored. For example, a non-transitory recording medium may include a buffer for temporarily storing data.

[0220] According to embodiments of this disclosure, methods according to various disclosed embodiments can be provided by including them in a computer program product. The computer program product, as a commodity, can be traded between a seller and a buyer. The computer program product is distributed in the form of a device-readable storage medium (e.g., an optical disc read-only memory (CD-ROM)), or can be distributed directly and online (e.g., downloaded or uploaded) between two user devices (e.g., smartphones) through an app store. In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable application (app)) may be stored at least temporarily in a device-readable storage medium (such as the memory of a manufacturer's server, an app store's server, or a relay server) or may be temporarily generated.

Claims

1. An electronic device comprising: A polarizing filter is configured to receive incident light and change the polarization of the incident light; A polarization-changing element is configured to receive light from the polarization filter; A superlens is configured to refract light received from the polarization changing element into different paths according to the polarization direction of the light received from the polarization changing element; An image sensor is configured to receive refracted light from a superlens; At least one processor; as well as At least one memory stores at least one instruction, wherein, when executed by the at least one processor, the electronic device causes the electronic device to perform the following operations: This causes the polarization filter to change the incident light into first polarized light with a first direction and transmit the first polarized light. The image sensor obtains a first coded image based on first polarized light refracted into a first path by the superlens. Based on the identification that the user's gaze information could not be obtained from the first encoded image: This causes the polarization-changing element to change the first polarized light into second polarized light with a second direction and transmit the second polarized light. The image sensor obtains a second coded image based on second polarized light refracted by the superlens into a second path. The user's gaze information is obtained from the second encoded image.

2. The electronic device according to claim 1, wherein, When executed by the at least one processor, the at least one instruction causes the electronic device to perform the following operations: Based on the identification that the user's gaze information can be obtained from the first encoded image, the user's gaze information is obtained from the first encoded image.

3. The electronic device according to claim 1, wherein, The first direction is perpendicular to the second direction.

4. The electronic device according to claim 1, wherein, When executed by the at least one processor, the at least one instruction causes the electronic device to perform the following operations: Based on the identification that the user's gaze information could not be obtained from the second encoded image: This causes the polarization changing element to change the first polarized light into a third polarized light with a third polarization and transmit the third polarized light. The image sensor obtains a third coded image based on third polarized light refracted by the superlens into a third path. The user's gaze information is obtained from the third encoded image.

5. The electronic device according to claim 1, in, The first coded image is formed by first polarized light arriving at a first point within the imaging surface of the image sensor along a first path, and The second coded image is formed by second polarized light that reaches a second point within the imaging surface of the image sensor along a second path.

6. The electronic device according to claim 1, in, The first path does not intersect the imaging surface of the image sensor, and The second encoded image is formed by second polarized light that reaches the imaging surface of the image sensor along a second path.

7. The electronic device according to claim 1, further comprising: The first light source is configured to output the first light. Wherein, the at least one instruction, when executed by the at least one processor, causes the electronic device to perform the following operations: The first polarized light is transmitted by allowing the first light to pass through the polarization filter.

8. The electronic device according to claim 4, further comprising: The first light source is configured to output the first light; as well as The second light source is configured to output a second light. Wherein, the at least one instruction, when executed by the at least one processor, causes the electronic device to perform the following operations: The transmission of a first polarized light is facilitated by allowing the first light to pass through the polarization filter, or the transmission of a third polarized light is facilitated by allowing the second light to pass through the polarization filter.

9. The electronic device according to claim 8, wherein, The wavelength regions of the first light and the second light are different from each other.

10. The electronic device according to claim 1, further comprising: The first light source is configured to output the first light; as well as The second light source is configured to output a second light. Wherein, the at least one instruction, when executed by the at least one processor, causes the electronic device to perform the following operations: The first polarized light is transmitted by allowing the first light to pass through the polarization filter. This causes the polarization-changing element to transmit first polarized light or to change the first polarized light into second polarized light and transmit the second polarized light. By allowing the second light to pass through the polarization filter, a third-polarized light of a third direction is induced to be transmitted. This causes the polarization-changing element to transmit third-polarized light or to change the third-polarized light into fourth-polarized light with a fourth direction and transmit the fourth-polarized light. The user's gaze information is obtained from a third coded image based on a first polarized light, a second polarized light, a third polarized light, or a fourth polarized light using the image sensor.

11. The electronic device according to claim 1, wherein, When executed by the at least one processor, the at least one instruction causes the electronic device to perform the following operations: Eye-related feature points are obtained from the second encoded image, and The user's gaze information is obtained based on the feature points.

12. The electronic device according to claim 1, wherein, When executed by the at least one processor, the at least one instruction causes the electronic device to perform the following operations: Obtain an eye-related restored image from the second encoded image, and The user's gaze information is obtained based on the recovered image.

13. The electronic device according to claim 1, in, The superlens includes a surface formed by a plurality of nanostructures arranged on a substrate, wherein the superlens is configured to refract the light received from the polarization-changing element into different paths by modulating the phase of the light received from the polarization-changing element based on one of the polarization of the light received from the polarization-changing element and the wavelength of the light received from the polarization-changing element.

14. The electronic device according to claim 1, wherein, The superlens is also configured to refract light received from the polarization-changing element into different paths based on at least one of the dimensions of each of the plurality of nanostructures, the shape of each of the plurality of nanostructures, and the arrangement of the plurality of nanostructures on the substrate.

15. A method comprising: A first coded image is obtained by receiving first polarized light refracted into a first path by a superlens, wherein the superlens is configured to refract the incident light into different paths according to the polarization direction of the incident light; It can be determined whether the user's gaze information can be obtained from the first encoded image; Based on the identification that the user's gaze information could not be obtained from the first encoded image: A second coded image is obtained by receiving second polarized light refracted by the superlens into a second path; and The user's gaze information is obtained from the second encoded image.

Images