Near infrared (NIR) transparent organic light emitting diode (OLED) display

Abstract

A micro OLED display includes a substrate having a coating made of an IR transparent material, such as zinc selenide or zinc sulfide, to allow a pupil tracking camera capable of including a photodiode to be mounted directly behind the display. The coating is transparent to near infrared (NIR) radiation but opaque to visible light. At least one camera is configured to generate an image from the IR light, the camera disposed to receive light passing through the coating when the IR floodlight emits IR light toward an eye of a wearer of the assembly.

Description

Technical Field

[0001] The present application generally relates to near-infrared transparent organic light-emitting diode (OLED) displays, such as head-mounted displays for extended reality (XR) applications (eg, computer gaming). Background Art

[0002] As recognized herein, an organic light emitting diode (OLED) display may be used in a head mounted display (HMD) or head mounted device to facilitate eye tracking of the wearer for purposes such as presenting augmented reality and / or virtual reality images on the display. As understood herein, eye tracking of the wearer is necessary for many XR applications, and further, the camera for eye tracking is optimally placed directly in front of the pupil. However, the principles of the present invention further understand that the eye tracking camera is not optimally positioned. This is because if the camera is located between the substrate of the display and the eye, then the camera directly in front of the pupil will block the display, and if located outside the display, then the opaque coating on the display substrate that allows the display to appear black when the pixel is inactive will block the image of the pupil from reaching the camera. Summary of the invention

[0003] A micro OLED display includes a substrate with a coating made of an IR transparent material such as zinc selenide or zinc sulfide to allow a pupil tracking camera, which may include a photodiode, to be mounted directly behind the display, i.e., the display is between the pupil and the camera. The coating is transparent to near infrared (NIR) radiation but not to visible light.

[0004] Thus, in one aspect, an assembly includes at least one organic light emitting diode (OLED) display having an innermost surface and at least one substrate. A coating is on the substrate. The coating has an infrared (IR) transparent material that is transparent to at least near infrared (NIR) radiation but opaque to visible light. At least one camera is configured to generate an image from the IR light. The camera is positioned to receive light that passes through the coating.

[0005] The coating may comprise zinc selenide and / or zinc sulfide.

[0006] In some examples, the OLED display includes a micro OLED display.

[0007] In an example embodiment, the at least one IR illuminator is configured to emit IR towards the eyes of a wearer of the device.

[0008] In an example embodiment, the assembly includes at least one processor configured with instructions to: begin activating the camera or light or both the camera and light at the beginning of a blank period between a first period and a second period of rendering respective first and second frames of a desired image. The instructions are executable to power off the camera or light or both the camera and light at the end of the blank period. Alternatively, the instructions are executable to power off the camera or light or both the camera and light at some time during the second period of rendering the second frame of the desired image.

[0009] In an example, the camera is disposed on an outer surface of the substrate. In another example, the camera is connected to an arm and oriented to image the outer surface of the substrate. In some examples, the camera is an IR image sensor. In a non-limiting embodiment, the IR image sensor is directly bonded to the back side (outer surface) of the substrate opposite the innermost surface, and the substrate acts as a lens.

[0010] In another aspect, a method includes presenting frames of a desired image on at least one head mounted display (HMD). The method also includes generating at least one image of a pupil of a wearer of the HMD during at least an idle period between a first frame and a second frame.

[0011] The method may include illuminating the pupil with infrared (IR) light during the idle period. In an example, the method includes generating an image of the pupil using the IR light after the IR light passes through a layer of material that blocks visible light. The image can be used for eye tracking of the wearer.

[0012] The method may include powering the camera only during the idle period plus part but not all of the period of scanning the second frame. Alternatively, the method may include powering the camera only during the idle period.

[0013] In another aspect, a head mounted display (HMD) for generating at least one computer simulated desired image based at least in part on tracking at least one pupil of a wearer of the HMD includes at least one organic light emitting diode (OLED) display. The HMD also includes at least one pupil tracking camera configured to generate an image from infrared (IR) light. At least one substance is configured to pass IR light from the pupil to the camera to generate an image by the camera and to block visible light from the OLED display from the camera so that at least one computer simulation engine configured to generate the desired image can do so based at least in part on the image from the camera.

[0014] The details of the present application, both as to its structure and operation, may be best understood by reference to the accompanying drawings in which like reference numerals refer to like parts, and in which: BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a block diagram of an example system according to the principles of the present invention;

[0016] Figure 2 A first embodiment of a head mounted display (HMD) with a tracking camera is illustrated;

[0017] Figure 3 A second embodiment of an HMD with a tracking camera is shown;

[0018] Figure 4 An exploded view showing the various layers of an example HMD is shown;

[0019] Figure 5 depicts an IR light emitting diode (LED) in an example configuration for illuminating a pupil;

[0020] Figure 6 depicts a first example timing diagram of live image presentation and pause intervals between frames and activation of tracking cameras and / or illumination LEDs;

[0021] Figure 7 schematically depicts a portion of a frame during active image presentation;

[0022] Figure 8 Drawing and Figure 7 a second example timing diagram of consistent live image presentation and pause intervals between frames and activation of tracking cameras and / or illumination LEDs; and

[0023] Fig. 9 Example logic consistent with the principles of the invention is depicted in example flow chart format. DETAILED DESCRIPTION

[0024] The present disclosure generally relates to a computer ecosystem including aspects of a network of consumer electronics (CE) devices, such as, but not limited to, a computer gaming network. The system herein may include server and client components that may be connected via a network such that data may be exchanged between the client and server components. The client components may include one or more computing devices including a gaming console, such as a Sony or a game console made by Microsoft or Nintendo or other manufacturers; an extended reality (XR) head-mounted device, such as a virtual reality (VR) head-mounted device; an augmented reality (AR) head-mounted device; a portable television (e.g., a smart TV, a TV with Internet capabilities); a portable computer, such as a laptop computer and a tablet computer; and other mobile devices including smartphones and additional examples discussed below. These client devices can operate in various operating environments. For example, as an example, some of the client computers may use a Linux operating system, an operating system from Microsoft, or a Unix operating system, or an operating system produced by Apple, Inc. or Google, or a Berkeley Software Distribution or Berkeley Standard Suite (BSD) OS (including descendants of BSD). These operating environments can be used to execute one or more browsing programs, such as browsers made by Microsoft or Google or Mozilla or other browser programs that can access websites hosted by Internet servers discussed below. In addition, an operating environment according to the principles of the present invention can be used to execute one or more computer game programs.

[0025] A server and / or gateway may be used, which may include one or more processors that execute instructions that configure the server to receive and transmit data over a network such as the Internet. Alternatively, the client and server may be connected via a local intranet or virtual private network. The server or controller may be connected via, for example, a Sony Personal computers and other game consoles to instantiate.

[0026] Information may be exchanged between the client and the server over the network. To this end and for security, the server and / or client may include firewalls, load balancers, temporary storage devices and proxies, and other network infrastructure for reliability and security. One or more servers may form an apparatus for implementing a method of providing a secure community, such as an online social networking site or a player network, to network members.

[0027] The processor may be a single chip or a multi-chip processor that may execute logic with the aid of various lines such as address lines, data lines, and control lines, as well as registers and shift registers.

[0028] The components included in one embodiment may be used in other embodiments in any appropriate combination. For example, any of the various components described herein and / or depicted in the drawings may be combined, interchanged, or excluded from other embodiments.

[0029] “A system having at least one of A, B, and C” (similarly, “a system having at least one of A, B, or C” and “a system having at least one of A, B, C”) includes systems having A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B and C together.

[0030] Reference now Figure 1 , showing an example system 10, which may include one or more of the example devices mentioned above and further described below according to the principles of the present invention. The first of the example devices included in the system 10 is a consumer electronics (CE) device, such as an audio video device (AVD) 12, such as but not limited to a projector-based theater display system or an Internet-enabled TV with a TV tuner (equivalently, a set-top box that controls the TV). Alternatively, the AVD 12 may also be a computerized Internet-enabled ("smart") phone, a tablet computer, a notebook computer, a head-mounted device (HMD) and / or a head-mounted device (such as smart glasses or a VR head-mounted device), another wearable computerized device, a computerized Internet-enabled music player, a computerized Internet-enabled headset, a computerized Internet-enabled implantable device (such as an implantable skin device), etc. Regardless, it should be understood that AVD 12 is configured to carry out the present principles (eg, communicate with other CE devices to carry out the present principles, execute the logic described herein, and perform any other functions and / or operations described herein).

[0031] Thus, to carry out such principles, AVD 12 may be built from some or all of the components shown. For example, AVD 12 may include one or more touch-enabled displays 14, which may be implemented by a high-definition or ultra-high-definition "4K" or higher flat screen. One or more touch-enabled displays 14 may include, for example, a capacitive or resistive touch sensing layer with an electrode grid for touch sensing consistent with the principles of the present invention.

[0032] AVD 12 may also include one or more speakers 16 for outputting audio according to the principles of the present invention, and at least one additional input device 18, such as an audio receiver / microphone, for inputting audible commands to AVD 12 to control AVD 12. Example AVD 12 may also include one or more network interfaces 20 to communicate through at least one network 22 such as the Internet, WAN, LAN, etc. under the control of one or more processors 24. Therefore, interface 20 may be, but is not limited to, a Wi-Fi transceiver as an example of a wireless computer network interface, such as, but not limited to, a mesh network transceiver. It should be understood that processor 24 controls AVD 12 to perform the principles of the present invention, including other elements of AVD 12 described herein, such as controlling display 14 to present images thereon and receiving input therefrom. In addition, it should be noted that network interface 20 may be a wired or wireless modem or router, or other suitable interfaces, such as a wireless telephone transceiver or a Wi-Fi transceiver as mentioned above, etc.

[0033] In addition to the foregoing, the AVD 12 may also include one or more input and / or output ports 26, such as a high-definition multimedia interface (HDMI) port or a universal serial bus (USB) port, to physically connect to another CE device and / or a headphone port to connect a headphone to the AVD 12, so as to present the audio from the AVD 12 to the user through the headphone. For example, the input port 26 may be connected to a cable TV or satellite source 26a of audio-video content via a wire or wirelessly. Thus, the source 26a may be a separate or integrated set-top box or satellite receiver. Alternatively, the source 26a may be a game console or disk player containing content. The source 26a may include some or all of the components described below with respect to the CE device 48 when implemented as a game console.

[0034] The AVD 12 may further include one or more computer memories / computer readable storage media 28, such as disk-based or solid-state storage devices that are not transient signals, which in some cases are embodied as separate devices in the chassis of the AVD, or as a personal video recorder (PVR) or videodisc player for playing back AV programs inside or outside the chassis of the AVD, or as a removable storage medium or a server described below. In addition, in some embodiments, the AVD 12 may include a location or positioning receiver, such as but not limited to a cell phone receiver, a GPS receiver, and / or an altimeter 30, which is configured to receive geographic location information from a satellite or cell phone base station and provide the information to the processor 24 and / or determine the altitude at which the AVD 12 is placed in conjunction with the processor 24. Component 30 may also be implemented by an inertial measurement unit (IMU) or by an event-based sensor (e.g., an event detection sensor (EDS)), which typically includes a combination of an accelerometer, a gyroscope, and a magnetometer to determine the position and orientation of the AVD 12 in three dimensions. An EDS consistent with the present disclosure provides an output indicating a change in light intensity sensed by at least one pixel of a light sensing array. For example, if the light sensed by the pixel is decreasing, the output of the EDS may be -1; if it is increasing, the output of the EDS may be +1. No change in light intensity below a certain threshold may be indicated by an output binary signal of 0.

[0035] Continuing with the description of the AVD 12, in some embodiments, the AVD 12 may include one or more cameras 32, which may be thermal imaging cameras, digital cameras such as webcams, IR sensors, event-based sensors, and / or cameras integrated into the AVD 12 and controllable by the processor 24 to collect pictures / images and / or videos in accordance with the principles of the present invention. The AVD 12 may also include A transceiver 34 and other near field communication (NFC) element 36 may be provided to communicate with other devices using Bluetooth and / or NFC technology, respectively. An example NFC element may be a radio frequency identification (RFID) element.

[0036] Still further, AVD 12 may include one or more auxiliary sensors 38 (e.g., pressure sensors, motion sensors (e.g., accelerometers, gyroscopes, rotoscopes) or magnetic sensors, infrared (IR) sensors, optical sensors, speed and / or cadence sensors, event-based sensors, gesture sensors (e.g., for sensing gesture commands)) that provide input to processor 24. For example, one or more of auxiliary sensors 38 may include one or more pressure sensors that form a layer of touch-enabled display 14 itself, and may be, but are not limited to, piezoelectric pressure sensors, capacitive pressure sensors, piezoresistive strain gauges, optical pressure sensors, electromagnetic pressure sensors, etc.

[0037] The AVD 12 may also include an air TV broadcast port 40 for receiving OTA TV broadcasts to provide input to the processor 24. In addition to the foregoing, it should be noted that the AVD 12 may also include an infrared (IR) transmitter and / or an IR receiver and / or an IR transceiver 42, such as an IR data association (IRDA) device. A battery (not shown) may be provided to power the AVD 12, such as a kinetic energy harvester that can convert kinetic energy into electricity to charge the battery and / or power the AVD 12. A graphics processing unit (GPU) 44 and a field programmable gate array 46 may also be included. One or more tactile / vibration generators 47 may be provided to generate tactile signals that can be sensed by an individual holding the device or in contact with the device. Thus, the tactile generator 47 may vibrate all or part of the AVD 12 using an electric motor connected to an off-center and / or unbalanced weight via a rotatable shaft of the motor, such that the shaft may rotate under the control of the motor (which in turn may be controlled by, for example, a processor 24) to produce vibrations of various frequencies and / or amplitudes in various directions as well as force simulation.

[0038] A light source such as a projector, such as an infrared (IR) projector, may also be included.

[0039] In addition to the AVD 12, the system 10 may also include one or more other CE device types. In one example, the first CE device 48 may be a computer game console that can be used to send computer game audio and video to the AVD 12 via commands sent directly to the AVD 12 and / or through a server described below, while the second CE device 50 may include components similar to the first CE device 48. In the example shown, the second CE device 50 may be configured as a computer game controller controlled by the player or a head-mounted display (HMD) worn by the player. The HMD may include a transparent or opaque head-up display for presenting AR / MR content or VR content (more generally, extended reality (XR) content), respectively. The HMD may be configured as a glasses-type display, or as a larger VR-type display sold by computer game equipment manufacturers.

[0040] In the example shown, only two CE devices are shown, but it should be understood that fewer or more devices may be used. The devices herein may implement some or all of the components shown for AVD 12. Any of the components shown in the following figures may incorporate some or all of the components shown in the case of AVD 12.

[0041] Reference is now made to the aforementioned at least one server 52, which includes at least one server processor 54, at least one tangible computer-readable storage medium 56 (e.g., disk-based or solid-state storage device), and at least one network interface 58, which allows communication with other depicted devices over the network 22 under the control of the server processor 54, and which can actually facilitate communication between the server and client devices in accordance with the principles of the present invention. It should be noted that the network interface 58 can be, for example, a wired or wireless modem or router, a Wi-Fi transceiver, or other suitable interface, such as a wireless telephone transceiver.

[0042] Thus, in some embodiments, server 52 may be an Internet server or an entire server "cluster," and may include and perform "cloud" functionality, such that the devices of system 10 may access a "cloud" environment via server 52 in an example embodiment for, for example, a networked gaming application. Alternatively, server 52 may be implemented by one or more gaming consoles or other computers in the same room or nearby as the other devices shown.

[0043] The components shown in the following figures may include some or all of the components shown herein.Any user interfaces (UIs) described herein may be merged and / or extended, and UI elements may be mixed and matched between UIs.

[0044] The present invention principle can adopt various machine learning models, including deep learning models. The machine learning model consistent with the present invention principle can use various algorithms trained in a manner including supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, feature learning, self-learning and other forms of learning. Examples of such algorithms that can be implemented by computer circuit systems include one or more neural networks, such as convolutional neural networks (CNNs), recursive neural networks (RNNs) and a type of RNN referred to as long short-term memory (LSTM) networks. Support vector machines (SVMs) and Bayesian networks (Bayesian networks) can also be considered as examples of machine learning models. In addition to the network types described above, the models herein can also be implemented by classifiers.

[0045] As understood herein, performing machine learning may thus involve accessing training data and then training a model based on the training data so that the model can process additional data to make inferences. An artificial neural network / artificial intelligence model trained by machine learning may thus include an input layer, an output layer, and a plurality of hidden layers between the input layer and the output layer, the hidden layers being configured and weighted to make inferences about appropriate outputs.

[0046] Reference now Figure 2 , can be combined with Figure 1 An HMD 200 of any or all of the components shown in any of the devices in the may include an external display surface 202 on which is mounted at least one eye tracking (also referred to herein as pupil tracking) camera 204. It should be noted that Figure 2 The size of the camera 204 shown in FIG200 may not be drawn to scale. The camera 204 may be adhesively bonded to the external display surface 202 or otherwise engaged with the surface 202 on the outside of the IR-transmissive layer discussed further below (i.e., on the side of the IR-transmissive layer opposite the eyes of the wearer of the HMD 200). When the wearer is looking straight ahead, the camera 204 is preferably located on the surface 202 along the nominal line of sight of the wearer's pupil.

[0047] Figure 3 An alternative HMD 300 is depicted in which a camera 302 is mounted to the HMD by means of an arm 304 to be positioned along the nominal line of sight 306 of the wearer's pupil when the wearer is looking straight ahead. The arm 304 may be immovable, or it may be connected to move the camera toward and away from the display.

[0048] Figure 4 Depicted are the layers of an example HMD 400 having an innermost layer 402 of the HMD 400 establishing a cathode closest to the pupil 404 of the wearer's eye 406. Next to the innermost layer 402 is an electron transport layer 408, followed by a blocking layer 410, followed by an emissive layer 412, which may include an organic light emitting diode (OLED). Continuing from the innermost layer to the outermost, next to the emissive layer 412 may be a hole transport layer 414, followed by a hole injection layer 416 and an anode 418 on a substrate 420.

[0049] like Figure 4, an infrared coating, such as a near infrared (NIR) coating 422, is deposited on the outer surface of substrate 420, which allows IR light to propagate through layer 422 but blocks visible light from propagating through layer 422. For example, coating 422 may include zinc selenide and / or zinc sulfide. In some embodiments, coating 422 may be applied to the inner layer of substrate 420. When the wearer is looking straight ahead, as described above, camera 424 (such as any of the cameras described herein) is preferably juxtaposed with the outer surface of substrate 420 along the line of sight of pupil 404 to directly image IR light reflected from the pupil. Thus, camera 424 is a camera capable of generating images from IR light.

[0050] Figure 5 A display 500 consistent with the principles described herein, such as any display disclosed herein, is shown including Figures 2 to 4 , wherein one or more (preferably multiple) IR emitting illuminators 502 are arranged on the inner surface 504 of the display 500 to illuminate the pupils of the wearer's eyes. In the example shown, the illuminators 502 are equally spaced along the periphery 506 of the display 500, and it should be understood that other arrangements of the illuminators 502 may be implemented. After the OLED is formed, the illuminators 502 may be added to the display 500. Figure 4 The OLED emitters in the emissive layer 412 shown in the figure may be OLED emitters in the emissive layer 412, or they may be established by peripheral OLED emitters that are native to the emissive layer 412 itself and therefore can perform two tasks, one of emitting light to form part of the image presented on the display during active image formation, and the other of emitting IR illumination light for the purpose of pupil tracking consistent with the principles discussed herein.

[0051] Reference now Figure 6 The OLED display in the HMD usually scans horizontal lines from top to bottom during the period "P" to present a frame of the desired image. During the period "P", the synchronization voltage V sync The voltage is driven from a low voltage (e.g., zero) to a high voltage, driven to V sync The length of period "P" may be, for example, between two and twelve milliseconds (2ms to 12ms), and in one embodiment may be 8.5ms, and a frame refresh rate of 60 frames per second (fps), 120fps, or 240fps may be used (as just three examples). During the frame rendering period "P", the pupil tracking camera is not powered.

[0052] On the other hand, between frame rendering, V sync Returns to low during an idle period Δt, during which no image rendering is required, and the pupil tracking camera described herein is activated, as indicated by Figure 6Tracking activation voltage curve V track In an example implementation, the length of the idle period Δt may be, for example, from two milliseconds to six milliseconds (2ms to 6ms). In addition, an IR lighting lamp (eg Figure 5 ) is activated to emit IR light during a period Δt so that when the camera is actively imaging the eye, the eye is actively illuminated by IR radiation. By timing the IR eye tracking to occur when no active frame rendering is taking place, the camera only detects IR from the pupil of the eye and not IR that may be emitted as part of frame rendering. Figure 6 In the example shown in , the camera (and IR illuminator) are powered off at the beginning of the next period "P" of frame rendering.

[0053] Figure 7 and Figure 8 The recognition that some pupil tracking may occur during active frame rendering is illustrated under the following conditions. As shown by LED activation lines 700 on an OLED display 702 (such as any display described herein), at some time during frame rendering period "P" (referred to herein as "scan time t"), line 1 is scanned first, followed by the lines below it, until the Nth line is scanned just above the nominal position of the pupil relative to the display 704. Lines N+1 below the Nth line and below are then scanned (after "scan time t") to render a frame of the desired image during period "P", thereby presenting a frame of the desired image.

[0054] Figure 8 The present invention illustrates the principle of understanding that the pupil can be tracked during part of the period "P" to present the frame of the desired image until the completion of Figure 7 More specifically, Figure 6 As in the example of Figure 8 In the period "P", the synchronous voltage V sync Driven high from low voltage, and between frame rendering, V sync Returns to low within period Δt. During period Δt, activates the pupil tracking camera described herein, as indicated by Figure 8 Tracking activation voltage curve V track In addition, IR lighting (e.g. Figure 5 LED 502) shown in FIG. 5 may be activated during a period Δt so that the eye is actively illuminated by IR radiation when the camera is actively imaging the eye.

[0055] However, with Figure 6Unlike the case of , after the next active frame rendering period "P" begins, the pupil tracking camera (and, if necessary, the IR illuminator) can remain powered until the "scan time t" within the next frame rendering period is reached, it being understood that during this time, the nominal gaze of the pupil is not yet directed to the active frame rendering. Once the "scan time t" is reached, the pupil camera (and, if necessary, the IR illuminator) can be deactivated until the next idle period Δt begins.

[0056] Fig. 9 Example logic consistent with the disclosure herein that may be implemented by any one or more of the processors described herein is shown. Beginning at block 900, a desired image from, for example, a computer simulation (e.g., a computer game) is rendered on a display (e.g., any of the HMDs described herein). Moving to block 902, in the example Figure 6 between the frames shown in or from the idle period Δt until Figure 7 and Figure 8 The pupil tracking camera described herein and, if necessary, the IR illuminator are activated to generate an IR image of one or more pupils of the wearer at a "scan time t" of the next active frame rendered as shown in FIG. At box 904, the image is input to one or more machine learning (ML) models to derive gaze direction information from the image, the gaze direction information is output by the ML model at box 906, and the gaze direction information is input to the game engine or other desired image source of box 900 at box 908.

[0057] The one or more ML models used at block 904 may be trained on a set of ground-truth pupil images and ground-truth gaze direction annotations.

[0058] While particular embodiments are herein shown and described in detail, it is to be understood that the subject matter encompassed by the present invention is limited only by the claims.

Claims

1. An assembly comprising: at least one organic light emitting diode (OLED) display comprising at least one innermost surface and at least one substrate; a coating on the substrate, the coating being transparent to at least near infrared (NIR) radiation but opaque to visible light; and At least one camera configured to generate an image from the IR light, the camera positioned to receive the light that has passed through the coating.

2. The assembly of claim 1, wherein the coating comprises zinc selenide.

3. The assembly of claim 1, wherein the coating comprises zinc sulfide.

4. The assembly of claim 1, wherein the OLED display comprises a micro OLED display.

5. The assembly according to claim 1, comprising: At least one IR illuminator configured to emit IR towards the eyes of a wearer of the device.

6. The assembly of claim 1, comprising at least one processor configured with instructions to: At the beginning of a vacant period between a first period and a second period of rendering respective first and second frames of a desired image, activation of the camera or the light or both of the camera and the light begins.

7. The assembly of claim 6, wherein the instructions are executable to: The camera or the light or both are powered off at the end of the idle period.

8. The assembly of claim 6, wherein the instructions are executable to: The camera or the light or both are powered off at some time during the second period of rendering the second frame of the desired image.

9. The assembly of claim 1, wherein the camera is disposed on an outer surface of the substrate.

10. The assembly of claim 1, wherein the camera is connected to an arm and oriented to image an outer surface of the substrate.

11. A method comprising: presenting frames of the desired image on at least one head mounted display (HMD); and At least one image of a pupil of a wearer of the HMD is generated during at least an empty period between a first frame and a second frame.

12. The method of claim 11, comprising illuminating the pupil with infrared (IR) light during the idle period.

13. The method of claim 12, comprising generating the image of the pupil using the IR light after the IR light passes through a layer of material that blocks visible light.

14. The method of claim 11, comprising using the image for eye tracking of the wearer.

15. The method of claim 11, comprising generating the image using an IR camera.

16. The method of claim 15, comprising powering the camera only during part but not all of the idle period plus a period of scanning the second frame.

17. The method of claim 15, comprising powering the camera only during the idle time periods.

18. A head mounted display (HMD) for generating at least one computer simulated desired image based at least in part on tracking at least one pupil of a wearer of the HMD, the head mounted display comprising: at least one organic light emitting diode (OLED) display; at least one pupil tracking camera configured to generate images from infrared (IR) light; at least one substance configured to pass IR light from the pupil to the camera to generate the image by the camera and to block visible light from the OLED display from the camera, such that at least one computer simulation engine configured to generate the desired image is able to do so based at least in part on the image from the camera.

19. The HMD of claim 18, wherein the substance is disposed between the pupil and the camera when the wearer is wearing the HMD.

20. The HMD of claim 18, comprising at least one processor configured to power on the camera during at least one idle period between first and second periods of scanning respective first and second frames of a desired image.

Images