Under-display sensor operation

Abstract

Example methods include programming pixels of a plurality of pixels of a display of a computing device based on image data of a frame during a non-emission period of the frame among a plurality of frames; causing the pixels of the plurality of pixels to emit light during an emission period of the frame, wherein an amount of light emitted by a pixel during the emission period of a particular frame is based on the programming of the particular frame; and synchronizing the operation of one or more sensors and the operation of the plurality of pixels by causing at least one or more sensors to alternately emit electromagnetic radiation through the display during the emission period and the non-emission period.

Description

[0001] This application claims the benefit of U.S. Provisional Application Serial No. 63 / 048,501, filed Jul. 6, 2020, the entire content of which is incorporated herein by reference. BACKGROUND OF THE INVENTION

[0002] Computing devices such as cellular phones and so-called smart phones may include a display through which images (including sequences of images forming video, animation, etc. and / or computer-generated user interfaces and other forms of images) are presented. Since smart phones and other types of power-constrained devices such as laptop computers, smart watches, smart glasses, smart hubs, extended reality (XR) devices, etc. may consume power from a power-constrained source such as a battery, these power-constrained devices may employ a more power-efficient display such as an organic light-emitting diode (OLED) display (including an active matrix OLED - AMOLED display) compared to a conventional LED display.

[0003] In addition, to provide a more comfortable viewing experience, power-constrained devices may employ a larger display. To increase the size of the display, various sensors may be configured to operate under the display (which may be referred to as "under-display sensors"), thereby avoiding notches, holes, or other modifications to the display that detract from the viewing experience. Although the size and shape of a display that allows under-display sensors (which may be referred to as a "through-display") may be improved compared to a display with a notch, hole, or other modification (which may be referred to as a "modified display"), the through-display may present an image that contains more noise compared to the image presented by the modified display. SUMMARY OF THE INVENTION

[0004] Aspects of the technology relate to a computing device configured to synchronize the operation of a display (such as an organic light-emitting diode - OLED - display or an active matrix OLED - AMOLED - display) with operations located beneath the display through the operation of sensors of the display. In operation, one or more sensors located beneath the display may emit electromagnetic radiation that passes directly through the display. For example, a proximity sensor may emit infrared (IR) light through the display, receive a return signal that includes some of the emitted light, and determine a distance between the sensor and another object based on the return signal. The emission of electromagnetic radiation by one or more sensors may interfere with the operation of the display. For example, the electromagnetic radiation may change the brightness value of one or more pixels in the display, which may be undesirable for a user to see. According to one or more techniques of the present disclosure, the computing device may synchronize the operation of the sensors with the operation of the display. For example, one or more sensors may emit electromagnetic radiation at an appropriate time during the operation of the display (e.g., just before the pixels of the display are to be programmed) to minimize the visible effects of the brightness value change.

[0005] Aspects of the technology relate to a computing device configured to synchronize the operation of a display (such as an organic light-emitting diode - OLED - display or an active matrix OLED - AMOLED - display) with operations located beneath the display through the operation of sensors of the display. In operation, one or more sensors located beneath the display may emit electromagnetic radiation that passes directly through the display. For example, a proximity sensor may emit infrared (IR) light through the display, receive a return signal that includes some of the emitted light, and determine a distance between the sensor and another object based on the return signal. The emission of electromagnetic radiation by one or more sensors may interfere with the operation of the display, such as by changing the brightness value of one or more pixels in the display, which may be undesirable for a user to see. Depending on the emission timing of the electromagnetic radiation, the brightness value may increase or decrease. According to one or more techniques of the present invention, the computing device may synchronize the operation of the sensors with the operation of the display. For example, one or more sensors may emit electromagnetic radiation at alternating times to cause alternating increases and decreases in the pixel brightness value. The alternating increases in brightness reduction may cancel each other out visually, thereby minimizing the visible effects of the brightness value change.

[0006] In one example, aspects of the technology relate to a computing device that includes: a display including a plurality of pixels; one or more sensors positioned under the display and configured to emit electromagnetic radiation through the display during operation; and one or more processors configured to: program pixels among the plurality of pixels during a non-emission period of a frame in a plurality of frames based on image data of the frame; cause the pixels among the plurality of pixels to emit light during an emission period of the frame, wherein an amount of light emitted by a pixel during the emission period of a particular frame is based on the programming of the particular frame; and synchronize the operation of the one or more sensors with the operation of the plurality of pixels by causing the one or more sensors to emit electromagnetic radiation alternately during the emission period and the non-emission period.

[0007] In another example, aspects of the technology relate to a method that includes: programming pixels among a plurality of pixels of a display of a computing device during a non-emission period of a frame in a plurality of frames based on image data of the frame; during an emission period of the frame, causing the pixels among the plurality of pixels to emit light, wherein an amount of light emitted by a pixel during the emission period of a particular frame is based on the programming of the particular frame; and synchronizing the operation of the one or more sensors and the operation of the plurality of pixels by causing the one or more sensors to emit electromagnetic radiation through the display alternately during the emission period and the non-emission period.

[0008] Details of one or more examples of the subject matter of this specification are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1A and 1B are diagrams illustrating example computing devices configured to perform aspects of the image modification techniques described in this disclosure.

[0010] Figure 2 is a more detailed diagram illustrating when configured to perform aspects of the image modification techniques described in this disclosure Figure 1A and Figure 1B of the example computing device shown.

[0011] Figure 3 is a more detailed diagram illustrating an example pixel circuit of a display system included in Figure 2 the example computing device shown.

[0012] Figure 4 is a conceptual diagram illustrating various signals of a display of the device.

[0013] Figure 5It is a conceptual diagram of various signals of the display of the illustrated device.

[0014] Figure 6 It is a conceptual diagram of various signals of the display of the illustrated device.

[0015] Figure 7 It is a conceptual diagram of various signals of the display of a device having synchronous operation of electromagnetic emission under the display according to one or more techniques of the present disclosure.

[0016] Figure 8 It is a block diagram of components of a device that synchronizes the operation of electromagnetic emission under the display with the display operation according to one or more techniques of the present disclosure.

[0017] Figure 9 It is a conceptual diagram of various signals of the display of a device having synchronous operation of electromagnetic emission under the display according to one or more techniques of the present disclosure.

[0018] Figures 10A - 10C It is a conceptual diagram of signals of a device for synchronous operation of electromagnetic emission under the display according to one or more techniques of the present disclosure.

[0019] Figure 11 It is a conceptual diagram of various signals of the display of a device having synchronous operation of electromagnetic emission under the display according to one or more techniques of the present disclosure.

[0020] Figure 12 It is a conceptual diagram of various signals of the display of a device having synchronous operation of electromagnetic emission under the display according to one or more techniques of the present disclosure.

[0021] Figure 13 It is a conceptual diagram of various signals of the display of a device having synchronous operation of electromagnetic emission under the display according to one or more techniques of the present disclosure.

[0022] Figure 14 It is a flowchart of a method for synchronizing the operation of a display with the operation of a sensor under the display according to one or more techniques of the present disclosure.

[0023] Figure 15 It is a flowchart of a method for synchronizing the operation of a display with the operation of a sensor under the display according to one or more techniques of the present disclosure. Detailed Description

[0024] Figure 1A and Figure 1Bis a view showing an example computing device 100 configured to perform aspects of the image modification techniques described in this disclosure. The computing device 100 may include a display 110 and an under-display sensor 120 (“UDS 120”). Figure 1A Shows a front perspective view of the computing device 100. Figure 1B Shows an example cross-sectional view of the computing device 100.

[0025] First, referring to Figure 1A the example of, the computing device 100 may represent any type of computing device, such as a smart phone, a smart TV, a smart watch, smart glasses, a laptop computer, a handheld game console, a smart hub, a smart display, etc. The display 110 may include an array of light-emitting pixels forming a display panel. In operation, the display 110 may display an image by activating the light-emitting pixels according to image data. The display 110 may be, for example, an active-matrix organic light-emitting diode (AMOLED) display or other type of OLED display, a light-emitting diode (LED) display, and / or a liquid crystal display (LCD). The computing device 100 includes the UDS 120, which, when considered from the front perspective view shown in Figure 1A the example of, is positioned below the display 110.

[0026] Referring to Figure 1B , the top layer of the cross-section of the computing device 100 includes the display 110, which represents an arrangement of a cover glass 106, a polarizing film 108, a display panel 109, a transparent PET film 111, and a rear cover 112 (of the display 110 rather than the computing device 100). The polarizing film 108 is arranged below the cover glass 106. The display panel 109 representing an array of light-emitting pixels is arranged below the polarizing film 108, where the transparent PET film 111 (representing a type of polyester film) is arranged below the display panel 109. The rear cover 112 is arranged below the transparent PET film 111.

[0027] The UDS 120 is at least partially arranged below the display 110. For example, when viewed from a cross-sectional view of the computing device 100, the UDS 120 may be located below the display panel 110. In some examples, the UDS 120 may be coupled to the motherboard or other logic circuitry of the computing device 100, while in other examples, the UDS 120 may be coupled to the rear cover 112 of the display 110.

[0028] UDS 120 may include a transmitter 124 and a receiver 114. In operation, the transmitter 124 emits and / or directs electromagnetic radiation through at least a portion of the pixel array that forms the display panel 109, for example, in the form of transmitted pulses 122. The receiver 114 may receive return pulses 116 of electromagnetic energy that pass through at least a portion of the pixel array that forms the display panel 109.

[0029] UDS 120 may represent, for example, an infrared (IR) sensor that emits and receives electromagnetic energy in the IR band of the electromagnetic spectrum. Thus, UDS 120 may represent a near-IR sensor or a short-wavelength IR sensor. Additionally, in some examples, UDS 120 may represent a UV sensor, a LIDAR sensor, or a RADAR sensor. In some examples, UDS 120 may emit and receive electromagnetic energy within a range of electromagnetic spectrum bands. For example, the electromagnetic radiation emitted by UDS 120 may include one or more of infrared radiation, ultraviolet radiation, or radio wave radiation. In some cases, UDS 120 may represent more than one electromagnetic sensor 120.

[0030] When representing an electromagnetic sensor, UDS 120 can facilitate, for example, remote and / or wireless control of devices such as televisions, set-top boxes, sound systems, gaming systems, smart TVs, smart speakers, smart watches, smart glasses, etc. In these electromagnetic examples, UDS 120 may provide IR illumination. When used for IR illumination, UDS 120 may project IR radiation onto an area and receive the IR radiation reflected from an object in that area. In this way, UDS 120 may represent an electromagnetic sensor configured to emit and receive IR radiation in combination with a visible light camera to capture an image of an area in low light.

[0031] When representing an electromagnetic sensor, UDS 120 uses the transmitter 124 to emit electromagnetic radiation, for example, IR pulses, and the transmitter 124 is configured to interfere with the circuitry within the pixel array of the display panel 109. The IR interference can cause visual display artifacts to appear on the display panel 109. As an example, the IR interference may cause a pixel to glow, resulting in a dot appearing on the display panel 110 above UDS 120. The dot brightness intensity may be higher than its programmed level. As another example, the IR interference may cause a pixel to dim, resulting in a dot appearing on the display panel 110 above UDS 120. The dot brightness intensity may be lower than its programmed level.

[0032] The size, shape, and intensity of bright / dark spots may depend on the characteristics of the UDS 120. For example, a larger-sized emitter 124 may produce more IR interference that results in larger-sized spots. In some examples, electromagnetic radiation of a smaller wavelength may cause additional interference and result in different brightness intensities of pixels near or above the UDS 120. The UDS 120 may have different effects on pixels within the pixel array of the display panel 109. For example, pixels located immediately adjacent to the location of the UDS 120 (e.g., directly above or near the UDS 120) may be more interfered with than pixels that are further away from the UDS 120. In some examples, an emitter 124 with a wider field of view may produce a larger spot due to the alteration of more pixels within the display panel 109.

[0033] Accordingly, the display 110 may allow one or more sensors to operate beneath the display 110, where sensor signals and other external signals may pass through the various layers of the display 110 (which may be represented as "through the display 110"). To facilitate the through nature of the display 110, during the construction of the through display 110, various back covers on the display 110 may be omitted at locations above and / or near the location of the UDS 120 beneath the display 110. That is, the through display 110 may include a back cover plate that is formed by removing foam (or other types of pads) and copper (Cu) film in the area of the back cover plate 112 above and / or near the UDS 120 location. The omission of the back cover plate 112 allows sensor signals and other external signals (e.g., light) to pass through the display 110, where examples of such UDS 120s include ambient light sensors, cameras, fingerprint sensors, proximity sensors, or other types of optical sensors, electromagnetic sensors, etc.

[0034] In accordance with one or more techniques of the present disclosure, the computing device 100 may synchronize the operation of the display 110 with the operation of the UDS 120. As an example, the UDS 120 may emit electromagnetic radiation at a point that minimizes the amount of time that a white dot caused by the emission may be visible during the operation of the display 110. For example, as discussed in further detail below, the UDS 120 may emit electromagnetic radiation just before the pixels of the display 110 directly above the UDS 120 are programmed. Since the programmed pixels of the display 110 may cancel out any alterations caused by the emission of electromagnetic radiation from the UDS 120, emitting electromagnetic radiation just before programming may reduce the amount of time that a white dot caused by the emission may be visible.

[0035] As another example, the UDS 120 may emit electromagnetic radiation at points in the operation of the display 110 to counteract the visual impact of the points caused by the emission. As discussed further in detail below, depending on when the UDS 120 emits electromagnetic radiation, the result may be a dark spot or a bright spot. Thus, the UDS 120 may emit electromagnetic quanta at alternating emission points synchronously with the operation of the display 110, thereby alternately causing dark spots and bright spots. The alternating dark spots and bright spots may cancel each other out visually, thereby reducing the emission impact of the UDS 120.

[0036] Figure 2 is shown in more detail Figure 1A and Figure 1B a view of a computing device as shown in, when configured to perform various aspects of the image modification techniques described in the present disclosure. As Figure 2 shown in the example of, the display 200 may represent an example of the display 110, where the display 200 represents an OLED display system including a light-emitting pixel array 212. Each light-emitting pixel includes an OLED.

[0037] A driver including a SCAN / EM driver 208 and a data driver 210 may drive the OLED display 200. The SCAN / EM driver 208 may be integrated, i.e., a stacked row line driver. In some examples, the SCAN / EM driver 208 identifies the pixel rows in the display, and the data driver 210 provides data signals (e.g., voltage data) to the pixels in the selected rows to cause the OLEDs to output light according to the image data. Signal lines such as scan lines, EM lines, and data lines may be used to control the pixels to display an image on the display. Although Figure 2 the OLED display 200 is shown as having the SCAN / EM driver 208 on one side, the SCAN / EM driver 208 may be arranged on both the left and right sides of the display 200 to improve the driving performance (e.g., speed) compared to such a driver being placed only on the left or only on the right side of the OLED display 200.

[0038] The OLED display 200 includes a pixel array 212, and the pixel array 212 includes a plurality of light-emitting pixels, e.g., pixels P11 to P43. A pixel is a small element on the display that can change color based on the image data supplied to the pixel. Each pixel within the pixel array 212 can be individually addressed to produce various color intensities. The pixel array 212 extends in a plane and includes rows and columns.

[0039] Each row extends horizontally across the pixel array 212. For example, the first row 220 of the pixel array 212 includes pixels P11, P12, and P13. Each column extends vertically down the pixel array 212. For example, the first column 230 of the pixel array 212 includes pixels P11, P21, P31, and P41. For purposes of illustration, Figure 2 only a subset of the pixels are shown, and the OLED display 200 may include hundreds, thousands, or millions of pixels (and potentially more in high-resolution displays). In fact, there may be millions of pixels in the pixel array 212. A greater number of pixels can result in a higher resolution.

[0040] The OLED display 200 includes a SCAN / EM driver 208 and a data driver 210. The SCAN / EM driver supplies SCAN and EM signals to the rows of the pixel array 212. In Figure 2 the example, the SCAN / EM driver 208 supplies scan signals via scan lines S1 to S4 and supplies EM signals via EM lines E1 to E4 to the corresponding pixel rows. The data driver 210 supplies signals to the columns of the pixel array 212. In Figure 2 the example, the data driver 210 supplies data signals via data lines D1 to D4 to the pixel columns.

[0041] Each pixel in the pixel array 212 can be addressed by a horizontal scan line and EM line and a vertical data line. For example, pixel P11 can be addressed by scan line S1, EM line E1, and data line D1. In another example, pixel P32 can be addressed by scan line S3, EM line E3, and data line D2.

[0042] The SCAN / EM driver 208 and the data driver 210 supply signals to the pixels to enable the pixels to reproduce an image. The SCAN / EM driver 208 and the data driver 210 supply signals to the pixels via scan lines, emission lines, and data lines. To supply signals to the pixels, the SCAN / EM driver 208 selects scan lines and controls the emission operation of the pixels. The data driver 210 supplies data signals to the pixels that can be addressed by the selected scan lines to light the selected OLEDs according to the image data.

[0043] The scan lines address each frame in sequence. A frame is a single image in a series of images to be displayed. The scan direction determines the order in which the scan lines are addressed. In the OLED display 200, the scan direction is from the top to the bottom of the pixel array 212. For example, scan line S1 is addressed first, then scan line S2, then S3, and so on.

[0044] The OLED display 200 includes a controller 206 that receives display input data 202. The controller 206 generates a scan control signal 222 and a data control signal 224 from the display input data 202. The scan control signal 222 may drive the SCAN / EM driver 208. The data control signal 224 may drive the data driver 210. The controller 206 controls the timing of the scan signal and the EM signal via the scan control signal 222. The controller 206 controls the timing of the data signal via the data control signal 224.

[0045] The controller 206 may also control the timing of the UDS 120. The controller 206 may control the timing of the UDS 120 via a sensor control signal 226, which may also be referred to as a synchronization signal. The sensor control signal 226 may include start and stop signals. The controller 206 may send a start signal to the UDS 120 to allow the UDS 120 to emit electromagnetic radiation, e.g., an IR pulse. The controller 206 may send a stop signal to the UDS 120 to cause the UDS 120 to abort the emission of electromagnetic radiation, or to prevent the UDS 120 from emitting electromagnetic radiation.

[0046] The controller 206 may synchronize the scan control signal 222, the data control signal 224, and the sensor control signal 226 to reduce interference between the emission of the UDS 120 and the light emission of the pixels. For example, the controller 206 may synchronize the sensor control signal 226 with the scan control signal 222 to prevent the UDS 120 from emitting electromagnetic energy during the EM signal pulse of a pixel row located near the UDS 120. The controller 206 may also synchronize the sensor control signal 226 with the scan control signal 222 to prevent the UDS 120 from emitting electromagnetic radiation during the scan period for a pixel row located near the UDS 120.

[0047] Figure 3 is shown in more detail Figure 2 is an example pixel circuit view of an example pixel circuit of a display system included in a computing device shown in the example of. In Figure 3 is shown in more detail in the example of the display system 200 (discussed in the example above with respect to Figure 2 of the example) the pixel P11. The pixel P11 represents an active matrix OLED (AMOLED) pixel. The pixel P11 may be addressed by a horizontal scan line S1, an emission line E1, a vertical data line D1, and an initialization signal line I1. The pixel P11 receives a scan signal "SCAN" from the scan line S1, a data voltage "DATA" from the data line D1, and an emission signal "EM" from the emission line E1. The pixel P11 also receives an initialization signal "SINIT" from the initialization signal line I1. The pixel P11 receives a power supply voltage VDD and an initial reference voltage VINIT The pixel P11 is connected to the common ground line VSS.

[0048] The pixel P11 includes an organic light-emitting diode (OLED) 320. The OLED 320 includes an organic compound layer that emits light in response to a current I OLED The organic layer is positioned between two electrodes: an anode and a cathode. A current source circuit 310 receives a supply voltage VDD and drives the OLED 320 to emit light.

[0049] The pixel P11 includes a storage capacitor C ST The storage capacitor C ST can maintain the gate voltage V during the illumination of the pixel P11 G .

[0050] The pixel P11 further includes a plurality of p-channel switching thin film transistors (TFTs). The switching TFTs include a signal TFT (T SW_S ), an initialization TFT (T SW_I ), and an emission TFT (T SW_E ). In some examples, the switching TFTs can be n-channel transistors with control signals of opposite polarities.

[0051] During operation, the switching TFT T SW_S starts and stops charging the storage capacitor C based on a SCAN signal received from the scan line S1 ST . During the addressing period, the scan line S1 turns on the switching TFT T SW_S . The switching TFT T SW_S provides the data voltage DATA from the data line D1 to the storage capacitor C ST and the current source circuit 310.

[0052] The pixel P11 is programmed by the following control signals: SCAN, SINIT, EM, and DATA. The OLED current I OLED varies with the gate voltage V G . When the gate voltage V G is stable, the pixel P11 maintains a stable brightness throughout the frame time, and the displayed light corresponds to the programmed supplied image data. The frame time or frame period is the amount of time between the start of one frame and the start of the next frame. The frame time can be the reciprocal of the frame rate of the display system. For example, a frame rate of 60 frames per second (fps) corresponds to a frame time of 1 / 60 second or 0.0167 second.

[0053] When the current source circuit 310 receives the data voltage DATA through the switching TFT T SW_S , the current source circuit 310 provides a specified current I to the OLED 320 based on the received data voltage DATAOLED such that the OLED 320 emits light according to the current I OLED The intensity or brightness of the emitted light depends on the amount of the applied current I OLED . Compared with a lower current that causes a lower relative brightness, a higher current can cause a brighter light. Thus, the intensity of the light emitted from the OLED 320 is based on the data voltage DATA corresponding to the image data for each pixel. The storage capacitor C ST maintains the pixel state (e.g., stores the gate voltage level V G ), such that the pixel P11 remains continuously illuminated after the addressing period.

[0054] Exposure to electromagnetic radiation may cause a leakage current I leakage to flow from the storage capacitor C ST through the TFT T SW_I . The leakage current I leakage may affect the OLED current I OLED , causing a change in the illumination level of the pixel P11.

[0055] Although Figure 2 and Figure 3 illustrate example components of an OLED display, the techniques may be applied to any panel display including a pixel array. For example, the process for reducing artifacts due to electromagnetic radiation may be applied to light emitting diode (LED) panels, liquid crystal displays (LCD), and plasma display panels (PDP).

[0056] Figure 4 is a conceptual diagram showing various signals of a display of a device. For the k-th pixel in the n-th pixel row of a display such as the display 110, Figure 4 the signals EM[n], SINIT[n], SCAN[n], and DATA[k] may correspond to the signals EM, SINIT, SCAN, and DATA from Figure 3 . As shown in Figure 4 , during a non-emission period (e.g., when EM[n] is high), a controller (e.g., one or more processors that generate the signals EM[n], SINIT[n], SCAN[n], and DATA[k], such as Figure 2 the controller 206) may initialize the gate voltage level VG (e.g., erase, to V INIT ) (e.g., in the case where T SW_I is a p-channel switch, the controller may output SINIT[n] as low to initialize the gate voltage level, and in the case where T SW_I is an n-channel switch, the controller may output SINIT[n] as high to initialize the gate voltage level) in order to turn off the switch T SW_IAfter initialization, the controller can disconnect switch T by outputting SCAN[n] low. SW_S while the programming gate voltage level V G In this way, the controller can cause the circuit to store a voltage level representing the emission intensity of a particular pixel. When the controller outputs EM[n] low, the display can operate during an emission period, where the emission element (e.g., Figure 3 320) emits electromagnetic radiation (e.g., visible light) having an intensity based on the gate voltage level V G .

[0057] Figure 5 is a conceptual diagram showing various signals of the display of the device. Figure 5 The signals of Figure 1A may represent the signals of the display of a computing device (such as the display 110 of the computing device 100 of Figure 5 ). As shown in Figure 4 , the operation of the display can be divided into non-emission periods 504A and 504B (collectively referred to as "non-emission period 504") and emission periods 506A and 506B (collectively referred to as "emission period 506"). As discussed above (e.g., with reference to Figure 4 ), the controller 206 may program the gate voltage level of the pixels during the non-emission period 504 and may cause the emission element to emit electromagnetic radiation having an intensity based on its corresponding gate voltage level during the emission period 506. For example, during the emission period 506A, the emission element can emit electromagnetic radiation at the intensity (e.g., programmed illumination level) programmed during the non-emission period 504A. Similarly, during the emission period 506B, the emission element may emit electromagnetic radiation at the intensity programmed during the non-emission period 504B. The non-emission period 504 may be referred to as pixel blanking time / pixel off time. Each frame of the image data may include a corresponding non-emission period during which the pixels are programmed and an emission period during which the pixels emit a certain amount of light based on the programming.

[0058] Figure 6 is a conceptual diagram showing various signals of the display of the device. Figure 6 may correspond to Figure 5 , but with sensor emission and the resulting change in brightness level added. As shown in Figure 6 , when a sensor such as UDS 120 emits electromagnetic radiation during the emission period of the emission period 506, the brightness level of one or more pixels above the sensor may change during the remainder of the emission period. For example, as discussed above, the electromagnetic radiation of the sensor can at least partially disconnect switch T SW_I , which may result in leakage current (e.g., Figure 3 I of Leakage) increases. This leakage current may cause the gate voltage stored in capacitor C ST to decrease, which in turn may cause current I OLED to increase (e.g., in the case where the current source circuit 310 of Figure 3 is p-channel). This increased current I OLED may cause the brightness value of the emitting element (e.g., OLED 320) to increase. In this way, the electromagnetic radiation emitted by one or more sensors may modify the stored voltage level.

[0059] According to one or more techniques of the present disclosure, a controller (e.g., controller 206) may synchronize the operation of the display and the sensors below the display to minimize the visual appearance of the brightness change caused by the sensor emissions. For example, the controller may cause one or more sensors to emit electromagnetic radiation during a specific portion of the emission period of a frame. As an example, to cause one or more sensors to emit electromagnetic radiation during a specific portion of the emission period, the controller may cause one or more sensors to emit electromagnetic radiation near the end of the emission period. Figure 7 is a conceptual diagram showing various signals of a display of a device having synchronous operation of electromagnetic emission under a display according to one or more techniques of the present disclosure. As Figure 7 shown, by causing one or more sensors to emit electromagnetic radiation near the end of the emission period, the controller can reduce the amount of time during which the emitting element (e.g., OLED) emits light at a changed brightness level.

[0060] In some examples, to cause one or more sensors to emit electromagnetic radiation near the end of the emission period, the controller may do one or both of the following: cause one or more sensors to emit electromagnetic radiation during a final sub-portion of the emission period; and inhibit one or more sensors from emitting electromagnetic radiation during portions of the emission period other than the final sub-period of the emission period. In some examples, the final sub-portion may be defined as the last percentage of the total time of the emission period. For example, the final sub-portion may be the last 1%, 5%, 10%, 20%, 30%, 40% of the emission period. In some examples, the final sub-portion may be defined as a time offset from the end of the emission period. For example, the final sub-portion may start 1 millisecond (ms), 2 ms, 5 ms, 10 ms, 50 ms from the end of the emission period and terminate at the end of the emission period.

[0061] In some examples, to cause one or more sensors to emit electromagnetic radiation near the end of a transmission period, the controller may do one or both of the following: cause one or more sensors to emit electromagnetic radiation after a predetermined delay period; and inhibit causing one or more sensors to emit electromagnetic radiation before the expiration of the predetermined delay period. The predetermined delay period may be an amount of time from a specific point in a frame. For example, the predetermined delay period may be an amount of time from the start of the transmission period, an amount of time from the start of a non-transmission period before the transmission period, or another characteristic of the signal.

[0062] Figure 8 is a block diagram showing components of a device that synchronizes operation of electromagnetic emission under a display with operation of the display in accordance with one or more techniques of the present disclosure. As discussed above, one or more processors of the device (e.g., controller 206) may synchronize operation of the display with operation of sensors under the display. In some examples, one or more processors may achieve synchronization by outputting signals that cause some operations to the sensors and / or the display. For example, in the case where one or more processors include a display driver integrated circuit (IC) 802 such as Figure 8 the one or more processors may output a synchronization signal (e.g., Figure 8 SSYNC of Figure 8 to one or more sensors (e.g., Figure 2 sensor module 806 of Figure 8 ), the synchronization signal causing the one or more sensors to emit electromagnetic radiation. The display driver IC 802 may be an example of

[0063] Figure 9A conceptual diagram showing various signals of a display of a device with synchronous operation of electromagnetic emission under the display according to one or more techniques of the present disclosure. As discussed above, one or more processors of the device may synchronize the operation of the display with the operation of sensors under the display by at least causing the sensors to emit electromagnetic radiation during a specific portion of the emission period of the display. According to one or more techniques of the present disclosure, in addition to or instead of causing the sensors to emit electromagnetic radiation during a specific portion of the emission period of the display, one or more processors may operate one or more sensors at a sensor operation frequency that is less than the display frame frequency. For example, one or more processors may cause one or more sensors to emit electromagnetic radiation during a first subset of frames of a plurality of frames; and inhibit causing one or more sensors to emit electromagnetic radiation during a second subset of frames of the plurality of frames. Thus, in some examples, one or more sensors may not emit electromagnetic radiation during the emission periods of consecutive frames. As a specific example, the sensor frequency may be half of the display frame frequency (e.g., when the display frame frequency is 60 Hz, the sensor frequency may be 30 Hz), such that the first subset of frames includes one of the even frames or odd frames, and the second subset of frames includes the other of the even frames or odd frames. Other fractions are possible, such as one-third, one-fourth, etc. In this way, one or more sensors may emit once every 'n' display frames instead of emitting electromagnetic radiation every frame, where 'n' can be 2, 3,..., 12, etc. frames. By operating one or more sensors at a sensor operation frequency that is less than the display frame frequency, one or more processors may reduce the visual appearance of any dots created due to sensor operation. For example, by using a sensor operation frequency that is half of the display frame frequency, one or more processors may halve the visual appearance of white dots (e.g., reduce the average white dot intensity by half).

[0064] Although reducing the sensor operation frequency relative to the display frame frequency may provide advantages, these advantages may be reduced when the sensor operation frequency is lowered too much. For example, if the sensor operation frequency is 1 Hz, the resulting bright spots may appear as flickers, which may be more distracting than brighter but static dots. Thus, according to one or more techniques of the present invention, one or more processors may inhibit using a sensor operation frequency that is lower than a threshold (e.g., 4 Hz).

[0065] Figures 10A to 10C A conceptual diagram showing device signals for synchronous operation of electromagnetic emission under a display according to one or more techniques of the present disclosure. Figures 10A to 10C Each of which shows different examples of how one or more processors may synchronize the operation of the display and sensors that emit light through the display. As Figures 10A to 10C shown in Figure 8The controller of the display driver IC 802 may output a synchronization signal including pulses that cause one or more sensors to emit electromagnetic radiation. When the controller outputs a synchronization pulse, the one or more sensors may not immediately emit electromagnetic radiation. Instead, in some examples, the one or more sensors may emit electromagnetic radiation after a transmitter delay time from the synchronization pulse. The controller may be programmed with this transmitter delay time (hereinafter referred to as T DP ), and may take this delay time into account when outputting the synchronization pulse.

[0066] As discussed above, one or more processors may synchronize the sensors with the display using a sensor operation frequency that is less than the display frame frequency. Figures 10A to 10C An example is shown where the sensor operation frequency is half of the display frame frequency, although other ratios are possible as discussed above. In Figure 10A the example, one or more processors may achieve a sensor operation frequency that is half of the display frame frequency by outputting synchronization pulses every other frame. In Figure 10B the example, one or more processors may achieve a sensor operation frequency that is half of the display frame frequency by outputting synchronization pulses every frame, and the sensor may count the synchronization pulses and emit once every X pulses (emit once every two pulses for a sensor operation frequency that is half of the display frame frequency). In Figure 10C the example, a second controller, such as a low-power microcontroller (uC), may be positioned between the first controller (e.g., DDIC 802) that outputs the synchronization pulse and the sensor. The second controller may receive a first synchronization pulse (SSYNC1) (e.g., at the display frame frequency) from the first controller, and output a second synchronization pulse (SSYNC2) to the sensor (e.g., at the sensor operation frequency).

[0067] Figure 11 is a conceptual diagram showing various signals of a display of a device having synchronous operation of electromagnetic emission under a display according to one or more techniques of the present disclosure. As discussed above, depending on when the sensor (e.g., UDS 120) emits electromagnetic radiation, the pixels above the sensor may appear as dark or bright spots. For example, emitting electromagnetic radiation during the emission period of a pixel may cause the pixel to appear as a bright spot. Alternatively, emitting electromagnetic radiation during the non-emission period of a pixel may cause the pixel to appear as a dark spot.

[0068] According to one or more techniques of the present disclosure, a controller of a device may synchronize the operation of a sensor with the operation of a display by causing the sensor to emit electromagnetic radiation alternately during a transmission period and a non-transmission period. By causing the sensor to emit electromagnetic radiation alternately between a transmission period and a non-transmission period of a pixel, the controller may cause alternating dark and bright spots (e.g., alternating increases and decreases in brightness). The alternating dark and bright spots may cancel each other out visually, thereby minimizing the visible impact of the sensor operation.

[0069] In some examples, to cause one or more sensors to emit electromagnetic radiation alternately during a transmission period and a non-transmission period, the controller may cause one or more sensors to emit electromagnetic radiation during a transmission period of one of a plurality of frames; and inhibit causing one or more sensors to emit electromagnetic radiation during a transmission period of subsequent frames of the plurality of frames until after causing one or more sensors to emit electromagnetic radiation during a non-transmission period. Thus, in some examples, the sensor may not emit electromagnetic radiation during two transmission periods without also emitting electromagnetic radiation during an intervening non-transmission period.

[0070] In some examples, to cause one or more sensors to emit electromagnetic radiation alternately during a transmission period and a non-transmission period, the controller may cause one or more sensors to emit electromagnetic radiation during a particular transmission period; and cause one or more sensors to emit electromagnetic radiation during a non-transmission period that is temporally adjacent to the particular transmission period. In some examples, the particular transmission period and the non-transmission period that is temporally adjacent to the particular transmission period may be in the same frame. For example, as Figure 12 shown, the particular transmission period may be transmission period 506B, and the non-transmission period that is temporally adjacent to the particular transmission period may be non-transmission period 504B). In some examples, the particular transmission period and the non-transmission period that is temporally adjacent to the particular transmission period may be in different frames. For example, as Figure 11 shown, the particular transmission period may be transmission period 506A, and the non-transmission period that is temporally adjacent to the particular transmission period may be non-transmission period 504C.

[0071] In some examples, to cause one or more sensors to emit electromagnetic radiation alternately during a transmission period and a non-transmission period, the controller may cause one or more sensors to emit electromagnetic radiation during a transmission period of the Nth frame of a plurality of frames; and cause one or more sensors to emit electromagnetic radiation during a non-transmission period of the (N + 1)th frame of the plurality of frames.

[0072] Figure 13Conceptual diagrams of various signals of a display of a device with synchronous operation of electromagnetic emission below the display in accordance with one or more techniques of the present disclosure are shown. As discussed above, in some examples, a controller may operate one or more sensors at a sensor operation frequency that is less than the display frame frequency. In some examples, the sensor operation frequency reduction technique may be combined with an alternating emission period / non-emission period operation. For example, as Figure 13 shown, the controller may cause the sensor to emit electromagnetic radiation during emission period 506A and then inhibit the sensor from emitting electromagnetic radiation until non-emission period 504D.

[0073] Additionally or alternatively, the alternating emission period / non-emission period operation technique may be combined with causing one or more sensors to emit electromagnetic radiation during a particular portion of the emission period of a frame. For example, as Figures 11 to 13 shown, when causing the sensor to emit electromagnetic radiation during the emission period, the controller may cause the sensor to emit electromagnetic radiation near the end of the emission period.

[0074] Additionally or alternatively, the alternating emission period / non-emission period operation technique may be combined with causing one or more sensors to emit electromagnetic radiation during a particular portion of the emission period of a frame and at a reduced sensor operation frequency. For example, as Figure 13 shown, while causing the sensor to emit electromagnetic radiation near the end of a selected emission period, the controller may perform all three alternating emissions of the sensor's electromagnetic radiation between the emission period and the non-emission period at a reduced sensor operation frequency.

[0075] Figure 14 is a flowchart of a method for synchronizing the operation of a display with the operation of a sensor below the display in accordance with one or more techniques of the present disclosure. Although described in the context of the Figure 1A and Figure 1B device 100, other devices may also perform the Figure 14 method.

[0076] Device 100 may program the pixels of the display (1402) during the non-emission period of a frame. For example, the controller 206 of display 200 may program the gate voltage of one or more pixels of display 200 (e.g., via data driver 210) during the non-emission period of non-emission period 504. As discussed above, the controller 200 may program the pixels of display 200 row by row. Also as discussed above, the controller 200 may cause the control signal EM to be in a first logic state (e.g., logic high) during the non-emission period.

[0077] Device 100 may cause pixels to emit light (1404) during a transmission period of a frame. For example, during the transmission period of transmission period 506, controller 206 may cause pixels to emit a programmed amount of light based on a particular frame (e.g., the amount of light emitted by a particular pixel may be a function of the gate voltage of the driver of the particular pixel). Also as discussed above, controller 200 may cause control signal EM to be in a second logic state (e.g., logic low) during non-transmission periods.

[0078] According to one or more techniques of the present disclosure, device 100 may synchronize the operation of one or more sensors with the operation of pixels. For example, device 100 may cause sensor 120 to alternately emit electromagnetic radiation through pixels of display 200 during the transmission period of transmission period 506 and the non-transmission period of non-transmission period 504 (1406). By alternately emitting through display 200 between the transmission period and the non-transmission period by sensor 120, device 100 may cause alternating black and white dots (e.g., alternating increases and decreases in brightness). The alternating black and white dots may visually cancel each other out, thereby minimizing the visible impact of sensor operation.

[0079] Figure 15 is a flowchart showing a method for synchronizing the operation of a display with the operation of a sensor under the display according to one or more techniques of the present disclosure. Although described in the context of device 100 of Figure 1A and Figure 1B other devices may also perform Figure 15 the method.

[0080] Device 100 may program pixels of a display during a non-transmission period of a frame (1502). For example, controller 206 of display 200 may program the gate voltage of one or more pixels of display 200 during the non-transmission period of non-transmission period 504 (e.g., via data driver 210). As discussed above, controller 200 may program the pixels of display 200 row by row. Also as discussed above, controller 200 may cause control signal EM to be in a first logic state (e.g., logic high) during non-transmission periods.

[0081] Device 100 may cause pixels to emit light during a transmission period of a frame (1504). For example, during the transmission period of transmission period 506, controller 206 may cause pixels to emit a programmed amount of light based on a particular frame (e.g., the amount of light emitted by a particular pixel may be a function of the gate voltage of the driver of the particular pixel). Also as discussed above, controller 200 may cause control signal EM to be in a second logic state (e.g., logic low) during non-transmission periods.

[0082] In accordance with one or more techniques of the present disclosure, device 100 may synchronize the operation of one or more sensors with the operation of pixels. For example, device 100 may cause sensor 120 to emit electromagnetic radiation through pixels (1506) of display 200 during a particular portion of a transmission period. For example, as Figures 11 to 13 shown, to cause sensor 120 to emit electromagnetic radiation during a particular portion of a transmission period, controller 206 may cause sensor 120 to emit electromagnetic radiation near the end of the transmission period. Since the effect of the luminance value change emitted by sensor 120 during the transmission period may be reset or cancelled during a subsequent non-transmission period, synchronizing the operation of sensor 120 and display 200 in this manner may minimize the visible effect of the luminance value change.

[0083] The following numbered examples may illustrate one or more aspects of the present disclosure:

[0084] Example 1. A computing device, comprising: a display including a plurality of pixels; one or more sensors positioned under the display and configured to emit electromagnetic radiation through the display during operation; and one or more processors configured to: program pixels of the plurality of pixels based on image data of a frame during a non-transmission period of the frame among a plurality of frames; cause the pixels of the plurality of pixels to emit light during a transmission period of the frame, wherein an amount of light emitted by the pixels during the transmission period of a particular frame is based on the programming of the particular frame; and synchronize the operation of the one or more sensors with the operation of the plurality of pixels by at least causing the one or more sensors to alternately emit the electromagnetic radiation during the transmission period and the non-transmission period.

[0085] Example 2. The computing device according to Example 1, wherein, to cause the one or more sensors to alternately emit the electromagnetic radiation during the transmission period and the non-transmission period, the one or more processors are configured to: cause the one or more sensors to emit the electromagnetic radiation during the transmission period of a frame among the plurality of frames; and inhibit causing the one or more sensors to emit the electromagnetic radiation during the transmission period of a subsequent frame of the plurality of frames until after causing the one or more sensors to emit the electromagnetic radiation during the non-transmission period.

[0086] Example 3. The computing device according to Example 1, wherein, in order to cause the one or more sensors to alternately emit the electromagnetic radiation during a transmission period and a non-transmission period, the one or more processors are configured to: cause the one or more sensors to emit the electromagnetic radiation during a specific transmission period; and cause the one or more sensors to emit the electromagnetic radiation during a non-transmission period that is temporally adjacent to the specific transmission period.

[0087] Example 4. The computing device according to Example 3, wherein the specific transmission period and the non-transmission period that is temporally adjacent to the specific transmission period are in the same frame.

[0088] Example 5. The computing device according to Example 3, wherein the specific transmission period and the non-transmission period that is temporally adjacent to the specific transmission period are in different frames.

[0089] Example 6. The computing device according to Example 1, wherein, in order to cause the one or more sensors to alternately emit the electromagnetic radiation during a transmission period and a non-transmission period, the one or more processors are configured to: cause the one or more sensors to emit the electromagnetic radiation during a transmission period of the Nth frame among the plurality of frames; and cause the one or more sensors to emit the electromagnetic radiation during a non-transmission period of the (N + 1)th frame among the plurality of frames.

[0090] Example 7. The computing device according to Example 1, wherein, in order to cause the one or more sensors to emit the electromagnetic radiation during a transmission period of the transmission period, the one or more processors are configured to cause the one or more sensors to emit the electromagnetic radiation during a specific part of the transmission period.

[0091] Example 8. The computing device according to Example 7, wherein, in order to cause the one or more sensors to emit the electromagnetic radiation during the specific part of the transmission period, the one or more processors are configured to cause the one or more sensors to emit the electromagnetic radiation near the end of the transmission period.

[0092] Example 9. The computing device according to Example 8, wherein, in order to cause the one or more sensors to emit the electromagnetic radiation near the end of the transmission period, the one or more processors are configured to perform one or both of the following: cause the one or more sensors to emit the electromagnetic radiation during a final sub-part of the transmission period; and inhibit the one or more sensors from emitting the electromagnetic radiation during a part of the transmission period other than the final sub-period of the transmission period.

[0093] Example 10. The computing device according to Example 9, wherein the final sub - portion of the emission period is the last 20% of the emission period.

[0094] Example 11. The computing device according to Example 7, wherein, in order to cause the one or more sensors to emit the electromagnetic radiation during the specific portion of the emission period, the one or more processors are configured to perform one or both of the following: cause the one or more sensors to emit the electromagnetic radiation after a predetermined delay period; and inhibit the one or more sensors from emitting the electromagnetic radiation before the expiration of the predetermined delay period.

[0095] Example 12. The computing device according to Example 11, wherein the predetermined delay period is an amount of time from a specific point in the frame.

[0096] Example 13. The computing device according to Example 7, wherein, in order to synchronize the operation of the one or more sensors with the operation of the plurality of pixels, the one or more processors are configured to operate the one or more sensors at a sensor operation frequency that is less than the display frame frequency.

[0097] Example 14. The computing device according to Example 13, wherein, in order to operate the one or more sensors at the sensor operation frequency, the one or more processors are configured to perform one or both of the following: cause the one or more sensors to emit the electromagnetic radiation during a first frame subset of the plurality of frames; and inhibit the one or more sensors from emitting the electromagnetic radiation during a second frame subset of the plurality of frames.

[0098] Example 15. The computing device according to Example 13, wherein the sensor operation frequency is an integer fraction of the display frame frequency.

[0099] Example 16. The computing device according to Example 15, wherein the sensor frequency is half of the display frame frequency, and wherein the first frame subset includes one of the even frames or odd frames, and the second frame subset includes the other of the even frames or odd frames.

[0100] Example 17. The computing device according to Example 1, wherein, in order to program a specific pixel among the plurality of pixels, the one or more processors are configured to cause a circuit to store a voltage level representing the emission intensity of the specific pixel, and wherein the emission of the electromagnetic radiation by the one or more sensors modifies the stored voltage level.

[0101] Example 18. The computing device according to Example 1, wherein the electromagnetic radiation includes one or more of infrared radiation, ultraviolet radiation, or radio wave radiation.

[0102] Example 19. The computing device according to Example 1, wherein the display includes an organic light emitting diode display (OLED).

[0103] Example 20. A method includes: programming a pixel among a plurality of pixels of a display of a computing device based on image data of a frame during a non-emission period of the frame among a plurality of frames; causing the pixel among the plurality of pixels to emit light during an emission period of the frame, wherein an amount of light emitted by the pixel during the emission period of a specific frame is based on the programming of the specific frame; and synchronizing operations of the one or more sensors and operations of the plurality of pixels by causing the one or more sensors to alternately emit electromagnetic radiation through the display during the emission period and the non-emission period.

[0104] Example 21. The method according to Example 20, wherein causing the one or more sensors to alternately emit the electromagnetic radiation during the emission period and the non-emission period includes: causing the one or more sensors to emit the electromagnetic radiation during the emission period of a frame among the plurality of frames; and suppressing causing the one or more sensors to emit the electromagnetic radiation during the emission period of a subsequent frame among the plurality of frames until after causing the one or more sensors to emit the electromagnetic radiation during the non-emission period.

[0105] Example 22. The method according to Example 20, wherein causing the one or more sensors to alternately emit the electromagnetic radiation during the emission period and the non-emission period includes: causing the one or more sensors to emit the electromagnetic radiation during a specific emission period; and causing the one or more sensors to emit the electromagnetic radiation during a non-emission period that is temporally adjacent to the specific emission period.

[0106] Example 23. The method according to Example 22, wherein the specific emission period and the non-emission period that is temporally adjacent to the specific emission period are in the same frame.

[0107] Example 24. The method according to Example 23, wherein the specific emission period and the non-emission period that is temporally adjacent to the specific emission period are in different frames.

[0108] Example 25. The method according to Example 20, wherein causing the one or more sensors to alternately emit the electromagnetic radiation during the emission period and the non-emission period includes: causing the one or more sensors to emit the electromagnetic radiation during the emission period of the Nth frame among the plurality of frames; and causing the one or more sensors to emit the electromagnetic radiation during the non-emission period of the (N + 1)th frame among the plurality of frames.

[0109] Example 26. The method according to Example 20, wherein causing the one or more sensors to emit the electromagnetic radiation during the emission period of the emission period comprises causing the one or more sensors to emit the electromagnetic radiation during a specific portion of the emission period.

[0110] Example 27. The method according to Example 26, wherein causing the one or more sensors to emit the electromagnetic radiation during the specific portion of the emission period comprises causing the one or more sensors to emit the electromagnetic radiation near the end of the emission period.

[0111] Example 28. The method according to Example 27, wherein causing the one or more sensors to emit the electromagnetic radiation near the end of the emission period comprises one or both of the following: causing the one or more sensors to emit the electromagnetic radiation during a final sub-portion of the emission period; and suppressing causing the one or more sensors to emit the electromagnetic radiation during portions of the emission period other than the final sub-period of the emission period.

[0112] Example 29. The method according to Example 28, wherein the final sub-portion of the emission period is the last 20% of the emission period.

[0113] Example 30. The method according to Example 26, wherein causing the one or more sensors to emit the electromagnetic radiation during the specific portion of the emission period comprises one or both of the following: causing the one or more sensors to emit the electromagnetic radiation after a predetermined delay period; and suppressing causing the one or more sensors to emit the electromagnetic radiation before the expiration of the predetermined delay period.

[0114] Example 31. The method according to Example 30, wherein the predetermined delay period is an amount of time from a specific point in the frame.

[0115] Example 32. The method according to Example 26, wherein synchronizing the operation of the one or more sensors with the operation of the plurality of pixels comprises operating the one or more sensors at a sensor operation frequency that is less than the display frame frequency.

[0116] Example 33. The method according to Example 32, wherein operating the one or more sensors at the sensor operation frequency comprises one or both of the following: causing the one or more sensors to emit the electromagnetic radiation during a first frame subset of the plurality of frames; and suppressing causing the one or more sensors to emit the electromagnetic radiation during a second frame subset of the plurality of frames.

[0117] Example 34. The method according to Example 32, wherein the sensor operating frequency is an integer fraction of the display frame frequency.

[0118] Example 35. The method according to Example 34, wherein the sensor frequency is half of the display frame frequency, and wherein the first frame subset includes one of the even frames or the odd frames, and the second frame subset includes the other of the even frames or the odd frames.

[0119] Example 36. The method according to Example 20, wherein programming a particular pixel among the plurality of pixels includes causing a circuit to store a voltage level representing the emission intensity of the particular pixel, and wherein the emission of electromagnetic radiation by the one or more sensors modifies the stored voltage level.

[0120] Example 37. The method according to Example 20, wherein the electromagnetic radiation includes one or more of infrared radiation, ultraviolet radiation, or radio wave radiation.

[0121] Example 38. The method according to Example 20, wherein the display includes an organic light emitting diode display (OLED).

[0122] Example 39. A computer-readable storage medium storing instructions that, when executed, cause one or more processors of a computing device to perform the method according to any one of Examples 20 to 38.

[0123] Example 40. An apparatus, comprising: a display including a plurality of pixels; one or more sensors positioned below the display and configured to emit electromagnetic radiation through the display during operation; and means for performing the method according to any one of Examples 20 to 38.

[0124] Embodiments of the subject matter and the functional operations described in this specification can be implemented in any suitable electronic device such as a personal computer, a mobile phone, a smart phone, a smart watch, a smart TV, a mobile audio or video player, a game console, or a combination of one or more of these devices.

[0125] The computing device can include various components such as a memory, a processor, a display, and an input / output unit. The input / output unit can include, for example, a transceiver that is capable of communicating with one or more networks to send and receive data. The display can be any suitable display, including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), or a light emitting diode (LED) display for displaying images.

[0126] The various implementations of the systems and techniques described herein can be implemented in digital electronic circuitry, integrated circuitry, specially designed ASICs (Application Specific Integrated Circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementations in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special purpose or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0127] One or more aspects of the techniques can be implemented as one or more computer program products, e.g., one or more modules of computer program instructions encoded on a computer-readable medium for execution by, or to control the operation of, a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The term “data processing apparatus” includes all apparatus, devices, and machines for processing data, e.g., including a programmable processor, a computer, or multiple processors or computers. The apparatus can also include code that creates an execution environment for the computer program being discussed, e.g., code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to an appropriate receiver apparatus.

[0128] A computer program (also called a program, software, software application, script, or code) can be written in any form of programming language, including a compiled or interpreted language, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program being discussed, or in multiple collaborating files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program can be deployed to execute on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[0129] For example, processors suitable for the execution of a computer program include both general and special purpose microprocessors, and any one or more processors of any type of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both.

[0130] The components of a computer can include a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data or operatively coupled to receive data therefrom or transfer data thereto or both. However, a computer may not have these devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0131] Although this specification contains many specific implementation details, these should not be construed as limitations on the scope of what is claimed, but rather as descriptions of features specific to particular embodiments. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination within a single embodiment. Conversely, various features that are described in a single embodiment can also be implemented separately in multiple embodiments or in any suitable sub-combination. Moreover, although the above features may be described as acting in certain combinations and even initially claimed as such, in some cases, one or more features from a claimed combination can be excluded from the combination, and the claimed combination can be directed to a sub-combination or a variation of a sub-combination.

[0132] Similarly, although operations are depicted in the drawings in a particular order, this should not be understood as requiring that the operations be performed in the particular order shown or in a sequential order, or that all of the illustrated operations be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Additionally, the separation of various system modules and components in the above embodiments should not be understood as required in all embodiments, and it should be understood that the described program components and systems can generally be integrated in a single software product or packaged into multiple software products.

[0133] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the acts recited in the claims can be performed in a different order and still achieve the desired result. As one example, the processes depicted in the figures do not necessarily need the particular order or sequential order shown to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. A computing device, comprising: A display, the display including a plurality of pixels; One or more sensors, the one or more sensors being positioned below the display and configured to emit electromagnetic radiation through the display during operation; And One or more processors, the one or more processors being configured to: Program pixels among the plurality of pixels based on image data of a frame during a non-emission period of the frame in a plurality of frames; Cause the pixels among the plurality of pixels to emit light during an emission period of the frame, wherein an amount of light emitted by the pixels during the emission period of a specific frame is based on the programming performed on the specific frame; and Synchronize the operation of the one or more sensors with the operation of the plurality of pixels by at least causing the one or more sensors to alternately emit the electromagnetic radiation during the emission period and the non-emission period, Wherein, in order to cause the one or more sensors to alternately emit the electromagnetic radiation during the emission period and the non-emission period, the one or more processors are configured to: Cause the one or more sensors to emit the electromagnetic radiation during the emission period of a frame among the plurality of frames; and Suppress causing the one or more sensors to emit the electromagnetic radiation during the emission period of a subsequent frame of the plurality of frames until after causing the one or more sensors to emit the electromagnetic radiation during the non-emission period.

2. A computing device, comprising: A display, the display including a plurality of pixels; One or more sensors, the one or more sensors being positioned below the display and configured to emit electromagnetic radiation through the display during operation; And One or more processors, the one or more processors being configured to: Program pixels among the plurality of pixels based on image data of a frame during a non-emission period of the frame in a plurality of frames; Cause the pixels among the plurality of pixels to emit light during an emission period of the frame, wherein an amount of light emitted by the pixels during the emission period of a specific frame is based on the programming performed on the specific frame; and Synchronize the operation of the one or more sensors with the operation of the plurality of pixels by at least causing the one or more sensors to alternately emit the electromagnetic radiation during the emission period and the non-emission period, Wherein, in order to cause the one or more sensors to alternately emit the electromagnetic radiation during the emission period and the non-emission period, the one or more processors are configured to: Cause the one or more sensors to emit the electromagnetic radiation during a specific emission period; and Cause the one or more sensors to emit the electromagnetic radiation during a non-emission period that is temporally adjacent to the specific emission period.

3. A computing device, comprising: A display, the display including a plurality of pixels; One or more sensors, the one or more sensors being positioned below the display and configured to emit electromagnetic radiation through the display during operation; And One or more processors, the one or more processors being configured to: Programming a pixel among the plurality of pixels based on image data of a frame during a non-emission period of the frame among a plurality of frames; Causing the pixels among the plurality of pixels to emit light during an emission period of the frame, wherein an amount of light emitted by the pixel during the emission period of a particular frame is based on the programming performed on the particular frame; and Synchronizing operations of the one or more sensors with operations of the plurality of pixels by causing the one or more sensors to alternately emit the electromagnetic radiation during the emission period and the non-emission period, wherein, to cause the one or more sensors to alternately emit the electromagnetic radiation during the emission period and the non-emission period, the one or more processors are configured to: Cause the one or more sensors to emit the electromagnetic radiation during an emission period of an Nth frame among the plurality of frames; and Cause the one or more sensors to emit the electromagnetic radiation during a non-emission period of an N+1th frame among the plurality of frames.

4. The computing device according to claim 2, wherein, The particular emission period and the non-emission period temporally adjacent to the particular emission period are in the same frame.

5. The computing device according to claim 2, wherein, The particular emission period and the non-emission period temporally adjacent to the particular emission period are in different frames.

6. The computing device according to any one of claims 1 to 3, wherein To cause the one or more sensors to emit the electromagnetic radiation during an emission period among the emission periods, the one or more processors are configured to cause the one or more sensors to emit the electromagnetic radiation during a particular portion of the emission period.

7. The computing device according to claim 6, wherein, To cause the one or more sensors to emit the electromagnetic radiation during the particular portion of the emission period, the one or more processors are configured to cause the one or more sensors to emit the electromagnetic radiation near an end of the emission period.

8. The computing device according to claim 7, wherein, To cause the one or more sensors to emit the electromagnetic radiation near the end of the emission period, the one or more processors are configured to perform one or both of the following: Cause the one or more sensors to emit the electromagnetic radiation during a final sub-portion of the emission period; and Suppress causing the one or more sensors to emit the electromagnetic radiation during portions of the emission period other than the final sub-period of the emission period.

9. The computing device according to claim 8, wherein, The final sub-portion of the emission period is the last 20% of the emission period.

10. The computing device according to claim 6, wherein, To cause the one or more sensors to emit the electromagnetic radiation during the particular portion of the emission period, the one or more processors are configured to perform one or both of the following: Cause the one or more sensors to emit the electromagnetic radiation after a predetermined delay period; and Suppress causing the one or more sensors to emit the electromagnetic radiation before the expiration of the predetermined delay period.

11. The computing device according to claim 10, wherein, The predetermined delay period is an amount of time from a particular point in the frame.

12. The computing device according to claim 6, wherein, To synchronize operations of the one or more sensors with operations of the plurality of pixels, the one or more processors are configured to operate the one or more sensors at a sensor operation frequency lower than a display frame frequency.

13. The computing device according to claim 12, wherein, To operate the one or more sensors at the sensor operating frequency, the one or more processors are configured to perform one or both of the following: Cause the one or more sensors to emit the electromagnetic radiation during a first subset of frames of the plurality of frames; and Suppress causing the one or more sensors to emit the electromagnetic radiation during a second subset of frames of the plurality of frames.

14. The computing device according to claim 13, wherein, The sensor operating frequency is an integer fraction of the display frame frequency.

15. The computing device according to claim 14, wherein The sensor operating frequency is half of the display frame frequency, and wherein the first subset of frames includes one of the even frames or the odd frames, and the second subset of frames includes the other of the even frames or the odd frames.

16. The computing device according to any one of claims 1 to 3, wherein, To program a particular pixel of the plurality of pixels, the one or more processors are configured to cause a circuit to store a voltage level representing the emission intensity of the particular pixel, and wherein the emission of the electromagnetic radiation by the one or more sensors modifies the stored voltage level.

17. The computing device according to any one of claims 1 to 3, wherein, The electromagnetic radiation includes one or more of infrared radiation, ultraviolet radiation, and radio wave radiation.

18. The computing device according to any one of claims 1-17, wherein, The display includes an organic light emitting diode display (OLED).

19. A method, comprising: Programming a pixel of a plurality of pixels of a display of a computing device based on image data of a frame during a non-emission period of the frame among a plurality of frames; Causing the pixels of the plurality of pixels to emit light during an emission period of the frame, wherein the amount of light emitted by the pixels during the emission period of a particular frame is based on the programming of the particular frame; and Synchronizing the operation of the one or more sensors and the operation of the plurality of pixels by causing at least one or more sensors to alternately emit electromagnetic radiation through the display during the emission period and the non-emission period, wherein causing the one or more sensors to alternately emit the electromagnetic radiation during the emission period and the non-emission period includes: Causing the one or more sensors to emit the electromagnetic radiation during the emission period of a frame among the plurality of frames; and Suppressing causing the one or more sensors to emit the electromagnetic radiation during the emission period of a subsequent frame of the plurality of frames until after causing the one or more sensors to emit the electromagnetic radiation during the non-emission period.

20. A method, comprising: Programming a pixel of a plurality of pixels of a display of a computing device based on image data of a frame during a non-emission period of the frame among a plurality of frames; Causing the pixels of the plurality of pixels to emit light during an emission period of the frame, wherein the amount of light emitted by the pixels during the emission period of a particular frame is based on the programming of the particular frame; and Synchronizing the operation of the one or more sensors and the operation of the plurality of pixels by causing at least one or more sensors to alternately emit electromagnetic radiation through the display during the emission period and the non-emission period, wherein causing the one or more sensors to alternately emit the electromagnetic radiation during the emission period and the non-emission period includes: Causing the one or more sensors to emit the electromagnetic radiation during a particular emission period; and Cause the one or more sensors to emit the electromagnetic radiation during a non-transmission period that is temporally adjacent to the specific transmission period.

21. A method includes: Programming pixels among a plurality of pixels of a display of a computing device based on image data of a frame during a non-transmission period of the frame; Causing the pixels among the plurality of pixels to emit light during a transmission period of the frame, wherein an amount of light emitted by the pixels during the transmission period of a specific frame is based on the programming of the specific frame; And Synchronizing operations of the one or more sensors and operations of the plurality of pixels by causing at least the one or more sensors to alternately emit electromagnetic radiation through the display during the transmission period and the non-transmission period, wherein causing the one or more sensors to alternately emit the electromagnetic radiation during the transmission period and the non-transmission period includes: Causing the one or more sensors to emit the electromagnetic radiation during a transmission period of an Nth frame among the plurality of frames; and Causing the one or more sensors to emit the electromagnetic radiation during a non-transmission period of an (N + 1)th frame among the plurality of frames.

22. The method according to claim 20, wherein The specific transmission period and the non-transmission period that is temporally adjacent to the specific transmission period are in the same frame.

23. The method according to claim 22, wherein, The specific transmission period and the non-transmission period that is temporally adjacent to the specific transmission period are in different frames.

24. The method according to any one of claims 19 to 21, wherein Causing the one or more sensors to emit the electromagnetic radiation during a transmission period within the transmission period includes causing the one or more sensors to emit the electromagnetic radiation during a specific portion of the transmission period.

25. The method according to claim 24, wherein, Causing the one or more sensors to emit the electromagnetic radiation during the specific portion of the transmission period includes causing the one or more sensors to emit the electromagnetic radiation near an end of the transmission period.

26. The method according to claim 25, wherein, Causing the one or more sensors to emit the electromagnetic radiation near an end of the transmission period includes one or both of the following: Causing the one or more sensors to emit the electromagnetic radiation during a final sub-portion of the transmission period; And Suppressing causing the one or more sensors to emit the electromagnetic radiation during portions of the transmission period other than the final sub-period of the transmission period.

27. The method according to claim 26, wherein, The final sub-portion of the transmission period is the last 20% of the transmission period.

28. The method according to claim 24, wherein Causing the one or more sensors to emit the electromagnetic radiation during the specific portion of the transmission period includes one or both of the following: Causing the one or more sensors to emit the electromagnetic radiation after a predetermined delay period; And Suppressing causing the one or more sensors to emit the electromagnetic radiation before the expiration of the predetermined delay period.

29. The method according to claim 28, wherein The predetermined delay period is an amount of time from a specific point in the frame.

30. The method according to claim 24, wherein, Synchronizing operations of the one or more sensors with operations of the plurality of pixels includes operating the one or more sensors at a sensor operating frequency that is less than a display frame frequency.

31. The method according to claim 30, wherein, Operating the one or more sensors at the sensor operating frequency includes one or both of the following: Causing the one or more sensors to emit the electromagnetic radiation during a first frame subset of the plurality of frames; And Suppress the emission of the electromagnetic radiation by the one or more sensors during a second subset of the plurality of frames.

32. The method according to claim 31, wherein, The sensor operating frequency is an integer fraction of the display frame frequency.

33. The method according to claim 32, wherein, The sensor operating frequency is half of the display frame frequency, and wherein the first subset of frames includes one of the even frames or the odd frames, and the second subset of frames includes the other of the even frames or the odd frames.

34. The method according to any one of claims 19 to 21, wherein Programming a particular pixel among the plurality of pixels includes causing a circuit to store a voltage level representative of the emission intensity of the particular pixel, and wherein the emission of the electromagnetic radiation by the one or more sensors modifies the stored voltage level.

35. The method according to any one of claims 19 to 21, wherein The electromagnetic radiation includes one or more of infrared radiation, ultraviolet radiation, or radio wave radiation.

36. The method according to any one of claims 19 - 35, wherein, The display includes an organic light emitting diode display (OLED).

37. A computer-readable storage medium storing instructions that, when executed, cause one or more processors of a computing device to perform the method according to any one of claims 19 to 36.

38. An apparatus, comprising: A display including a plurality of pixels; One or more sensors positioned below the display and configured to emit electromagnetic radiation through the display during operation; And Means for performing the method according to any one of claims 19 to 36.

Images