Display compatible ambient light sensor

Abstract

A user device includes a display device, a light sensor, and a timing controller configured to provide one or more timing signals to (1) enable the light sensor to sense ambient light during a first portion of a repeating timing pattern, and to (2) enable the light sensor to sense ambient light during a second portion of the repeating timing pattern. And (2) enabling the one or more light emitters to emit display light during a second portion of the repeating timing pattern, where the first portion is substantially non-overlapping with the second portion of the repeating timing pattern, where the one or more light emitters emit display light during the second portion of the repeating timing pattern. The timing controller is configured to adjust the duty cycle for sensing ambient light within the repeating timing pattern and the duty cycle for emitting display light within the repeating timing pattern in response to ambient light intensity measurements based on one or more readings of the light sensor.

Description

Technical Field

[0001] Aspects of the present disclosure generally relate to ambient light sensing. More specifically, the aspects discussed below relate to ambient light sensing in conjunction with display device operation. Background Art

[0002] Ambient light sensors (ALS) are widely used in user devices such as smartphones, laptops, wearable devices, automotive displays, television displays, etc. Ambient light sensors provide a measurement of ambient light intensity indicative of the environment of the user device. Ambient light sensor readings have many applications, such as dynamic brightness adjustment of displays, location and / or context determination (e.g., indoors vs. outdoors), and other uses. Typically, the ALS is implemented in conjunction with a display device, such as an organic light emitting diode (OLED) display, a light emitting diode (LED) display, a liquid crystal display (LCD) display, etc. The readings from the ALS can be used to dynamically adjust the brightness of an OLED display or LED display, or the backlight of an LCD display. Given the usefulness of ALS readings in conjunction with display device operation, there is a particular need to improve the accuracy and usability of light intensity readings obtained from an ALS device in the presence of a display device. Summary of the invention

[0003] The following briefly introduces one or more aspects of the present disclosure. This summary is not an extensive description of all features and is not intended to identify key or important elements of all aspects of the present disclosure. Instead, this summary is intended to provide some ideas of one or more related concepts as a prelude to a more detailed description presented in subsequent sections.

[0004] An example user device includes a display device comprising one or more light emitters forming at least a portion of the display device; a light sensor positioned such that an optical path exists between the light sensor and the one or more light emitters; and a timing controller coupled to the light sensor and the display device, the timing controller configured to provide one or more timing signals to (1) enable the light sensor to sense ambient light during a first portion of a repeating timing pattern, and (2) enable the one or more light emitters to emit display light during a second portion of the repeating timing pattern, wherein the first portion does not substantially overlap with the second portion of the repeating timing pattern, wherein The first portion of the repeating timing pattern corresponds to a duty cycle for sensing ambient light within the repeating timing pattern, and the second portion of the repeating timing pattern corresponds to a duty cycle for emitting display light within the repeating timing pattern, wherein the timing controller is configured to adjust the duty cycle for sensing ambient light within the repeating timing pattern and the duty cycle for emitting display light within the repeating timing pattern in response to an ambient light intensity measurement based on one or more readings of the light sensor, and wherein the timing controller is configured to continue to provide the one or more timing signals after adjusting the duty cycle for sensing ambient light within the repeating timing pattern.

[0005] According to the present disclosure, an example method for sensing ambient light intensity in a user device including a display device and a light sensor, the light sensor being positioned so that an optical path exists between the light sensor and one or more light emitters forming at least a portion of the display device, according to the present disclosure, the method may include: providing one or more timing signals to (1) enable the light sensor to sense ambient light during a first portion of a repeating timing pattern, and (2) enable one or more light emitters to emit display light during a second portion of the repeating timing pattern, wherein the first portion does not substantially overlap with the second portion of the repeating timing pattern, and wherein the first portion of the repeating timing pattern corresponds to a duty cycle for sensing ambient light within the repeating timing pattern, and the second portion of the repeating timing pattern corresponds to a duty cycle for emitting display light within the repeating timing pattern. The method may also include generating an ambient light intensity measurement based on one or more readings of the light sensor. The method may also include adjusting the duty cycle for sensing ambient light within the repeating timing pattern and the duty cycle for emitting display light within the repeating timing pattern in response to the ambient light intensity measurement. The method may also include continuing to provide the one or more timing signals after adjusting the duty cycle for sensing ambient light within the repeating timing pattern.

[0006] According to the present disclosure, an exemplary non-transitory computer-readable medium has instructions embedded thereon, which, when executed by one or more processors, cause the one or more processors to perform functions including: providing one or more timing signals to (1) enable the light sensor to sense ambient light during a first portion of a repeating timing pattern, and (2) enable one or more light emitters to emit display light during a second portion of the repeating timing pattern, wherein the light sensor is positioned such that an optical path exists between the light sensor and the one or more light emitters, and wherein the first portion of the repeating timing pattern does not substantially overlap with the second portion of the repeating timing pattern, and wherein the first portion of the repeating timing pattern corresponds to a duty cycle for sensing ambient light within the repeating timing pattern, and the second portion of the repeating timing pattern corresponds to a duty cycle for emitting display light within the repeating timing pattern. The instructions may also include code for generating an ambient light intensity measurement based on one or more readings of the light sensor. The instructions may also include code for adjusting the duty cycle for sensing ambient light within the repeating timing pattern in response to the ambient light intensity measurement. The instructions may also include code for continuing to provide the one or more timing signals after adjusting the duty cycle for sensing ambient light within the repeating timing pattern.

[0007] An example computerized device includes: means for providing one or more timing signals to (1) enable a light sensor to sense ambient light during a first portion of a repeating timing pattern, and (2) enable one or more light emitters to emit display light during a second portion of the repeating timing pattern, wherein the light sensor is positioned such that an optical path exists between the light sensor and the one or more light emitters, and wherein the first portion does not substantially overlap with the second portion of the repeating timing pattern, and wherein the first portion of the repeating timing pattern corresponds to a duty cycle for sensing ambient light within the repeating timing pattern, and the second portion of the repeating timing pattern corresponds to a duty cycle for emitting display light within the repeating timing pattern, means for generating ambient light intensity measurements based on one or more readings of the light sensor, means for adjusting the duty cycle for sensing ambient light within the repeating timing pattern in response to the ambient light intensity measurements, and means for continuing to provide the one or more timing signals after adjusting the duty cycle for sensing ambient light within the repeating timing pattern. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Various aspects of the present disclosure are described by way of example. In the accompanying drawings, like reference numerals indicate like elements.

[0009] Figure 1An efficient method for improving the accuracy of light intensity readings generated by an ALS device used in conjunction with a display device is shown according to one aspect of the present disclosure.

[0010] Figure 2 An exploded view of some illustrative components of a user device that positions an ALS sensor below a display panel according to one aspect of the present disclosure is presented.

[0011] Figure 3 is a block diagram illustrating one or more time division duplex (TDD) timing signals provided by a semiconductor device separate from a timing controller for a light sensor, a timing controller for a display device, and a processor coupled to the light sensor and the display device according to one aspect of the present disclosure.

[0012] Figure 4 is a block diagram illustrating one or more time division duplex (TDD) timing signals provided by a processor coupled to a light sensor and a display device according to one aspect of the present disclosure.

[0013] Figure 5 is a block diagram illustrating one or more time division duplex (TDD) timing signals provided by a combined timing controller for a light sensor and display device according to one aspect of the present disclosure.

[0014] Figure 6 is a block diagram showing further details of a display driver integrated circuit (DDIC) according to some aspects of the present disclosure.

[0015] Figure 7 is a block diagram showing further details of an ambient light sensor (ALS) device according to some aspects of the present disclosure.

[0016] Figure 8 is a block diagram showing further details of a combined DDIC and ALS controller according to some aspects of the present disclosure.

[0017] Fig. 9 is a timing diagram illustrating various signals based on a time division duplex (TDD) structure for supporting periodic, substantially non-overlapping durations for sensing ambient light and emitting display light according to some aspects of the present disclosure.

[0018] Fig.10 A display and ALS duty cycle that are dynamically adjusted based on current ALS values ​​are shown in accordance with some aspects of the present disclosure.

[0019] Fig.11 It is shown that blanking periods and overlapping periods are used in enable signals provided for display operations and ALS operations according to some aspects of the present disclosure.

[0020] Fig.12 A three-dimensional (3D) lookup table (LUT) that maps input red, green, blue (RBG) color values ​​to output RGB color values.

[0021] Fig.13A Examples of different display duty cycles and their corresponding color calibration parameters are shown.

[0022] Fig. 13B Examples of different display duty cycle ranges and their corresponding color calibration parameters are shown.

[0023] Fig.14A Examples of different display duty cycles and their corresponding brightness level adjustment ratios are shown.

[0024] Fig. 14B Examples of different display duty cycle ranges and their corresponding brightness adjustment ratios are shown.

[0025] Fig.15 is a flow chart illustrating an example of steps performed in a technique for sensing ambient light intensity while presenting display content according to one aspect of the present disclosure.

[0026] Fig.16 is a block diagram of various hardware and software components of a device employing a display and an ALS sensor capable of coordinating emission of display content and capturing ambient light for ALS measurements according to one aspect of the present disclosure. DETAILED DESCRIPTION

[0027] Several illustrative aspects will now be described in conjunction with the accompanying drawings which form a part hereof.While specific aspects are described below in which one or more aspects of the disclosure may be implemented, other aspects may be used and various modifications may be made without departing from the scope of the disclosure or the spirit of the appended claims.

[0028] As previously mentioned, ALS readings are typically obtained in conjunction with the operation of a display device. In many embodiments, the ALS device may be located near the display device. As an example of such proximity, the ALS device may be positioned so that an optical path exists between the ALS device and one or more light emitters that form at least a portion of the display device. The optical path may include a direct optical path, a reflected optical path, or a combination thereof. The optical path may pass through different materials, cross different boundaries, and the like. The presence of such an optical path allows the display light to reach the ALS device. A significant disadvantage of such proximity is that light emitted from the display device may be inadvertently captured by the ALS device, thereby contaminating the ALS readings. Specifically, in addition to photons of light from the environment, the ALS device may also capture additional photons emitted from the display device. Capturing these additional photons attributed to the display light results in a positive deviation in the ALS reading, indicating that the light intensity level is higher than the true ambient light intensity. The positive deviation is also time-varying and difficult to predict. This is because the amount of light emitted from the display is typically not constant; it changes according to the changing display content presented by the display device.

[0029] One way to improve the accuracy of light intensity readings produced by an ALS device used in conjunction with a display device is to position the ALS device away from any light emitting sources of the display device. An example of this approach is to employ a display panel having a "notch" design. Such a display panel lacks any light emitting elements (e.g., OLED pixels) in the notch area. The ALS device can be positioned within the notch area, thereby ensuring that the ALS device is separated from any light emitting elements by a minimum distance. The larger the notch area, the more separation can be ensured between the ALS device and any light emission. Thus, positive deviations in the ALS readings attributed to light emissions from the display device are reduced. While this approach can improve the accuracy of the ALS readings, the presence of the notch area reduces the useful display area of ​​the display device and introduces an element that is not necessarily aesthetically pleasing to the overall appearance of the display of the user device.

[0030] Another method to improve the accuracy of light intensity readings produced by an ALS device used in conjunction with a display device is to position the ALS device below the display panel and apply runtime calibration / compensation to the ALS measurements obtained by taking into account the current display content. Using the knowledge of the display content presented in each frame of the display data, an estimate of the positive bias in the ALS measurement caused by the additional photons emitted by the display device can be made. The estimate of the positive bias can then be used in the runtime calibration / compensation to adjust the ALS measurement in order to remove the positive bias. However, such runtime calibration / compensation can be expensive in terms of computational, memory, and power resources. Large amounts of display frame content may need to be dumped into memory in real time, which requires a large amount of power. Even if the memory dump is performed in hardware, software running in a processor such as a system on a chip (SOC) may be required to control the hardware for large memory transfers. In addition, the calibration / compensation algorithm for estimating and removing positive biases may be complex, taking into account the large amount of display frame content and predicting the impact of the display frame content on the ALS measurement.

[0031] Aspects of the present disclosure provide an efficient method to improve the accuracy of light intensity while allowing the ALS device to be positioned near the light emitting elements of the display panel, for example, by placing the ALS device below the display panel. This approach avoids the significant computational, memory, and power costs associated with performing runtime calibration / compensation of ALS measurements that take into account display frame content. At the same time, the benefits of improved aesthetics of a display panel without a "notch" area can be achieved.

[0032] Figure 1 An effective method for improving the accuracy of light intensity readings generated by an ALS device used in conjunction with a display device according to one aspect of the present disclosure is shown. A front view of a user device 100 is presented. In this example, the user device 100 is a smartphone. However, the techniques discussed herein are not limited to use in smartphone devices. Other types of user devices, such as laptops, wearable devices, automotive displays, television displays, etc., may implement the described techniques and components according to aspects of the present disclosure. Figure 1 As shown, the user device 100 includes a display device including a display panel 102 for presenting display content. The ALS device 104 is located below the display panel 102 and is therefore Figure 1 The user only sees the content displayed on the display panel 102, but not the ALS device 104. This arrangement increases the display area of ​​the display device and provides a beautiful appearance for the user device 100. Figure 1 An optional front side camera 106 is also shown in FIG. 1. In this example, a small opening is introduced in the display panel 102 for image capture by the optional front side camera 106.

[0033] Figure 2 An exploded view of some illustrative components of a user device 200 is presented that positions an ALS sensor below a display panel according to one aspect of the present disclosure. The user device 200 may be Figure 1 . As shown, the user device 200 includes a protective glass layer 202, an organic light emitting diode (OLED) panel 204 including a plurality of individual OLED sub-pixels 206, a printed circuit board (PCB) 208, and a base 210, among others. According to various aspects of the present disclosure, the OLED panel 204 is optically transparent and is controlled using various drive signals. As an example, the OLED panel 204 can be an active matrix organic light emitting diode (AMOLED) panel including a plurality of AMOLED sub-pixels. Optionally, the user device 200 can also include a front camera 220 that can capture images through an opening 222 in the OLED panel 204 and an area 224 of the protective glass layer 202.

[0034] Figure 2 The various layers shown in the exploded view of the user device 200, including the protective glass layer 202, the OLED panel 204, the PCB 208, and the base 210, appear to be separated from each other along the vertical axis. However, once the components of the user device 200 are assembled together, the layers are positioned very close to each other along the vertical axis. In particular, one or more light emitters 212, which may be specific ones of the plurality of individual OLED sub-pixels 206, may be positioned adjacent to and directly above the ALS device 214 mounted on the PCB 208.

[0035] In an outward direction, one or more light emitters 212 can emit display light 216 from the user device 200 to the external environment. In an inward direction, ambient light 218 from the external environment can pass through the transparent OLED panel 204 to the ALS device 214. Each of the one or more light emitters 212 can emit light according to an emission profile. In the case of a transparent display such as the OLED panel 204, the emission profile can extend beyond the outward direction such as shown by the display light 216. In fact, the emission profile can include a direction pointing inwardly toward the user device 200 and reaching the ALS device 214. Therefore, in addition to photons of light from the environment, the ALS device 214 can also potentially collect photons of light emitted from the display panel 204, thereby introducing a positive bias in the ALS measurement.

[0036] In order to effectively reduce such positive bias in ALS measurements, aspects of the present disclosure provide an improved technique for sensing ambient light intensity. The technique may include positioning a light sensor near one or more light emitters that form at least a portion of a display device. The technique may also include providing one or more timing signals to (1) enable the light sensor to sense ambient light during a first portion of a repeating timing pattern, and (2) enable one or more light emitters to emit display light during a second portion of the repeating timing pattern, wherein the repeating timing pattern is defined by the one or more timing signals, wherein the first portion does not substantially overlap with the second portion of the repeating timing pattern, and wherein the first portion of the repeating timing pattern corresponds to a duty cycle for sensing ambient light within the repeating timing pattern, and the second portion of the repeating timing pattern corresponds to a duty cycle for emitting display light within the repeating timing pattern. According to various embodiments of the present disclosure, one or more light emitters emit light in response to enabling the first portion of the timing signal. According to various embodiments, the light sensor senses light in response to enabling the second portion of the timing signal. The technique may also include generating an ambient light intensity measurement based on one or more readings of the light sensor. In addition, the technology can include adjusting the duty cycle for sensing ambient light and the duty cycle for emitting display light within a repeating timing pattern in response to ambient light intensity measurements. In this context, the term "substantially non-overlapping" should be understood as not having a significant degree of overlap between two time periods. No overlap between two time periods, or a slight overlap of no more than 50%, 40%, 30%, 20% or 10% of the duration of the shorter of the two time periods, should be considered to have no substantial overlap, as the term is understood herein. According to one aspect of the present disclosure, the first portion substantially does not overlap with the second portion of the repeating timing pattern, meaning that when the light sensor is operating to sense ambient light, the first portion and the second portion ensure that no display light is emitted from one or more light emitters. Various aspects of this technology according to various aspects of the present disclosure are discussed in more detail below.

[0037] Figure 3 1 is a block diagram illustrating one or more time division duplex (TDD) timing signals provided by a semiconductor device separate from a timing controller for a light sensor, a timing controller for a display device, and a processor coupled to the light sensor and the display device according to one aspect of the present disclosure. As shown, a user device 300 may include components such as an OLED panel 302, a display driver integrated circuit (DDIC) 304, an ALS device 306, a TDD timing controller integrated circuit (IC) 308, a SOC 310, and a memory 312. The OLED panel 302 may be a device previously described in Figure 2310 is coupled to the DDIC 304 and the ALS device 306 via a bus 314. A memory 312 is also connected to the bus 314 and can be accessed by the SOC 310, the TDD timing controller IC 308, the DDIC 304 and / or the ALS device 306.

[0038] The display device for user device 300 may include an OLED panel 302 and a DDIC 304. Image frame data may be provided to DDIC 304 from a source such as a memory 312, which works in conjunction with a processor such as SOC 310 and / or a dedicated graphics processor (not shown). DDIC 304 may receive display content data for each frame to be displayed and provide appropriate drive signals 316 to OLED panel 302. Such drive signals may follow pulse width modulation (PWM) or other schemes for converting display content data into corresponding drive signals of a specific amplitude and / or duration for each sub-pixel of OLED panel 302. Each "pixel" of OLED panel 302 may include multiple sub-pixels of various colors, such as (to name a few) red, green, and blue (RGB) sub-pixels, red, green, green, blue (RGGB) sub-pixels, and red, green, blue, white (RGBW) sub-pixels.

[0039] like Figure 3 As shown, the TDD timing controller IC 308 is an example of a semiconductor device that is separate from the timing controller of the ALS device 306, separate from the timing controller of the DDIC 304 of the display device, and separate from the SOC 310. The TDD timing controller IC 308 can be implemented in various ways. As an example only, the TDD timing controller IC 308 can be implemented as a simple microcontroller (MCU). In different aspects of the present disclosure, the TDD timing controller IC 308 can be implemented as a part of any one or combination of an application specific integrated circuit (ASIC), an application specific standard product (ASSP), an MCU and / or a full custom integrated circuit. The TDD timing controller IC 308 provides a display enable timing signal 318 to the DDIC 304. The display enable timing signal 318 defines a display light-emitting period, during which the DDIC 304 is allowed to provide a drive signal 316 to drive the sub-pixels of the OLED panel 302. Any display signal modulation scheme, such as PWM, will be implemented within each display light-emitting period defined by the display enable timing signal 318. Additionally, the TDD timing controller IC 308 provides an ALS enable timing signal 320 to the ALS device 306. The ALS enable timing signal 320 defines an ALS sensing period during which the ALS device 306 is allowed to capture ambient light from the external environment and generate ALS measurements.

[0040] Here, a time division duplex (TDD) timing structure is proposed as an example of a repeating timing pattern. The first portion of the repeating timing pattern corresponds to a duty cycle for sensing ambient light. The second portion of the repeating timing pattern corresponds to a duty cycle for emitting display light. The TDD structure positions the duty cycle for sensing ambient light so that it does not substantially overlap with the duty cycle for emitting display light. By doing so, the TDD timing structure provides a framework to ensure that the light captured by the ALS device consists primarily of ambient light from the external environment and is not contaminated by display light from the display device. The timing structure allows display content to be presented to the user while providing the ALS device with sufficient opportunities to make ambient light intensity measurements.

[0041] Figure 4 4 is a block diagram illustrating one or more time division duplex (TDD) timing signals provided by a processor coupled to a light sensor and a display device according to one aspect of the present disclosure. As shown, a user device 400 may include components such as an OLED panel 402, a display driver integrated circuit (DDIC) 404, an ALS device 406, a TDD timing controller module 408, a SOC 410, and a memory 412. The OLED panel 402 may be a device previously described in Figure 2 4. The SOC 410 is coupled to the DDIC 404 and the ALS device 406 via a bus 414. A memory 412 is also connected to the bus 414 and can be accessed by the SOC 410, the TDD timing controller module 408, the DDIC 404, and / or the ALS device 406.

[0042] A display device for user device 400 may include an OLED panel 402 and a DDIC 404. Image frame data may be provided to DDIC 404 from a source such as a memory 412, which works in conjunction with a processor such as SOC 410 and / or a dedicated graphics processor (not shown). DDIC 404 may receive display content data for each frame to be displayed and provide appropriate drive signals 416 to OLED panel 402. Such drive signals may be formed based on pulse width modulation (PWM) or other light modulation schemes, as previously described.

[0043] like Figure 4As shown, the TDD timing controller module 408 is an example of a timing controller implemented within a processor, in this case, the processor is the SOC 410. As such, the TDD timing controller module 408 can be implemented in the form of a software module, i.e., a portion of executable instructions that can be executed by the SOC 410 to generate input / output (I / O) signals to control display and ALS timing. Here, the TDD timing controller module 408 provides a display enable timing signal 418 to the DDIC 404. The display enable timing signal defines a display light emission period during which the DDIC 404 is allowed to provide a drive signal 416 to drive the sub-pixels of the OLED panel 402. In addition, the TDD timing controller module 408 also provides an ALS enable timing signal 420 to the ALS device 406. The ALS enable timing signal defines an ALS sensing period during which the ALS device 306 is allowed to capture ambient light from the external environment and generate an ALS measurement.

[0044] Again, time division duplex (TDD) timing is used as an example of a repeating timing pattern. The first portion of the repeating timing pattern corresponds to a duty cycle for sensing ambient light. The second portion of the repeating timing pattern corresponds to a duty cycle for emitting display light. The TDD structure ensures that the duty cycle for sensing ambient light does not substantially overlap with the duty cycle for emitting display light.

[0045] Figure 5 1 is a block diagram illustrating one or more time division duplex (TDD) timing signals provided by a combined timing controller for a light sensor and a display device according to one aspect of the present disclosure. As shown, a user device 500 may include components such as an OLED panel 502, a combined display driver integrated circuit (DDIC) and an ALS controller 504 (which includes a TDD timing controller 506), a SOC 508, and a memory 510. The OLED panel 502 may be a previously disclosed Figure 2 . The SOC 508 is coupled to the combined DDIC and ALS controller 504 via a bus 512. A memory 510 is also connected to the bus 512 and can be accessed by the SOC 508 and / or the combined DDIC and ALS controller 504.

[0046] A display device for user device 500 may include an OLED panel 502 and a portion of a combined DDIC and ALS controller 504. Image frame data may be provided to the combined DDIC and ALS controller 504 from a source such as a memory 510, which works in conjunction with a processor such as an SOC 508 and / or a dedicated graphics processor (not shown). The combined DDIC and ALS controller 504 may receive display content data for each frame to be displayed and provide appropriate drive signals 514 to the OLED panel 502. Such drive signals may be formed based on pulse width modulation (PWM) or other techniques, as previously described.

[0047] like Figure 5 As shown, the combined DDIC and ALS controller 504 includes a TDD timing controller 506, which is an example of a combined timing controller for a light sensor and a display device. The TDD timing controller 506 provides a display enable timing signal to the DDIC portion of the combined DDIC and ALS controller 504. The display enable timing signal defines a display light emission period, during which the DDIC portion of the combined DDIC and ALS controller 504 is allowed to provide a drive signal 514 to drive the sub-pixels of the OLED panel 502. In addition, the TDD timing controller 506 also provides an ALS enable timing signal to the ALS portion of the combined DDIC and ALS controller 504. The ALS enable timing signal defines an ALS sensing period, during which the ALS portion of the combined DDIC and ALS controller 504 is allowed to capture ambient light from the external environment and generate ALS measurements. By way of example only, the combined DDIC and ALS controller 504 can be implemented as an application specific integrated circuit (ASIC). In various aspects of the present disclosure, the combined DDIC and ALS controller 504 may be implemented as part of any one or a combination of an ASIC, an application specific standard product (ASSP), and / or a full custom integrated circuit.

[0048] Again, time division duplex (TDD) timing is used as an example of a repeating timing pattern. The first portion of the repeating timing pattern corresponds to a duty cycle for sensing ambient light. The second portion of the repeating timing pattern corresponds to a duty cycle for emitting display light. The TDD structure ensures that the duty cycle for sensing ambient light does not substantially overlap with the duty cycle for emitting display light.

[0049] Figure 6 6 is a block diagram showing further details of a display driver integrated circuit (DDIC) according to some aspects of the present disclosure. As shown, DDIC 600 may include components such as one or more sets of column drivers 602, one or more sets of row drivers 604, and a display timing controller 606. DDIC 600 may be a display driver integrated circuit (DDIC) as previously described. Figure 3DDIC 304 or Figure 4 An example of DDIC 404 is shown in FIG. 606. Display timing controller 606 provides appropriate timing and / or control signals to column driver 602 and row driver 604, which in turn generate column drive signals and row drive signals, respectively, as output signals provided from DDIC 600 to a display panel (not shown). Column drive signals and row drive signals are coordinated to individually address and turn on each sub-pixel in the display panel with a specific amplitude and / or a specific duration (e.g., in a PWM scheme) according to the brightness level of each sub-pixel, thereby presenting a frame of color image data on the display panel. Image frames are updated according to a frame rate, for example, at 60 frames per second (FPS), 90 FPS, or some other frame rate. Therefore, display timing controller 606 plays an important role in the correct generation of drive signals for the display panel.

[0050] According to some aspects of the present disclosure, the DDIC 600 receives one or more TDD timing signals 608 from an external source and provides them to the display timing controller 606. An example of the one or more TDD timing signals 608 is a display enable timing signal, such as the previously described Figure 3 The display enable timing signal 318 shown in or Figure 4 4. Such display enable timing signals may define time periods during which the DDIC 600 is allowed to provide column drive and row drive signals to turn on sub-pixels in the display panel.

[0051] DDIC 600 can include Figure 6 DDIC 600 may include other components not explicitly shown in the drawings. For example, DDIC 600 may include a power generation circuit for providing sufficient voltage / current to the column drive signal and the row drive signal. DDIC 600 may also include a memory such as a random access memory (RAM) for temporarily storing sub-pixel values ​​and other data. DDIC 600 may also include an interface module that supports a data interface for transferring data to and from DDIC 600.

[0052] Figure 7 7 is a block diagram showing further details of an ambient light sensor (ALS) device according to some aspects of the present disclosure. As shown, the ALS device 700 includes at least one photodiode 702, a low pass filter (LPF) 704, an analog-to-digital converter (ADC) 706, an output buffer 708, and an ALS timing controller 710. The ALS device 700 may be the previously described Figure 3 The ALS device 306 shown in or Figure 4An example of an ALS device 406 is shown in FIG. 700. The ALS device 700 measures the intensity of ambient light. As a light sensor, at least one photodiode 702 converts photons of ambient light from an external environment into electric charge. The more intense the ambient light, the greater the charge generated in a given duration. The LPF 704 can remove noise and other high-frequency components from the signal generated by the at least one photodiode 702. According to some aspects of the present disclosure, the generated charge can be accumulated as an analog signal. For example, the charge can be accumulated in a charge storage component, such as a capacitive circuit, i.e., a circuit including one or more capacitors (not shown). Such a charge storage component can form a part of the LPF 704. The accumulated charge, which can be reflected as a voltage, can then be converted by the ADC 706 into a digital light intensity value representing an ALS reading. According to some aspects of the present disclosure, the generated charge can be converted by the ADC 706 into a digital light intensity value, which can be digitally accumulated to generate an ALS measurement. In some aspects of the present disclosure, a combination of analog accumulation and digital accumulation of charge and / or light intensity values ​​can be performed to generate an ALS measurement. Each ALS reading can be caused by multiple exposures of at least one photodiode 702 over one or more specified time periods. Additionally, multiple ALS readings may be accumulated in digital and / or analog form to generate an ALS measurement.

[0053] According to some aspects of the present disclosure, the ALS device 700 receives one or more TDD timing signals 712 from an external source and provides them to the ALS timing controller 710. An example of the one or more TDD timing signals is an ALS enable signal, such as the one described above. Figure 3 The ALS enable timing signal 320 or Figure 4 ALS enable timing signal 420 is shown. Such an ALS enable timing signal can define a time period during which the ALS device 700 is allowed to sense photons for generating ALS measurements. Thus, when the ALS enable timing signal indicates that ALS measurements are not allowed (e.g., the ALS enable timing signal exhibits a LOW or OFF value), analog and / or digital accumulation of charge and / or light intensity values ​​can be stopped, and any storage of such charge or values ​​can be retrieved and / or cleared. For example, during such a time period, any capacitive circuit used to store charge generated by at least one photodiode 702 can be disconnected from the signal source, and any digital accumulation component can be paused. When the ALS enable timing signal indicates that ALS measurements are allowed (e.g., the ALS enable timing signal exhibits a HIGH or ON value), analog and / or digital accumulation of charge and / or light intensity values ​​can begin and continue until an ALS measurement is generated.

[0054] Figure 88 is a block diagram showing further details of a combined DDIC and ALS controller according to some aspects of the present disclosure. As shown, the combined DDIC and ALS controller 800 includes one or more sets of row drivers 802, one or more sets of column drivers 804, a low pass filter (LPF) 808, an analog-to-digital converter (ADC) 810, an output buffer 812, and a combined display and ALS timing controller 814. The combined DDIC and ALS controller 800 may be a previously described Figure 5 8. An example of a combined DDAC and ALS controller 504 is shown in FIG. Row drivers 802 and column drivers 804 generate row and column drive signals as output signals provided to a display panel (not shown) from the combined DDIC and ALS controller 800. The row and column drive signals are coordinated to individually address and turn on each sub-pixel in the display panel at a specific amplitude and / or a specific duration (e.g., according to a PWM scheme) based on the brightness level of each sub-pixel, thereby presenting a color image data frame on the display panel.

[0055] As a light sensor, the photodiode 806 converts photons of ambient light from the external environment into electric charge. The more intense the ambient light, the greater the charge generated in a given duration. The LPF 808 can remove noise and other high-frequency components from the signal generated by the photodiode 806. According to some aspects of the present disclosure, the generated charge can be accumulated as an analog signal. For example, the charge can be accumulated in a charge storage component, such as a capacitive circuit, i.e., a circuit including one or more capacitors (not shown). Such a charge storage component can form a part of the LPF 808. The accumulated charge, which can be reflected as a voltage, can then be converted by the ADC 810 into a digital light intensity value representing an ALS reading. According to some aspects of the present disclosure, the generated charge can be converted by the ADC 810 into a digital light intensity value, which can be digitally accumulated to generate an ALS measurement. In some aspects of the present disclosure, a combination of analog accumulation and digital accumulation of charge and / or light intensity values ​​can be performed to generate an ALS measurement. Each ALS reading can be caused by multiple exposures of the photodiode 806 over one or more specified time periods. In addition, multiple ALS readings can be accumulated in digital and / or analog form to generate an ALS measurement.

[0056] According to certain aspects of the present disclosure, the combined display and ALS timing controller 814 provides one or more timing signals to coordinate the timing of display and ALS operations based on a time division duplex (TDD) scheme. As shown, the combined display and ALS timing controller 814 provides appropriate timing and / or control signals to the row driver 802 and the column driver 804. Such display enable timing signals can define a time period during which the DDIC portion of the combined DDIC and ALS controller 800 is allowed to provide column drive and row drive signals to turn on sub-pixels in the display panel. In addition, the combined display and ALS timing controller 814 also provides one or more ALS enable timing signals to the ADC 810 and the output buffer 812 to define a time period during which the ALS portion of the combined DDIC and ALS controller 800 is allowed to sense photons to generate ALS measurements. Thus, when the ALS enable timing signal indicates that ALS measurement is not allowed (e.g., the ALS enable timing signal presents a LOW or OFF value), analog and / or digital accumulation of charge and / or light intensity values ​​can be stopped, and any storage of such charge or values ​​can be retrieved and / or cleared. For example, any capacitive circuit used to store charge generated by the photodiode 806 can be disconnected from the signal source during such a time, and any digital accumulation component can be paused. When the ALS enable timing signal indicates that ALS measurement is allowed (e.g., the ALS enable timing signal presents a HIGH or ON value), analog and / or digital accumulation of charge and / or light intensity values ​​can begin and continue until an ALS measurement is generated. The use of a combined DDIC and ALS controller 800 can be associated with benefits such as reducing the overall physical size and / or thickness of a user device in which display and ALS functions are implemented. Such a unified display and ALS hardware chipset can provide improved, flexible functional control of display and ALS operations.

[0057] Fig. 9 is a timing diagram illustrating various signals based on a time division duplex (TDD) structure for supporting periodic, substantially non-overlapping durations for sensing ambient light and emitting display light according to some aspects of the present disclosure. Shown in the figure are a trigger signal 902, a TDD signal 904, a display enable timing signal 906, and an ALS enable timing signal 908. The TDD signal 904 is a periodic signal consisting of a plurality of TDD periods 910, which reflects how time is divided between display and ALS functions. The trigger signal 902 may be based on a clock signal (not shown), and is configured to provide a periodic trigger to start each TDD period 910. Each TDD period 910 of the TDD signal 904 includes a display enable portion 912 and an ALS enable portion 914. In Fig. 9In the example shown, a HIGH value of the TDD signal 904 represents a display enabled portion 912 of the TDD period, while a LOW value of the TDD signal 904 represents an ALS enabled portion 914 of the TDD period. In other examples, the assignment of HIGH and LOW values ​​of the TDD signal 904 may be swapped or otherwise changed. Fig. 9 In the example shown, the display enabled portion 912 of the TDD period does not substantially overlap - in fact, does not overlap at all - with the ALS enabled portion 914 of the TDD period. In other examples discussed later, the display enabled portion 912 of the TDD period may slightly overlap with the ALS enabled portion 914 of the TDD period.

[0058] According to one aspect of the present disclosure, the TDD signal 904 can be used as a common signal provided to both the display timing controller and the ALS timing controller. Figure 6 , the TDD signal 904 can be used as the TDD timing signal 608 provided to the display timing controller 606. The display timing controller 606 will recognize that the HIGH value of the TDD signal 904 corresponds to the display enable portion 912 of each TDD period, and will enable the display operation accordingly. Return to Reference Figure 7 , the same TDD signal 904 can also be used as the TDD timing signal 712 provided to the ALS timing controller 710. The ALS timing controller 710 will recognize that the LOW value of the TDD signal 904 corresponds to the ALS enable portion 914 of each TDD period, and will enable ALS operation accordingly. For example, using such a common timing signal for both display and ALS enable can reduce the number of signals and simplify the implementation in terms of routing.

[0059] In contrast, according to another aspect of the present disclosure, multiple timing signals may be used to enable display and ALS operations. For example, the multiple timing signals may include (1) a first timing signal (e.g., ALS enable timing signal 908) to enable the light sensor to sense ambient light during a first portion of a repeating timing pattern, and (2) a second timing signal (e.g., display enable timing signal 906) different from the first timing signal to enable the one or more light emitters to emit display light during the second portion of the repeating timing pattern. Here, the ALS enable timing signal 908 may also be used as the TDD timing signal 712 provided to the ALS timing controller 710. The ALS timing controller 710 will recognize that the HIGH value of the ALS enable timing signal 908 corresponds to the ALS enable portion 914 of each TDD period, and will enable ALS operation accordingly. In addition, the display enable timing signal 906 may be used as the TDD timing signal 608 provided to the display timing controller 606. The display timing controller 606 will recognize that the HIGH value of the display enable timing signal 906 corresponds to the display enable portion 912 of each TDD period, and will enable display operation accordingly. From the perspective of the ALS timing controller and the display timing controller, using separate timing signals for display and ALS enable can simplify the interpretation of the timing signals.

[0060] By using periodic temporal structures (such as Fig. 9The ALS and display operations are spaced by a repetitive TDD period 910 as shown, providing a flexible framework that is well integrated with the display function. For example, a display modulation scheme such as pulse width modulation (PWM) for modulating the duration of light to achieve an appropriate brightness level can be limited to the display enable portion 912 of each TDD period 910. For example, a display driver such as a DDIC device can control the display drive signal according to the PWM scheme. According to various aspects of the present disclosure, only during the display enable portion 912 of the TDD period 910, the display driver (e.g., DDIC304, DDIC 404, combined DDIC / ALS controller 504, DDIC 600, and combined DDIC / ALS controller 800) is allowed to turn on the sub-pixels in the display panel. For example, the PWM period can be implemented on the display enable portion 912 of the TDD period. Under such an arrangement, the full brightness of the sub-pixel will correspond to the illumination of the sub-pixel for the entire duration (e.g., 100%) of the display enable portion 912 of the TDD period 910. If the brightness level of the defined sub-pixel is less than full brightness, the sub-pixel may be illuminated for a corresponding percentage (e.g., 40%) of the display-enabled portion 912 of the TDD period 910, as defined by the PWM scheme. According to some aspects of the present disclosure, the TDD / PWM rate is higher than the frame rate of the image data presented by the display unit. In one example, the PWM rate is 1000 Hz and the frame rate of the display unit is 120 Hz. Here, according to the TDD / PWM scheme, each pixel frame may be turned on / off 8 times. In other words, during each frame of display data, 8 TDD periods may occur, and each TDD period may include one PWM period.

[0061] Fig.10 A display and ALS duty period dynamically adjusted based on current ALS values ​​according to some aspects of the present disclosure are shown. According to one aspect, in response to a decrease in ambient light intensity measurement, the duty cycle for sensing ambient light within a repeating timing pattern (e.g., repeating TDD period 1008) is increased. According to one aspect, in response to an increase in ambient light intensity measurement, the duty cycle for sensing ambient light within a repeating timing pattern (e.g., repeating TDD period 1008) is decreased. Fig.10 The operation of making such dynamic adjustments through the presentation of TDD signals is illustrated as being present at three levels of detected ambient light intensity, corresponding to TDD signal 1002 , TDD signal 1004 , and TDD signal 1006 .

[0062] The TDD signal 1002 present when the ambient light intensity is relatively low has a relatively short display duty cycle and a relatively long ALS duty cycle. For example, such low ambient light intensity may correspond to a user device (e.g., user device 100) in an indoor environment or other low light environment. As shown, the display enabled portion of the TDD period 1008 is relatively short, reflecting a shorter display duty cycle. Under such low ambient light conditions, the required display light intensity is not very high, and therefore a relatively short display duty cycle may be sufficient to present image content on the screen of the user device. In contrast, the ALS enabled portion of the TDD period 1008 is relatively long, reflecting a longer ALS duty cycle. Under such low light conditions, fewer photons can be obtained from the surrounding environment. According to one aspect of the present disclosure, the duration of the ALS enabled portion of the TDD period 1008 is controlled as a function of the detected ambient light intensity, resulting in an extension of the exposure period of the ALS sensor under low light conditions to capture a sufficient number of photons, thereby producing accurate and reliable ALS measurements.

[0063] As the ambient light intensity increases, the display duty cycle and the ALS duty cycle can be dynamically adjusted. For example, the TDD signal 1004 that exists when the ambient light intensity is approximately midway between the low extreme and the high extreme has a longer display duty cycle and a shorter ALS duty cycle than the TDD signal 1002. In the particular example shown, the display duty cycle and the ALS duty cycle are approximately equal.

[0064] With the further increase of ambient light intensity, the display duty cycle and the ALS duty cycle may be further dynamically adjusted. For example, the TDD signal 1006 present when the ambient light intensity is relatively high has a relatively long display duty cycle and a relatively short ALS duty cycle. Such high ambient light intensity may correspond to, for example, a user device in an outdoor environment or other high brightness environment. As shown in the figure, the display enabling portion of the TDD period 1008 is relatively long, reflecting a longer display duty cycle. Under such high brightness conditions, the required display light intensity may also be very high, which requires a longer display duty cycle, so as to generate enough light for the user to distinguish the image content presented on the screen of the user device in the presence of bright ambient light. In contrast, the ALS enabling portion of the TDD period 1008 is relatively short, reflecting a short ALS duty cycle. According to one aspect of the present disclosure, the duration of the ALS-enabled portion of the TDD period 1008 is controlled as a function of the detected ambient light intensity, resulting in shortening the exposure period of the ALS sensor under high brightness conditions to capture a sufficient number of photons to produce accurate and reliable ALS measurements, while leaving a longer display-enabled portion of the TDD period 1008 for producing sufficient display light in the presence of bright ambient light.

[0065] Although Fig.10 The previous discussion of describes a transition from a low light condition to a high light condition, but dynamic adjustment of the display and ALS duty cycle may also occur when the ambient light intensity transitions in the opposite direction, i.e., from a high light condition to a low light state. Specifically, as the ALS intensity value decreases, the display duty cycle may be decreased, and the ALS duty cycle may be increased. The ALS intensity may be continuously monitored, and the display and ALS duty cycle may be continuously adjusted in a dynamic manner.

[0066] In general, the brighter the ambient light environment, the longer the display duty cycle, and the shorter the ALS duty cycle. The darker the ambient light environment, the shorter the display duty cycle, and the longer the ALS duty cycle. As a simple example, in an outdoor environment, the display duty cycle may be, for example, 95%, and the ALS duty cycle may be, for example, 5%. In an indoor environment, the display duty cycle may be, for example, 5%, and the ALS duty cycle may be, for example, 95%. The specific duty cycle values ​​selected in this example are for illustration only. Additionally, Fig.10 The specific ratios of display duty cycles and ALS duty cycles shown in the figures, including those shown for TDD signals 1002, 1004, and 1006, are provided for illustration purposes only. The durations and relative duty cycles shown in the figures are not meant to limit possible implementations of the present disclosure. For example, Fig.10 One implementation is shown in which the TDD signal 1004 exhibits a display duty cycle and an ALS duty cycle that appear equal in duration when the ambient light intensity is approximately midway between the low extreme and the high extreme. In a different implementation, the TDD signal 1004 may exhibit a display duty cycle and an ALS duty cycle that are unequal in duration for ALS intensity values ​​that are approximately midway between the high extreme and the low extreme (e.g., the display duty cycle may be longer or shorter than the ALS duty cycle).

[0067] In fact, in one aspect of the present disclosure, the display duty cycle (Δ_DutyCycleDisplay) and the change in the ALS duty cycle (Δ_DetyCycleALS) can be flexibly defined as a function of the change in the detected ambient light intensity (Δ_ALS). For example: Δ_DutyCycleDisplay = F1{Δ_ALS} (Equation 1)

[0068] Δ_DutyCycleALS=F2{Δ_ALS} (Equation 2)

[0069] In accordance with aspects of the present disclosure, function F1 is an increasing function and function F2 is a decreasing function. In some aspects, one or both of functions F1 and F2 are linear functions. In other aspects, one or both of functions F1 and F2 are non-linear functions, such as exponential functions. In certain aspects, the display duty cycle and the ALS duty cycle also satisfy the constraint that they have a combined duration that is equal to or less than the TDD period:

[0070] (Δ_DutyCycleDisplay + Δ_DutyCycleALS) ≤ TDD period (Equation 3)

[0071] Fig.11 Illustrated are aspects of the present disclosure in which blank periods and overlapping periods are used in enable signals provided for display operations and ALS operations. Such blank periods can be used to accommodate various types of delays associated with physical components that do not respond immediately to enable signals but can have a non-zero response time. Such an arrangement can be employed to ensure that a first portion of a repeating timing pattern does not substantially overlap with a second portion of the repeating timing pattern. Specifically, when the light sensor is operating to sense ambient light, the first and second portions ensure that no display light is emitted from one or more light emitters. The figure illustrates a TDD signal 1102, a display enable signal 1104, and an ALS enable signal 1106, which form part of a repeating timing pattern. The repeating timing pattern is presented in TDD periods 1108 that repeat over time.

[0072] In one aspect of the present disclosure, the repeating timing pattern includes one or more enable blank periods 1112, during which one or more light emitters (e.g., sub-pixels of the OLED panel 204) and light sensors (e.g., the ALS device 214) are disabled. For example, each of the one or more enable blank periods can be located between the end of the second portion of the repeating timing pattern and the beginning of the first portion of the repeating timing pattern. Here, each of the one or more enable blank periods 1112 can be based on a display off response time associated with the one or more light emitters. In addition, the one or more enable blank periods 1112 can also be based on parameters other than the display off response time. According to one aspect of the present disclosure, the display off response time is a delay from the end of the second portion of the repeating timing pattern to the time when the one or more light emitters are actually turned off. There is a response time or delay (referred to herein as the "display off response time") between the moment when the display enable signal 1104 switches from HIGH to LOW and the moment when the light emitters of the display panel are actually turned off. To avoid turning on the ALS sensor immediately while the emitter is still on, an enable blanking period 1112 may be introduced to allow the display's light emitters time to turn off before allowing the ALS enable signal 1106 to transition from LOW to HIGH. In this aspect of the disclosure, the display off response time tends to increase the duration of the enable blanking period 1112.

[0073] Each of the one or more enable blank periods 1112 can be further based on a sensor turn-on response time associated with the light sensor. In addition, the one or more enable blank periods 1112 can also be based on parameters other than the sensor turn-on response time. According to one aspect of the present disclosure, the sensor turn-on response time is the delay from the start of the first portion of the repeating timing pattern to the time when the light sensor is actually turned on. Like the light emitter, the ALS sensor is a physical device and may also have a response time. In particular, there may be a response time or delay (referred to herein as the "ALS turn-on response time") between the moment when the ALS enable signal 1106 transitions from LOW to HIGH and the moment when the ALS sensor actually begins collecting photons to generate an ALS reading. The enable blank period 1112 can be adjusted to take advantage of this delay in the activation of the ALS sensor. Here, the ALS turn-on response time tends to reduce the duration of the enable blank period 1112.

[0074] In another aspect of the present disclosure, the repeating timing pattern includes one or more enable overlap periods 1114 during which one or more light emitters and light sensors are enabled. The presence of response time for turning on the emitter and / or turning off the ALS sensor can allow for overlapping of enable signals. For example, each of the one or more enable overlap periods is formed by the overlap of the end portion of the first portion of the repeating timing pattern and the beginning portion of the second portion of the repeating timing pattern. Here, each of the one or more enable overlap periods 1114 can be based on a display turn-on response time associated with the one or more light emitters. In addition, the one or more enable overlap periods 1114 can be further based on parameters other than the display turn-on response time. According to one aspect of the present disclosure, the display turn-on response time is the delay from the start of the second portion of the repeating timing pattern to the time when the one or more light emitters are actually turned on. There may be a response time or delay (referred to herein as the "display turn-on response time") between the moment when the display enable signal 1104 transitions from LOW to HIGH and the moment when the light emitters of the display panel are actually turned on. With this delay, the display enable signal 1104 may transition from LOW to HIGH before the ALS enable signal 1106 transitions from HIGH to LOW, which introduces an overlap of the HIGH states of the display enable signal 1104 and the ALS enable signal 1106. Here, the display-on response time tends to increase the duration of the enable overlap period 1114. Although the enable overlap period 1114 represents a time region in which the HIGH state of the display enable signal 1104 slightly overlaps with the HIGH state of the ALS enable signal 1106, the duration of the overlap is relatively short. Even when there is such a slight overlap, the HIGH state of the display enable signal 1104 and the HIGH state of the ALS enable signal 1106 still do not substantially overlap with each other.

[0075] Each of the one or more enable overlap periods 1114 may be further based on a sensor off response time associated with the light sensor. In addition, the one or more enable overlap periods 1114 may be further based on parameters other than the sensor off response time. According to one aspect of the present disclosure, the sensor off response time is a delay from the end of the first portion of the repeating timing pattern to the time when the light sensor is actually turned off. There may be a response time or delay (referred to herein as the "ALS off response time") between the moment when the ALS enable signal 1106 switches from HIGH to LOW and the moment when the ALS sensor actually stops collecting photons for ALS readings. The enable overlap period 1114 may be adjusted to accommodate such delays to ensure that the ALS sensor has actually stopped collecting photons for ALS readings before the emitter of the display panel is turned on. Here, the ALS off response time tends to reduce the duration of the enable overlap period 1114. In one implementation, the enable blank period 1112 and the enable overlap period 1114 may be calculated as:

[0076] Enable blanking period = (display off response time – ALS on response time) (Formula 4)

[0077] Enable overlap period = (display on response time – ALS off response time) (Formula 5)

[0078] Fig.12 A three-dimensional (3D) lookup table (LUT) for mapping input red, green, and blue (RBG) color values ​​to output RGB color values. According to certain aspects of the present disclosure, techniques for coordinating emitted display light and sensed ambient light intensity also include (1) selecting a color mapping data structure (e.g., a 3DLUT) based on a duty cycle for sensing ambient light within a repeating timing pattern and (2) applying a color shift by mapping input display color values ​​to output display color values ​​for one or more frames of displayed content using the selected color mapping data structure. Modern displays such as OLED displays may incorporate a 3D LUT to map input color values ​​to output color values ​​in order to account for specific color characteristics of the display panel. Fig.12A 3D LUT 1200 based on an RGB color architecture is shown, and includes mapped color values ​​organized in a data structure indexed along three axes, namely, a red (R) axis 1202, a green (G) axis 1204, and a blue (B) axis 1206. The three-dimensional space occupied by the 3D LUT defines the color gamut of all possible input RGB values. Each entry in the 3D LUT corresponds to a mapping of an input RGB value to an output RGB value. The input RGB value is a three-dimensional index or address (i.e., x, y, and z coordinate values) that reflects the position of the entry within the 3D LUT. The output RGB value is a value stored as an entry at a specific three-dimensional index or address of the 3D LUT. By using the input RGB value as an index of the 3D LUT, the corresponding output RGB value can be retrieved by reading out the stored entry value. The 3D LUT therefore defines a color transfer function for all possible input RGB color values ​​within the color gamut supported by the three-dimensional data structure. The 3D LUT can occupy a three-dimensional space spanned by the three axes 1202, 1204, and 1206 in full or in part. Although an RGB color architecture is shown here, other color schemes such as a red, green, green, and blue (RGGB) color architecture, a cyan, magenta, yellow, and black (CMYB) color architecture may also be implemented.

[0079] Adjusting the display duty cycle (i.e., the duration that a display subpixel is turned on) may cause significant changes in the color and / or brightness characteristics of the subpixels in the display panel. With respect to color shift, aspects of the present disclosure employ a different 3D LUT 1200 for each display duty cycle or range of display duty cycles in order to compensate for any display duty cycle-dependent color shift that may occur. Fig.13A Examples of different display duty cycles and their corresponding color calibration parameters are shown. As shown, the color shift compensation table 1302 includes an entry indicating that a display duty cycle of 20% may correspond to a color calibration parameter value of 0, which in turn may correspond to a particular 3D LUT (e.g., 3D LUT 0). A display duty cycle of 50% may correspond to a color calibration parameter of 1, which in turn may correspond to another 3D LUT (e.g., 3D LUT 1). A display duty cycle of 70% may correspond to a color calibration parameter of 2, which in turn may correspond to another 3D LUT (e.g., 3D LUT 2). Here, the color calibration parameters of 0, 1, and 2 may correspond to different 3D LUTs in turn. Therefore, the color shift compensation table 1302 compensates for different color shifts that depend on the display duty cycle.

[0080] Fig. 13BExamples of different display duty cycle ranges and their corresponding color calibration parameters are shown. Here, similar operations are performed. However, instead of mapping a specific display duty cycle to an associated 3D LUT, the table maps a range of display duty cycles to an associated 3D LUT. As shown, the color shift compensation table 1304 includes entries indicating that a display duty cycle range of 0%–35% may correspond to a color calibration parameter value of 0. A display duty cycle range of 35%–65% may correspond to a color calibration parameter value of 1. A display duty cycle range of 65%–100% may correspond to a color calibration parameter value of 2. Once again, the color calibration parameters of 0, 1, and 2 may correspond to different 3D LUTs in turn. Any display duty cycle from 0% to 100% may be identified as belonging to one of the specified display duty cycle ranges, and a corresponding 3D LUT may be selected. Storage of a large number of 3D LUTs may be memory intensive. By defining ranges, Fig. 13B The implementation shown in allows the use of a smaller number of 3D LUTs, one 3D LUT per display duty cycle range.

[0081] Regarding color shift, aspects of the present disclosure employ different brightness adjustment ratios for each display duty cycle or range of display duty cycles to compensate for brightness shifts that may occur depending on any display duty cycle. Fig.14A Examples of different display duty cycles and their corresponding brightness level adjustment ratios are shown. As shown, the brightness offset compensation table 1402 includes an entry indicating that a display duty cycle of 20% may correspond to a brightness level adjustment ratio of 3. A display duty cycle of 50% may correspond to a brightness level adjustment ratio of 2. A display duty cycle of 70% may correspond to a brightness level adjustment ratio of 0.8. Here, the adjustment ratios of 3, 2, and 0.8 are illustrative scalar values ​​used as multipliers to modify the brightness level of the display. Therefore, the brightness offset compensation table 1402 compensates for different brightness level offsets that depend on the display duty cycle.

[0082] Fig. 14BExamples of different display duty cycle ranges and their corresponding brightness adjustment ratios are shown. Here, similar operations are performed. However, instead of mapping a specific display duty cycle to an associated brightness adjustment ratio, the table maps a range of display duty cycles to associated brightness adjustment ratios. As shown, the brightness offset compensation table 1404 includes an entry indicating that a display duty cycle range of 0%–35% can correspond to a brightness adjustment ratio of 3. A display duty cycle range of 35%–65% can correspond to a brightness adjustment ratio of 2. A display duty cycle range of 65%–100% can correspond to a brightness adjustment ratio of 0.8. Again, any display duty cycle from 0% to 100% can be identified as belonging to one of the specified display duty cycle ranges, and a corresponding brightness adjustment ratio can be selected. By defining a range, Fig. 14B The implementation shown in allows a smaller number of brightness adjustment ratios to be used, one brightness adjustment ratio for each range of display duty cycles.

[0083] According to one aspect of the present disclosure, various techniques for coordinating the emission of display light and the measurement of ambient light intensity may include a light sensor located below one or more light emitters from the perspective of a user of the display device. Figure 2 , for example, the ALS device 214 is positioned below one or more light emitters 212 of the OLED display panel 204. According to another aspect of the present disclosure, various techniques for coordinating the emission of display light and the measurement of ambient light intensity may include a light sensor positioned side by side with the one or more light emitters from the perspective of a user of the display device. Although not shown in the figures, such an arrangement may include a combined display and light sensor panel consisting of a plurality of composite pixels. For example, each composite pixel may include a set of light emitting sub-pixels (e.g., red sub-pixels, green sub-pixels, and blue sub-pixels) and a light sensing sub-pixel (e.g., an ALS device). In this manner, the ALS device may be placed side by side with one or more light emitters. The techniques described herein for controlling the timing of light emission and light sensing (e.g., using a time division duplex (TDD) timing structure) may be implemented to prevent display light emitted from one or more light emitters from positively biasing ambient light intensity measurements obtained by the ALS device.

[0084] In another aspect of the present disclosure, a display panel such as the OLED display panel 204 may be divided into a plurality of regions, including a first region and a second region. The first region may include a light emitter near a light sensor, while the second region may include a light emitter not near the light sensor. For example, the first region of the OLED display panel 204 may include a separate OLED sub-pixel 212 (light emitter) near the ALS device 214, and the second region may include a separate OLED sub-pixel 206 (light emitter) not near the ALS device 214 (e.g., the remaining light emitters in the OLED display panel 204). For example, the OLED sub-pixel 212 (light emitter) may be positioned so that an optical path exists between the ALS device 214 and the OLED sub-pixel 212 (light emitter). Conversely, the OLED sub-pixel 206 (light emitter) may be positioned so that no optical path exists between the ALS device 214 and the OLED sub-pixel 206 (light emitter). A first repeating timing pattern may be used to control the light emitters in the first region. A second repeating timing pattern may be used to control the light emitters in the second region. In one implementation, a first repeating timing pattern may utilize a TDD structure to ensure that the display duty cycle does not substantially overlap with the ALS duty cycle, while a second repeating timing pattern may not utilize a TDD architecture but maximize the display duty cycle without specifying a duty cycle for ALS sensing. In another implementation, a first repeating timing pattern may utilize a first TDD structure to ensure that the display duty cycle does not substantially overlap with the ALS duty cycle and place more emphasis on the ALS duty cycle. A second repeating timing pattern may utilize a second TDD structure that also ensures that the display duty cycle does not substantially overlap with the ALS duty cycle, but places less emphasis on the ALS duty cycle than the first TDD structure.

[0085] Fig.15 15 is a flow chart illustrating an example of steps performed in a technique for sensing ambient light intensity while presenting display content according to one aspect of the present disclosure. In step 1502, a light sensor is positioned proximate to one or more light emitters that form at least a portion of a display device. An example of the light sensor is ALS device 214, and an example of the one or more light emitters is light emitter 212, such as Figure 2 A unit for positioning a light sensor near one or more light emitters includes, for example, a user device 200 including a stacked structure including a protective glass layer 202, an organic light emitting diode (OLED) panel 204 including a plurality of individual OLED sub-pixels 206, a printed circuit board (PCB) 208, and a chassis 210, all of which are Figure 2 Shown in.

[0086] In step 1504, one or more timing signals are provided to (1) enable a light sensor to sense ambient light during a first portion of a repeating timing pattern, and (2) enable one or more light emitters to emit display light during a second portion of the repeating timing pattern. According to this aspect of the present disclosure, the first portion does not substantially overlap with the second portion of the repeating timing pattern. In addition, the first portion of the repeating timing pattern corresponds to a duty cycle for sensing ambient light within the repeating timing pattern, and the second portion of the repeating timing pattern corresponds to a duty cycle for emitting display light within the repeating timing pattern. The unit for providing one or more timing signals to enable a light sensor to sense ambient light during the first portion of the repeating timing pattern and enable one or more light emitters to emit display light during the second portion of the repeating timing pattern includes, for example, a TDD timing controller IC 308, a TDD timing control module 408 implemented within the SOC 410, and a TDD timing controller 506 implemented within the combined DDIC and ALS controller 504, respectively. Figure 3 , 4 and 5.

[0087] In step 1506, an ambient light intensity measurement is generated based on one or more readings of the light sensor. The means for generating the ambient light intensity measurement includes, for example Figure 2 . In step 1508, in response to the ambient light intensity measurement, the duty cycle for sensing ambient light within the repeating timing pattern is adjusted. The means for adjusting the duty cycle for sensing ambient light within the repeating timing pattern includes, for example, the TDD timing controller IC 308, the TDD timing control module 408 implemented within the SOC 410, and the TDD timing controller 506 implemented within the combined DDIC and ALS controller 504, respectively. Figure 3 , Figure 4 and Figure 5 shown.

[0088] Fig.1616 is a block diagram of various hardware and software components of a device 1600 employing a display and an ALS sensor capable of coordinating the emission of display content and the capture of ambient light for ALS measurements according to one aspect of the present disclosure. Examples of device 1600 may be an environmental monitoring device, a mobile handheld device, a wearable device, a vehicle, a drone, or another device that takes image data as an input. In one example, device 1600 may communicate with other devices and / or wireless communication networks via wireless transceiver 1630 and wireless antenna 1632 by sending wireless signals to or receiving wireless signals from a remote wireless transceiver over a wireless communication link, the remote wireless transceiver may include another device, a base station (e.g., a NodeB, eNodeB, or gNodeB), or a wireless access point.

[0089] Similarly, the device 1600 can send wireless signals to or receive wireless signals from a local transceiver via a wireless communication link, for example, by using a WLAN and / or PAN wireless transceiver, represented here by a wireless transceiver 1630 and one of the wireless antennas 1632. In one aspect, the wireless transceiver 1630 can include various combinations of WAN, WLAN, and / or PAN transceivers. In one aspect, the wireless transceiver 1630 can also include a Bluetooth transceiver, a ZigBee transceiver, or other PAN transceivers. In one aspect, the device 1600 can send wireless signals to or receive wireless signals from the wireless transceiver 1630 on the device 1600 via a wireless communication link 1634. The local transceiver, WAN wireless transceiver and / or mobile wireless transceiver may include a WAN transceiver, an access point (AP), a femtocell, a home base station, a small cell base station, an HNB, a HeNB or a gNodeB, and may provide access to a wireless local area network (WLAN, such as an IEEE802.11 network), a wireless personal area network (PAN, such as a Bluetooth network) or a cellular network (e.g., an LTE network or other wireless wide area network such as those discussed in the next paragraph). Of course, it should be understood that these are merely examples of networks that can communicate with the device via a wireless link, and the claimed subject matter is not limited in this regard. It should also be understood that the wireless transceiver 1630 can be located on various types of devices 1600, such as ships, ferries, cars, buses, drones, and various means of transportation. In one aspect, the device 1600 can be used for passenger transportation, package transportation, or other purposes. In one aspect, the GNSS signals 1674 from the GNSS satellites are used by the device 1600 for position determination and / or for determining GNSS signal parameters and demodulating data. In one aspect, signals 1634 from WAN transceivers, WLAN, and / or PAN local transceivers, alone or in combination with GNSS signals 1674, are used for position determination.

[0090] Examples of network technologies that can support the wireless transceiver 1630 are GSM, CDMA, WCDMA, LTE, 5G or new radio access technology (NR), HRPD, and V2X vehicle-to-vehicle communication. As previously mentioned, V2X communication protocols can be defined in various standards, such as SAE and ETS-ITS standards. GSM, WCDMA, and LTE are technologies defined by 3GPP. CDMA and HRPD are technologies defined by the Third Generation Partnership Project II (3GPP2). WCDMA is also part of the Universal Mobile Telecommunications System (UMTS) and can be supported by HNB.

[0091] The wireless transceiver 1630 can communicate with the communication network through a WAN wireless base station, which can include a deployment of equipment that provides user access to a wireless telecommunications network for a service (e.g., according to a service contract). Here, the WAN wireless base station can perform the functions of a WAN or cell base station to serve user equipment within a cell determined at least in part based on the range to which the WAN wireless base station can provide access service. Examples of WAN base stations include GSM, WCDMA, LTE, CDMA, HRPD, Wi-Fi, Bluetooth, WiMAX, 5G NR base stations. In one aspect, additional wireless base stations may include WLAN and / or PAN transceivers.

[0092] In one aspect, the device 1600 may include one or more cameras 1635, which may include an image capture device capable of capturing an extended field of view according to various aspects of the present disclosure. In one aspect, the camera may include a camera sensor and a mounting assembly. Different mounting assemblies may be used for different cameras on the device 1600. The camera may provide object detection and distance estimation, particularly for objects of known size and / or shape. When used in conjunction with other sensors, the camera may be calibrated using other systems, such as by using LIDAR, wheel scale / distance sensors, and / or GNSS to verify the distance traveled and angular orientation. The camera may be similarly used to verify and calibrate other systems to verify that the distance measurement is correct, such as by calibrating against known distances between known objects (landmarks, roadside signs, road mile markers, etc.), and also verify that object detection is accurately performed so that the object is mapped to the correct position relative to the car by the LIDAR and other systems accordingly.

[0093] In one aspect, the accelerometer, gyroscope, and magnetometer 1640 may be used to provide and / or verify motion and orientation information. LIDAR 1650 uses pulsed lasers to measure the distance to an object. While cameras may be used for object detection, LIDAR 1650 provides a unit to more definitively detect the distance (and orientation) of an object, particularly for objects of unknown size and shape. LIDAR 1650 measurements may also be used to estimate travel rate, vector direction, relative position, and stopping distance by providing accurate distance measurements and varying distance measurements.

[0094] Memory 1660 may be used with processor 1610 and / or DSP 1620 and may include random access memory (RAM), read-only memory (ROM), disk drive, FLASH or other memory devices or various combinations thereof. In one aspect, memory 1660 may include instructions for implementing the various methods described throughout this specification. In one aspect, memory may include instructions for operating and calibrating sensors and for receiving maps, weather and other data.

[0095] A global navigation satellite system (GNSS) receiver 1670 may be used to determine a position relative to the earth (absolute position), and when used with other information (e.g., measurements from other objects and / or map data), may be used to determine a position relative to other objects, such as relative to other vehicles and / or relative to the road surface. To determine position, the GNSS receiver / transceiver / receiver 1670 may receive RF signals 1674 from GNSS satellites using one or more antennas 1672 (which may be the same as antenna 1632, depending on functional requirements). The GNSS receiver / transceiver / receiver 1670 may support one or more GNSS constellations as well as other satellite-based navigation systems. For example, in one aspect, the GNSS receiver / transceiver / receiver 1670 may support global navigation satellite systems such as GPS, GLONASS, Galileo, and / or BeiDou, or any combination thereof. In one aspect, the GNSS receiver / transceiver 1670 may support regional navigation satellite systems, such as NavIC or QZSS or a combination thereof, and various augmentation systems (e.g., satellite augmentation system (SBAS) or ground augmentation system (GBAS)), such as Doppler orbit imaging and satellite integrated radio positioning (DORIS) or wide area augmentation system (WAAS) or European Geostationary Navigation Overlay Service (EGNOS) or Multifunction Satellite Augmentation System (MSAS) or Local Area Network Augmentation System (LAAS). In one aspect, the GNSS receiver / transceiver 1630 and antenna 1632 may support multiple frequency bands and sub-bands, such as GPS L1, L2 and L5 bands, Galileo E1, E5 and E6 bands, Compass (Beidou) B1, B3 and B2 bands, GLONASS G1, G2 and G3 bands, and QZSS L1C, L2C and L5-Q bands.

[0096] The GNSS receiver / transceiver 1670 may be used to determine position and relative position that may be used for positioning, navigation, and to calibrate other sensors when appropriate, such as for determining the distance between two points in time under clear sky conditions, and for calibrating other sensors (such as odometers and / or LIDAR) using distance data. In one aspect, GNSS-based relative position based on, for example, Doppler and / or pseudorange measurements shared between vehicles may be used to determine a highly accurate distance between two vehicles, and when combined with vehicle information such as shape and model information and GNSS antenna position, may be used to calibrate, validate, and / or influence the confidence level associated with information from LIDAR, cameras, radar, sonar, and other distance estimation techniques.

[0097] Radar 1653 uses transmitted radio waves reflected from objects. The reflected radio waves are analyzed based on the time taken for the reflection to arrive and other signal characteristics of the reflected waves to determine the position of nearby objects. Radar 1653 can be used to detect the position of nearby cars, roadside objects (signs, other vehicles, pedestrians, etc.), and usually will be able to detect objects even if there is blurry weather such as snow, rain or hail. Therefore, when vision-based systems usually fail, radar 1653 can be used to supplement LIDAR 1650 systems and camera 1635 systems in the following aspects: provide ranging information to other objects by providing ranging and distance measurements and information. In addition, radar 1653 can be used to calibrate and / or sanity check other systems, such as LIDAR 1650 and camera 1635.

[0098] In one aspect, the device 1600 may include a display 1656. The display 1656 may include, for example, an organic light emitting diode (OLED) panel including a plurality of OLED sub-pixels, including a display driver integrated circuit (DDIC) for providing appropriate drive signals to present display content by turning on the individual sub-pixels of the display panel for an appropriate duration at a specified color and brightness level. In one aspect, the device 1600 may include an ambient light sensor (ALS) device 1658. The ALS device 1658 may include one or more photodiodes for converting photons of light into electrical signals. The photons of light may correspond to ambient light from the environment. The ALS device 1658 may further include a low pass filter (LPF) for filtering out noise and high frequency components of the electrical signal. The ALS device 1658 may further include an analog-to-digital converter (ADC) to convert the filtered electrical signal into a digital signal. The ALS device 1658 may also include an output buffer for storing initial, intermediate, and / or final results derived from a raw, accumulated, or otherwise processed version of the digital signal as a measure of the ambient light intensity sensed by the ALS device 165 .

[0099] It will be apparent to those skilled in the art that substantial variations may be made depending on specific requirements. For example, customized hardware may also be used, and / or specific elements may be implemented in hardware, software (including portable software, such as applets, etc.), or both. In addition, connections to other computing devices such as network input / output devices may be employed.

[0100] Referring to the drawings, a memory component (eg, Fig.16The components of the memory 1660) may include non-temporary machine-readable media. The terms "machine-readable medium" and "computer-readable medium" used herein refer to any storage medium that participates in providing data that causes the machine to operate in a particular manner. In the aspects provided above, various machine-readable media may involve providing instructions / codes to a processing unit and / or other devices for execution. Additionally or alternatively, a machine-readable medium may be used to store and / or carry such instructions / codes. In many implementations, a computer-readable medium is a physical and / or tangible storage medium. This medium may take a variety of forms, including but not limited to non-volatile media, volatile media, and transmission media. Common forms of computer-readable media include, for example, magnetic and / or optical media, any other physical media with a hole pattern, RAM, PROM, EPROM, FLASH-EPROM, any other memory chip or cartridge, carrier waves described below, or any other medium from which a computer can read instructions and / or codes.

[0101] The methods, systems and devices discussed herein are examples. Various aspects may be appropriately omitted, replaced or added to various processes or components. For example, features described with respect to certain aspects may be combined in various other aspects. Different aspects and elements of these aspects may be combined in a similar manner. The various components of the drawings provided herein may be implemented with hardware and / or software. In addition, technology is developing, and therefore, many elements are examples that do not limit the scope of this disclosure to those specific examples.

[0102] It proves convenient at times, primarily for reasons of common usage, to refer to such signals as bits, information, values, elements, symbols, characters, variables, terms, digits, numbers, and the like. It will be understood, however, that all of these or similar terms are to be associated with the appropriate physical quantities and are merely convenient labels. Unless otherwise specifically stated, it will be apparent from the foregoing discussion that discussions in this specification using terms such as "process," "compute," "calculate," "determine," "ascertain," "identify," "correlate," "measure," "perform," and the like will refer to the actions or processes of a specific apparatus, such as a special purpose computer or similar special purpose electronic computing device. Thus, in the context of this specification, a special purpose computer or similar special purpose electronic computing device is capable of manipulating or transforming signals, which are typically represented as physical electronic, electrical, or magnetic quantities within a memory, register, or other information storage device, transmission device, or display device of a special purpose computer or similar special purpose electronic computing device.

[0103] The terms "and" and "or" as used herein may include various meanings, which are also expected to depend at least in part on the context in which the terms are used. The term "one or more" as used herein may be used to describe any feature, structure, or characteristic in the singular, or may be used to describe some combination of features, structures, or characteristics. However, it should be noted that this is merely an illustrative example, and the claimed subject matter is not limited to this example. In addition, if the term "at least one" is used for an association list (such as A, B, or C), it may be interpreted to mean any combination of A, B, and / or C, such as A, AB, AA, AAB, AABBCCC, etc.

[0104] Having described several aspects, various modifications, alternative configurations, and equivalents may be used without departing from the spirit of the present disclosure. For example, the above elements may be merely components of a larger system, wherein other rules may take precedence over or otherwise modify the application of various aspects. In addition, multiple steps may be performed before, during, or after considering the above elements. Therefore, the above description does not limit the scope of the present disclosure.

[0105] In view of this description, various aspects may include different combinations of features. Implementation examples are described in the following numbered clauses.

[0106] Item 1. A user device comprising: a display device including one or more light emitters forming at least a portion of the display device; a light sensor positioned such that an optical path exists between the light sensor and the one or more light emitters; and a timing controller coupled to the light sensor and the display device, the timing controller being configured to provide one or more timing signals to (1) enable the light sensor to sense ambient light during a first portion of a repeating timing pattern, and (2) enable the one or more light emitters to emit display light during a second portion of the repeating timing pattern, wherein the first portion does not substantially overlap with the second portion of the repeating timing pattern , wherein the first portion of the repeating timing pattern corresponds to a duty cycle for sensing ambient light within the repeating timing pattern, and the second portion of the repeating timing pattern corresponds to a duty cycle for emitting display light within the repeating timing pattern, wherein the timing controller is configured to adjust the duty cycle for sensing ambient light within the repeating timing pattern and the duty cycle for emitting display light within the repeating timing pattern in response to an ambient light intensity measurement based on one or more readings of the light sensor, and wherein the timing controller is configured to continue to provide the one or more timing signals after adjusting the duty cycle for sensing ambient light within the repeating timing pattern.

[0107] Clause 2. A user device according to clause 1, wherein the first portion does not substantially overlap with the second portion of the repeating timing pattern, meaning that the first portion and the second portion ensure that display light is not emitted from the one or more light emitters when the light sensor is operating to sense ambient light.

[0108] Clause 3. A user device according to any one of clauses 1-2, wherein the user device further comprises a controller for the light sensor, a controller for the light sensor, and a processor coupled to the light sensor and the display device, wherein the timing controller is implemented as a semiconductor device separate from the controller for the light sensor, the controller for the display device, and the processor.

[0109] Clause 4. A user device according to any one of clauses 1-2, wherein the user device further comprises a processor coupled to the light sensor and the display device, and the timing controller is implemented within the processor.

[0110] Clause 5. The user equipment of any one of clauses 1-2, wherein the timing controller is implemented as a combined timing controller for the light sensor and the display device.

[0111] Clause 6. A user device according to any one of clauses 1-5, wherein the one or more timing signals include (1) a first timing signal for enabling the light sensor to sense ambient light during the first portion of the repeating timing pattern and (2) a second timing signal, different from the first timing signal, for enabling the one or more light emitters to emit display light during the second portion of the repeating timing pattern.

[0112] Clause 7. A user device according to any one of clauses 1-5, wherein the one or more timing signals include a common timing signal for enabling the light sensor to sense ambient light during the first portion of the repeating timing pattern and enabling the one or more light emitters to emit display light during the second portion of the repeating timing pattern.

[0113] Clause 8. A user device according to any of clauses 1-7, further comprising one or more buffers configured to store digital values, and wherein the ambient light intensity measurement is generated based on using the buffers to store digital accumulated light intensity values ​​from the light sensor across one or more exposures of the light sensor.

[0114] Clause 9. A user device according to any one of clauses 1-7, further comprising one or more charge storage circuits, and wherein the ambient light intensity measurement is generated based on using the one or more charge storage circuits to accumulate an analog signal from the light sensor across one or more exposures of the light sensor.

[0115] Clause 10. A user device as described in any of clauses 1-9, wherein the timing controller is configured to increase a duty cycle for sensing ambient light within the repeating timing pattern in response to a decrease in the ambient light intensity measurement.

[0116] Clause 11. A user device as described in any of clauses 1-9, wherein the timing controller is configured to reduce a duty cycle for sensing ambient light within the repeating timing pattern in response to an increase in the ambient light intensity measurement.

[0117] Clause 12. A user device according to any one of clauses 1-11, wherein the repeating timing pattern includes one or more enable blanking periods during which one or more light emitters and light sensors are disabled, wherein each of the one or more enable blanking periods is located between the end of the second portion of the repeating timing pattern and the beginning of the first portion of the repeating timing pattern. Clause 13. A user device according to clause 12, wherein each of the one or more enable blanking periods is based on a display off response time associated with the one or more light emitters, wherein the display off response time is a delay from the end of the second portion of the repeating timing pattern to the time when the one or more light emitters are actually turned off.

[0118] Clause 14. A user device according to clause 13, wherein each of the one or more enable blanking periods is also based on a sensor turn-on response time associated with the light sensor, wherein the sensor turn-on response time is a delay from the start of the first portion of the repeating timing pattern to the time when the light sensor is actually turned on.

[0119] Clause 15. A user device according to any one of clauses 1-14, wherein the repeating timing pattern includes one or more enable overlap periods during which one or more optical emitters and optical sensors are enabled, wherein each of the one or more enable overlap periods is formed by an overlap of an end portion of a first portion of the repeating timing pattern and a start portion of a second portion of the repeating timing pattern.

[0120] Clause 16. A user device according to clause 15, wherein each of the one or more enabled overlap periods is based on a display turn-on response time associated with the one or more light emitters, wherein the display turn-on response time is a delay from the start of the second portion of the repeating timing pattern to the time when the one or more light emitters are actually turned on.

[0121] Clause 17. A user device according to clause 16, wherein each of the one or more enabled overlap periods is also based on a sensor off response time associated with the light sensor, wherein the sensor off response time is a delay from the end of the first portion of the repeating timing pattern to the time when the light sensor is actually turned off.

[0122] Clause 18. A user device according to any one of clauses 1-17, further comprising a processor coupled to the display device, wherein the processor is configured to: select a color mapping data structure based on the duty cycle for sensing ambient light within the repeating timing pattern; and apply a color shift using the selected color mapping data structure by mapping input display color values ​​to output display color values ​​for one or more frames of displayed content.

[0123] Clause 19. A user device according to clause 18, wherein the color mapping data structure comprises a three-dimensional (3D) lookup table (LUT). Clause 20. A user device according to any one of clauses 1-19, further comprising a processor coupled to the display device, wherein the processor is configured to: select a brightness adjustment value based on the duty cycle for sensing ambient light within the repeating timing pattern; and apply a brightness offset for one or more frames of displayed content using the brightness adjustment value.

[0124] Clause 21. A user device according to any of clauses 1-20, wherein, from the perspective of a user of the display device, the light sensor is located below the one or more light emitters.

[0125] Clause 22. A user device according to any one of clauses 1-20, wherein, from the perspective of a user of the display device, the light sensor is positioned side by side with the one or more light emitters.

[0126] Clause 23. A user device according to any one of clauses 1-22, wherein the one or more light emitters are part of a first area of ​​the display device, and the display device also includes one or more additional light emitters that are part of a second area of ​​the display device, and wherein the timing controller is further configured to provide one or more timing signals in a second repeating timing pattern to enable the light sensor to sense ambient light while enabling the one or more additional light emitters in the second area of ​​the display device to emit display light.

[0127] Item 24. A method for sensing ambient light intensity in a user device comprising a display device and a light sensor, the light sensor being positioned so that an optical path exists between the light sensor and one or more light emitters forming at least a portion of the display device, the method comprising: providing one or more timing signals to (1) enable the light sensor to sense ambient light during a first portion of a repeating timing pattern, and (2) enable one or more light emitters to emit display light during a second portion of the repeating timing pattern, wherein the first portion does not substantially overlap with the second portion of the repeating timing pattern, and wherein the first portion of the repeating timing pattern corresponds to a duty cycle for sensing ambient light within the repeating timing pattern, and the second portion of the repeating timing pattern corresponds to a duty cycle for emitting display light within the repeating timing pattern, and generating an ambient light intensity measurement based on one or more readings of the light sensor; adjusting the duty cycle for sensing ambient light within the repeating timing pattern and the duty cycle for emitting display light within the repeating timing pattern in response to the ambient light intensity measurement; and continuing to provide the one or more timing signals after adjusting the duty cycle for sensing ambient light within the repeating timing pattern.

[0128] Clause 25. A method according to clause 24, wherein the first portion does not substantially overlap with the second portion of the repeating timing pattern, meaning that the first portion and the second portion ensure that display light is not emitted from the one or more light emitters when the light sensor is operating to sense ambient light.

[0129] Clause 26. A method according to any one of clauses 24-25, wherein the one or more timing signals are provided by a semiconductor device separate from a controller for the light sensor, a controller for a display device, and a processor coupled to the light sensor and the display device.

[0130] Clause 27. The method of any of clauses 24-25, wherein the one or more timing signals are provided by a processor coupled to the light sensor and the display device.

[0131] Clause 28, the method of any one of clauses 24-25, wherein the one or more timing signals are provided by a combined timing controller for the light sensor and the display device.

[0132] Clause 29. A method according to any one of clauses 24-28, wherein the one or more timing signals include (1) a first timing signal for enabling the light sensor to sense ambient light during the first portion of the repeating timing pattern and (2) a second timing signal, different from the first timing signal, for enabling the one or more light emitters to emit display light during the second portion of the repeating timing pattern.

[0133] Clause 30. A method according to any one of clauses 24-28, wherein the one or more timing signals include a common timing signal for enabling the light sensor to sense ambient light during the first portion of the repeating timing pattern and enabling the one or more light emitters to emit display light during the second portion of the repeating timing pattern.

[0134] Clause 31. The method of any of clauses 24-30, wherein the ambient light intensity measurement is generated based on a digitally accumulated light intensity value from the light sensor across one or more exposures of the light sensor.

[0135] Clause 32. The method of any of clauses 24-30, wherein the ambient light intensity measurement is generated based on an accumulated analog signal from the light sensor across one or more exposures of the light sensor.

[0136] Clause 33. The method of any of clauses 24-32, wherein in response to a decrease in the ambient light intensity measurement, a duty cycle for sensing ambient light within the repeating timing pattern is increased.

[0137] Clause 34. The method of any of clauses 24-32, wherein in response to an increase in the ambient light intensity measurement, a duty cycle for sensing ambient light within the repeating timing pattern is reduced.

[0138] Clause 35. A method according to any one of clauses 24-34, wherein the repeating timing pattern includes one or more enable blank periods during which one or more light emitters and light sensors are disabled, wherein each of the one or more enable blank periods is located between the end of the second portion of the repeating timing pattern and the beginning of the first portion of the repeating timing pattern.

[0139] Clause 36. A method according to clause 35, wherein each of the one or more enabled blanking periods is based on a display off response time associated with the one or more light emitters, wherein the display off response time is a delay from the end of the second portion of the repeating timing pattern to the time when the one or more light emitters are actually turned off.

[0140] Clause 37. A method according to clause 36, wherein each of the one or more enable blanking periods is also based on a sensor turn-on response time associated with the light sensor, wherein the sensor turn-on response time is a delay from the start of the first portion of the repeating timing pattern to the time when the light sensor is actually turned on.

[0141] Clause 38. A method according to any one of clauses 24-37, wherein the repeating timing pattern includes one or more enable overlap periods during which one or more light emitters and light sensors are enabled, wherein each of the one or more enable overlap periods is formed by an overlap of an end portion of a first portion of the repeating timing pattern and a start portion of a second portion of the repeating timing pattern.

[0142] Clause 39. A method according to clause 38, wherein each of the one or more enable overlap periods is based on a display turn-on response time associated with the one or more light emitters, wherein the display turn-on response time is a delay from the start of the second portion of the repeating timing pattern to the time when the one or more light emitters are actually turned on.

[0143] Clause 40. A method according to clause 39, wherein each of the one or more enable overlap periods is also based on a sensor off response time associated with the optical sensor, wherein the sensor off response time is a delay from the end of the first portion of the repeating timing pattern to the time when the optical sensor is actually turned off.

[0144] Clause 41. A method according to any one of clauses 24-40, further comprising selecting a color mapping data structure based on the duty cycle for sensing ambient light within the repeating timing pattern; and applying a color shift using the selected color mapping data structure by mapping input display color values ​​to output display color values ​​for one or more frames of displayed content.

[0145] Clause 42. The method of clause 41, wherein the color mapping data structure comprises a three-dimensional (3D) lookup table (LUT). Clause 43. The method of any one of clauses 24-42, further comprising selecting a brightness adjustment value based on the duty cycle for sensing ambient light within the repeating timing pattern; and applying a brightness offset for one or more frames of displayed content using the brightness adjustment value.

[0146] Clause 44. The method of any of clauses 24-43, wherein the light sensor is located below the one or more light emitters from the perspective of a user of the display device.

[0147] Clause 45. The method of any of clauses 24-43, wherein the light sensor is positioned side-by-side with the one or more light emitters from the perspective of a user of the display device.

[0148] Clause 46. A method according to any one of clauses 24-45, wherein the one or more light emitters are part of a first area of ​​the display device, and the display device also includes one or more additional light emitters that are part of a second area of ​​the display device, and wherein the method also includes providing one or more timing signals in a second repeating timing pattern to enable the light sensor to sense ambient light while enabling the one or more additional light emitters in the second area of ​​the display device to emit display light.

[0149] Item 47. A non-transitory computer-readable medium having instructions embedded thereon that, when executed by one or more processors, cause the one or more processors to perform functions including: providing one or more timing signals to (1) enable a light sensor to sense ambient light during a first portion of a repeating timing pattern, and (2) enable one or more light emitters to emit display light during a second portion of the repeating timing pattern, wherein the light sensor is positioned such that an optical path exists between the light sensor and the one or more light emitters, and wherein the first portion of the repeating timing pattern does not substantially overlap with the second portion of the repeating timing pattern, and wherein the first portion of the repeating timing pattern corresponds to a duty cycle for sensing ambient light within the repeating timing pattern, and the second portion of the repeating timing pattern corresponds to a duty cycle for emitting display light within the repeating timing pattern; generating an ambient light intensity measurement based on one or more readings of the light sensor; adjusting the duty cycle for sensing ambient light within the repeating timing pattern in response to the ambient light intensity measurement; and continuing to provide the one or more timing signals after adjusting the duty cycle for sensing ambient light within the repeating timing pattern.

[0150] Item 48. A computerized device comprising: means for providing one or more timing signals to (1) enable a light sensor to sense ambient light during a first portion of a repeating timing pattern, and (2) enable one or more light emitters to emit display light during a second portion of the repeating timing pattern, wherein the light sensor is positioned such that an optical path exists between the light sensor and the one or more light emitters, and wherein the first portion does not substantially overlap with the second portion of the repeating timing pattern, and wherein the first portion of the repeating timing pattern corresponds to a duty cycle for sensing ambient light within the repeating timing pattern, and the second portion of the repeating timing pattern corresponds to a duty cycle for emitting display light within the repeating timing pattern, means for generating ambient light intensity measurements based on one or more readings of the light sensor, means for adjusting the duty cycle for sensing ambient light within the repeating timing pattern in response to the ambient light intensity measurements, and means for continuing to provide the one or more timing signals after adjusting the duty cycle for sensing ambient light within the repeating timing pattern.

Claims

1. A user equipment, comprising: a display device comprising one or more light emitters forming at least a portion of the display device; a light sensor positioned such that a light path exists between the light sensor and the one or more light emitters; as well as a timing controller coupled to the light sensor and the display device, the timing controller configured to provide one or more timing signals to (1) enable the light sensor to sense ambient light during a first portion of a repeating timing pattern, and (2) enable the one or more light emitters to emit display light during a second portion of the repeating timing pattern, wherein the first portion does not substantially overlap with the second portion of the repeating timing pattern, wherein the first portion of the repeating timing pattern corresponds to a duty cycle for sensing ambient light within the repeating timing pattern, and the second portion of the repeating timing pattern corresponds to a duty cycle for emitting display light within the repeating timing pattern, wherein the timing controller is configured to adjust the duty cycle for sensing ambient light within the repeating timing pattern and the duty cycle for emitting display light within the repeating timing pattern in response to an ambient light intensity measurement based on one or more readings of the light sensor, and Wherein, the timing controller is configured to continue to provide the one or more timing signals after adjusting the duty cycle for sensing ambient light within the repeating timing pattern.

2. The user equipment according to claim 1, wherein: The first portion does not substantially overlap with the second portion of the repeating timing pattern means that: The first portion and the second portion ensure that display light is not emitted from the one or more light emitters when the light sensor is operating to sense ambient light.

3. The user equipment according to claim 1, in, The user device also includes a controller for the light sensor, a controller for the light sensor, and a processor coupled to the light sensor and the display device, The timing controller is implemented as a semiconductor device separate from the controller for the optical sensor, the controller for the display device, and the processor.

4. The user equipment according to claim 1, wherein: The user device also includes a processor coupled to the light sensor and the display device, and the timing controller is implemented within the processor.

5. The user equipment according to claim 1, wherein: The timing controller is implemented as a combined timing controller for the light sensor and the display device.

6. The user equipment according to claim 1, wherein: The one or more timing signals include (1) a first timing signal for enabling the light sensor to sense ambient light during the first portion of the repeating timing pattern and (2) a second timing signal, different from the first timing signal, for enabling the one or more light emitters to emit display light during the second portion of the repeating timing pattern.

7. The user equipment according to claim 1, wherein: The one or more timing signals include a common timing signal for enabling the light sensor to sense ambient light during the first portion of the repeating timing pattern and enabling the one or more light emitters to emit display light during the second portion of the repeating timing pattern.

8. The user equipment of claim 1, further comprising one or more buffers configured to store digital values, and wherein: The ambient light intensity measurement is generated based on using the buffer to store digitally accumulated light intensity values ​​from the light sensor across one or more exposures of the light sensor.

9. The user device of claim 1 , further comprising one or more charge storage circuits, and wherein: The ambient light intensity measurement is generated based on using the one or more charge storage circuits to accumulate an analog signal from the light sensor across one or more exposures of the light sensor.

10. The user equipment according to claim 1, wherein: The timing controller is configured to increase a duty cycle for sensing ambient light within the repeating timing pattern in response to a decrease in the ambient light intensity measurement.

11. The user equipment according to claim 1, wherein: The timing controller is configured to decrease a duty cycle for sensing ambient light within the repeating timing pattern in response to an increase in the ambient light intensity measurement.

12. The user equipment according to claim 1, wherein: The repeating timing pattern includes one or more enable blank periods during which the one or more light emitters and the light sensor are disabled, wherein each of the one or more enable blank periods is located between an end of the second portion of the repeating timing pattern and a beginning of the first portion of the repeating timing pattern.

13. The user equipment according to claim 12, wherein: Each of the one or more enable blanking periods is based on a display off response time associated with the one or more light emitters, wherein the display off response time is a delay from the end of the second portion of the repeating timing pattern to the time when the one or more light emitters are actually turned off.

14. The user equipment according to claim 13, wherein: Each of the one or more enable blanking periods is also based on a sensor turn-on response time associated with the light sensor, wherein the sensor turn-on response time is a delay from the start of the first portion of the repeating timing pattern to the time when the light sensor is actually turned on.

15. The user equipment according to claim 1, wherein: The repeating timing pattern includes one or more enable overlap periods during which the one or more light emitters and the light sensor are enabled, wherein each of the one or more enable overlap periods is formed by the overlap of an end portion of the first portion of the repeating timing pattern and a start portion of the second portion of the repeating timing pattern.

16. The user equipment according to claim 15, wherein: Each of the one or more enable overlap periods is based on a display turn-on response time associated with the one or more light emitters, wherein the display turn-on response time is a delay from the start of the second portion of the repeating timing pattern to the time when the one or more light emitters are actually turned on.

17. The user equipment according to claim 16, wherein: Each of the one or more enable overlap periods is also based on a sensor off response time associated with the optical sensor, wherein the sensor off response time is a delay from the end of the first portion of the repeating timing pattern to when the optical sensor is actually turned off.

18. The user device of claim 1, further comprising a processor coupled to the display device, wherein: The processor is configured to: selecting a color mapping data structure based on the duty cycle for sensing ambient light within the repeating timing pattern; and A color offset is applied using the selected color mapping data structure by mapping input display color values ​​to output display color values ​​for one or more frames of the display content.

19. The user equipment according to claim 18, wherein: The color mapping data structure includes a three-dimensional (3D) look-up table (LUT).

20. The user device of claim 19, further comprising a processor coupled to the display device, wherein: The processor is configured to: selecting a brightness adjustment value based on the duty cycle for sensing ambient light within the repeating timing pattern; and A brightness offset is applied to one or more frames of the display content using the brightness adjustment value.

21. The user equipment according to claim 1, wherein: From the perspective of a user of the display device, the light sensor is located below the one or more light emitters.

22. The user equipment according to claim 1, wherein: From the perspective of a user of the display device, the light sensor is positioned side by side with the one or more light emitters.

23. The user equipment according to claim 1, in, The one or more light emitters are part of a first area of ​​the display device, and the display device further comprises one or more additional light emitters that are part of a second area of ​​the display device, and The timing controller is further configured to provide one or more timing signals in a second repeating timing pattern to enable the light sensor to sense ambient light and simultaneously enable the one or more additional light emitters in the second area of ​​the display device to emit display light.

24. A method of sensing ambient light intensity in a user device comprising a display device and a light sensor, the light sensor being positioned such that a light path exists between the light sensor and one or more light emitters forming at least a portion of the display device, the method comprising: providing one or more timing signals to (1) enable the light sensor to sense ambient light during a first portion of a repeating timing pattern, and (2) enable the one or more light emitters to emit display light during a second portion of the repeating timing pattern, wherein the first portion does not substantially overlap with the second portion of the repeating timing pattern, and wherein the first portion of the repeating timing pattern corresponds to a duty cycle for sensing ambient light within the repeating timing pattern and the second portion of the repeating timing pattern corresponds to a duty cycle for emitting display light within the repeating timing pattern, and generating an ambient light intensity measurement based on one or more readings from the light sensor; responsive to the ambient light intensity measurement, adjusting the duty cycle for sensing ambient light within the repeating timing pattern and the duty cycle for emitting display light within the repeating timing pattern; and The one or more timing signals are continued to be provided after adjusting the duty cycle for sensing ambient light within the repeating timing pattern.

25. The method of claim 24, wherein the first portion does not substantially overlap with the second portion of the repeating timing pattern means: The first portion and the second portion ensure that display light is not emitted from the one or more light emitters when the light sensor is operating to sense ambient light.

26. The method according to claim 24, wherein: The one or more timing signals are provided by a semiconductor device separate from a controller for the light sensor, a controller for the display device, and a processor coupled to the light sensor and the display device.

27. The method according to claim 24, wherein: The one or more timing signals are provided by a processor coupled to the light sensor and the display device.

28. The method of claim 24, wherein: The one or more timing signals are provided by a combined timing controller for the light sensor and the display device.

29. A non-transitory computer readable medium having instructions embedded thereon, which when executed by one or more processors cause the one or more processors to perform functions comprising: providing one or more timing signals to (1) enable a light sensor to sense ambient light during a first portion of a repeating timing pattern and (2) enable one or more light emitters to emit display light during a second portion of the repeating timing pattern, wherein the light sensor being positioned such that an optical path exists between the light sensor and the one or more light emitters, and wherein the first portion of the repeating timing pattern does not substantially overlap with the second portion of the repeating timing pattern, and wherein the first portion of the repeating timing pattern corresponds to a duty cycle for sensing ambient light within the repeating timing pattern and the second portion of the repeating timing pattern corresponds to a duty cycle for emitting display light within the repeating timing pattern; generating an ambient light intensity measurement based on one or more readings from the light sensor; adjusting the duty cycle for sensing ambient light within the repeating timing pattern in response to the ambient light intensity measurement; and The one or more timing signals are continued to be provided after adjusting the duty cycle for sensing ambient light within the repeating timing pattern.

30. A computerized device comprising: means for providing one or more timing signals to (1) enable a light sensor to sense ambient light during a first portion of a repeating timing pattern, and (2) enable one or more light emitters to emit display light during a second portion of the repeating timing pattern, wherein the light sensor is positioned such that an optical path exists between the light sensor and the one or more light emitters, and wherein the first portion does not substantially overlap with the second portion of the repeating timing pattern, and wherein the first portion of the repeating timing pattern corresponds to a duty cycle for sensing ambient light within the repeating timing pattern, and the second portion of the repeating timing pattern corresponds to a duty cycle for emitting display light within the repeating timing pattern; means for generating an ambient light intensity measurement based on one or more readings of the light sensor; means for adjusting the duty cycle for sensing ambient light within the repeating timing pattern in response to the ambient light intensity measurement; and Means for continuing to provide the one or more timing signals after adjusting the duty cycle for sensing ambient light within the repeating timing pattern.

Images