DEVICE AND METHOD FOR AMBIENT LIGHT MEASUREMENT USING STORED REFERENCE DATA

Abstract

Device comprising the following: a screen (12); a storage facility; an ambient light sensor (14) located behind the screen (12); and an electronic control unit (16) configured to: a first data set is received from the ambient light sensor (14), which includes measurement data from each of a plurality of optical channels; Retrieving a second data set from memory, which includes reference data based on previous measurements for the multiple optical channels, the previous measurements being for the screen (12); and to generate an ambient light measurement by using the reference data for the multiple optical channels to modify the measurement data from the multiple optical channels in the first data set, wherein the electronic control unit (16) is further configured to generate a first vector from the first data set, wherein the first vector has a number of dimensions corresponding to a number of the multiple optical channels, and wherein each value within the first vector corresponds to a measurement of one of the multiple optical channels.

Description

AREA OF DISCLOSURE

[0001] This disclosure relates to ambient light sensors. BACKGROUND

[0002] A recent trend in smartphone industrial design, fueled by consumer demand for a higher screen-to-body ratio, is to maximize screen real estate by reducing the bezel width and decluttering the remaining bezel area by removing openings for optical sensors and other holes for microphones, speakers, and / or fingerprint readers. This trend has led to the placement of many optical sensors, including ambient light sensors, behind the screen. Although ambient light sensors are capable of taking light measurements behind the screen, the screen content can often severely distort these measurements. For example, a white screen increases the amount of light reaching the ambient light sensor, causing the sensor to overestimate the ambient lux value.A system is known from CN 1 07 957 294 in which previously stored adjustment values ​​for the ambient light measurement are subtracted from the measured values ​​of the ambient light sensor. SUMMARY

[0003] The present disclosure describes a method and apparatus, in particular for improving a system of the type described in the unpublished DE 11 2019 005 100 T5, with which the distorted measured values ​​resulting from the ambient light sensor being located behind the display screen can be compensated. The strategy of the disclosure relies, at least in part, on the spectral decomposition of raw light measurements into independent sources (e.g., red, green, and blue display components of an OLED screen and ambient light). The spectral decomposition technique can, in some cases, achieve a more accurate measurement of the ambient light. This technique enables determinations such as ambient lux and correlated color temperature independent of the content displayed on the screen.

[0004] The described techniques may, for example, be performed by an electronic control unit or other hardware device. In some embodiments, the hardware device or electronic control unit may be used in combination with software to perform the actions described herein. In particular, the electronic control unit receives a first data set comprising measurements from each of a plurality of optical channels from an ambient light sensor located behind a display. In some implementations, for example, the ambient light sensor comprises eleven optical channels that may be used to measure light. Each of the eleven optical channels may be configured to filter light for a particular portion of the visible light spectrum. The electronic control unit may receive spectral data from each optical channel.In some embodiments, the electronic control unit may receive infrared data from optical channels configured for infrared detection.

[0005] Additionally, the electronic control unit can retrieve from the memory a second set of reference data containing reference data based on previous measurements for the optical channels, where the previous measurements apply to the display screen. For example, the control unit can be part of a smartphone device that also includes an ambient light sensor and an OLED ("Organic Light Emitting Diode") display screen. The OLED display screen can have a specific spectral power distribution (e.g., reference data). This reference data can have been measured at a previous time (e.g., at the factory, in a store, when the phone was first set up, or at another suitable time). In particular, each of the red, green, and blue colors of the RGB spectrum can have specific values ​​(or unit values) associated with the OLED display. The electronic control unit can retrieve these values ​​from the memory.These values ​​can be in the same format as the spectral sensitivity data. For example, if the spectral sensitivity data received from the ambient light sensor is formatted using eleven optical channels, the spectral sensitivity data for the OLED display (e.g., the reference data) can also be formatted to use eleven optical channels. Reference data for certain optical channels may be insignificant with respect to the type of display being measured. Therefore, the reference data may contain zero values ​​for one or more optical channels.

[0006] The electronic control unit may generate an ambient light measurement by using the reference data for the optical channels to modify the measurement data from the optical channels in the first data set. For example, the electronic control unit may take the spectral response data collected from each of the optical channels and subtract the spectral power distribution data for the OLED display (e.g., the reference data) from the collected spectral response data. In some embodiments, the spectral response data may be stored as unit data (i.e., without a magnitude). In these and other embodiments, the magnitude may, for example, be calculated based on the spectral response data (i.e., the data collected from the ambient light sensor). In some embodiments, the magnitude may be calculated based on the display output.

[0007] In some embodiments, the first data (e.g., spectral sensitivity data) may be stored as a vector with a number of dimensions equal to the number of optical channels. In particular, in such cases, the electronic control unit generates a first vector from the first data set, wherein the first vector has a number of dimensions equal to a number of optical channels, and wherein each value within the first vector corresponds to a measurement from one of the optical channels used in the measurement. For example, for a device with eleven optical channels, the first data includes a vector with eleven dimensions, wherein each dimension corresponds to a specific optical channel.Furthermore, in some embodiments, the electronic control unit retrieves a second vector, where each value within the second vector corresponds to a stored reference light unit measurement for a corresponding optical channel. For example, each OLED may have a specific associated light unit measurement. The light unit measurement may be spectral data for a particular light source (e.g., OLED) without a magnitude. Thus, in some embodiments, the electronic control unit generates the ambient light measurement by subtracting from the first vector (e.g., the collected spectral data) the second vector (e.g., the reference data) modified by a magnitude value of the light generated by the screen.

[0008] When the reference data is removed from the spectral response data, an ambient light estimate is generated. Based on this estimate, the electronic control unit can determine which type of light source(s) are affecting the ambient light sensor. In some embodiments, the electronic control unit retrieves from memory multiple unit vectors for known ambient light sources, where the unit vectors include one or more unit vectors for a single light source and one unit vector for multiple light sources of one type. Using the unit vectors and the light measurement, the electronic control unit identifies one or more existing ambient light sources.

[0009] For example, in some embodiments, the electronic control unit is capable of accessing a database that stores reference data for different light sources and retrieving the reference data. In some embodiments, the database may store reference data for a combination of light sources of one type. The reference data may be stored as vector data for a certain number of dimensions (e.g., one dimension per optical channel) so that the reference data may be compared to an ambient light measurement to determine which light source(s) were detected by the ambient light sensor. In some embodiments, the reference data may be stored in a different format and then formatted as needed for a certain number of dimensions. For example, the electronic control unit may retrieve reference data for one or more light sources (e.g.,an eleven-dimensional unit vector) and compare the retrieved reference data with the ambient light measurement. If the values ​​match, the electronic control unit determines that a light bulb is hitting the ambient light sensor.

[0010] In some embodiments, the electronic control unit identifies the one or more ambient light sources by generating linear equations corresponding to the optical channels, where a number of unknowns within the linear equations corresponds to the number of one or more ambient light sources. The same method can be used to identify a combination of light sources of the same type.

[0011] In some embodiments, different functions may be performed depending on the type of ambient light(s) detected. In particular, the electronic control unit may modify an output of the screen based on one or more types and an intensity of the one or more ambient light sources. For example, if only a small amount of ambient light is detected, the electronic control unit may instruct the host device (e.g., a smartphone) to adjust the screen. In some embodiments, the electronic control unit modifies a setting of a camera based on one or more types and an intensity of the one or more ambient light sources. For example, the electronic control unit may modify a setting of a flash of a camera based on the type of light to achieve a correct light temperature in an image.

[0012] It is also possible to make this modification based on the ambient light measurement itself, rather than on the types of light sources. In particular, in such cases, the electronic control unit modifies a screen output based on the ambient light measurement. In some embodiments, the electronic control unit modifies a camera setting based on the ambient light measurement. For example, the host device (e.g., a smartphone) may contain a database of various ambient light measurements and corresponding screen changes. The electronic control unit may compare the detected ambient light measurement with various ambient light measurements in the database to determine which function to perform. These functions may be camera modification options, screen modification options, and other suitable options.

[0013] The details of one or more embodiments are illustrated in the accompanying drawings and the following description. Further features and advantages will become apparent from the description and drawings, as well as from the claims. Brief description of the drawings Fig. shows an example of a host device (e.g. a smartphone). Fig. is a block diagram illustrating actions to generate an ambient light measurement. Fig. shows a diagram of the spectral power distribution for an LED light source and a diagram of the spectral power distribution for an OLED display. Fig. shows a spectral power distribution diagram combining the linear components of an LED light source and an OLED display. Fig. shows the spectral power for each optical channel of each of the red, green, and blue light components of a display's light output. Fig. illustrates possible vectors that can be used in generating an ambient light measurement. Fig. shows different possible vectors for different types of light. Fig. shows a series of vectors that contain both the vectors for a display and vectors for two light sources acting on the ambient light sensor. Fig. shows a vector equation. Fig. shows a graph of the transformed measurements. Fig. shows theoretical spectral vectors for the screen and the ambient light source. Fig. shows a measurement with the display switched on while the sensor is under the light source. Fig. shows vectors for two light sources in addition to the screen and how a measurement can be adjusted to one light source. Detailed description

[0014] As in Fig. As shown, a host device 10, such as a portable computing device (e.g., a smartphone, a personal digital assistant (PDA), a laptop, or a wearable), includes an OLED-type display 12 or other display that may be disposed directly beneath a front glass 20. An ambient light sensor (ALS) 14 is disposed directly beneath a portion of the display 12 and may detect ambient light (e.g., sunlight or other background light). The ALS 14 may also detect light generated by the display 12 itself. The ALS 14 may include one or more photodiodes or other light sensing elements, each of which is sensitive to a respective wavelength or range of wavelengths, which may be different from one another. An electronic control unit (ECU) 16 is configured to receive, process, and analyze signals from the ALS 14 and to perform certain functions (e.g.,brightness and color) of the screen 12. The ECU 16 may, for example, be a processor for the sensor hub or another processor in the portable computing device 10.

[0015] The components described above can be used to generate an ambient light measurement that is adapted to the light produced by a display behind which there is an ambient light sensor. Fig. is a block diagram illustrating an example of actions for generating an ambient light measurement. At 202, an electronic control unit receives a first data set including measurements from each of a plurality of optical channels from an ambient light sensor located behind a display. For example, the control unit 16 may receive the first data set from the ALS 14. The image sensor may include a certain number of optical channels (e.g., eleven channels). Each optical channel may be configured to generate a spectral response to a specific portion of the visible light spectrum, while filtering out other portions of the visible light spectrum and allowing other optical channels to generate a spectral response to those portions. For example, if a host device has eleven optical channels, the visible light spectrum may be divided into eleven regions.The visible light spectrum has a wavelength of approximately 380 nanometers to 750 nanometers. Thus, each optical channel can be configured to produce a spectral response to approximately 34 nanometers of visible light. In some embodiments, optical channels that produce a spectral response to infrared light can be used.

[0016] As described above, a spectral decomposition of ambient light measurements into independent sources can be performed to generate the ambient light measurement. This decomposition can be applied to the received initial data set. The principle of spectral decomposition is illustrated in the following figures. Fig. shows a spectral power distribution (“SPD”) curve 300 and a spectral power distribution curve 320. Curve 302 illustrates a spectral power distribution for a light-emitting diode (“LED”) light source. The spectral power distribution is shown for different wavelengths. Curve 320 illustrates the spectral power distribution for each of the red, green, and blue components of the light emitted by an OLED display. Curve 322 illustrates the blue component, curve 324 the green component, and curve 326 the red component. Combining the linear components of these two light sources results in a diagram as shown in Fig. Thus, the SPD diagram shows 400 in Fig. that the spectrum of both the LED and OLED light sources is a linear sum of the RGB primary color contributions of the OLED and the LED. Given sufficient spectral information, the control unit (e.g., ECU 15) can be programmed to draw significant conclusions about the relative contributions of the OLED and the ambient light.

[0017] As in Fig. As further illustrated, at 204, the ECU (e.g., ECU 16) retrieves from memory a second set of data containing reference data based on previous measurements for the plurality of channels, the previous measurements being for the display screen. The diagram 500 in Fig. For example, it illustrates the spectral power for each optical channel of each of the red, green, and blue components of the display (e.g., OLED display). This data is an example of reference data that can be retrieved by the ECU (e.g., ECU 16).

[0018] At 206 of Fig. The control unit (e.g., ECU 16) generates an ambient light measurement by using the reference data for the optical channels to modify the measurement data from the optical channels in the first data set. For example, the control unit may generate a first vector from the first data set, wherein the first vector has a number of dimensions equal to the number of optical channels, and wherein each value within the first vector corresponds to a measurement from one of the optical channels. The vector 610 in Fig. shows an example of such a vector.

[0019] The control unit (e.g., ECU 16) may retrieve a second vector corresponding to a second data set, wherein each value within the second vector corresponds to a stored reference light unit measurement for a corresponding optical channel. As in Fig. For example, as shown, the controller may retrieve vectors 602, 604, and 606, which correspond to the RGB values ​​for a display. In some embodiments, the vectors may be unit vectors and may need to be modified by an amount to obtain a true measurement of the display. The unit vectors may be modified, for example, by identifying a quantity that fits within the vector associated with the first data set.

[0020] Therefore, the control unit (e.g., ECU 16) generates the ambient light measurement by subtracting from the first vector the second vector, which is modified by a magnitude value of the light generated by the screen. This is represented by vector 608 in Fig. When the control unit subtracts vectors 602, 604, and 606 from vector 610, vector 608 corresponds to the values ​​for an ambient light measurement.

[0021] In some embodiments, the ECU (e.g., ECU 16) identifies one or more light sources acting on the sensor. Specifically, in such cases, the ECU (e.g., ECU 16) retrieves from memory a plurality of unit vectors for a plurality of known ambient light sources, wherein the unit vectors include one or more unit vectors for a single light source and a unit vector for light sources of one type. For example, diagram 700 in Fig. Different possible vectors for different light types. This example contains vectors with eleven dimensions, but vectors with other dimensions are possible. Generally, the number of dimensions corresponds to the number of optical channels available for measurements, but can also be smaller than this.

[0022] The control unit (e.g., ECU 16) uses the unit vectors and the ambient light measurement to identify one or more ambient light sources detected by the ambient light sensor. For example, if the ambient light measurement matches a unit vector for incandescent light, as shown in Figure 7 (e.g., by applying a magnitude to the unit vector), the ECU (e.g., ECU 16) determines that light from an incandescent lamp is being detected by the ALS (e.g., ALS 14).

[0023] In some embodiments, the ECU (e.g., ECU 16) may generate linear equations corresponding to the optical channels, where a number of unknowns within the linear equations corresponds to the number of one or more ambient light sources. Fig. shows a number of vectors that include both the vectors for RGB values ​​for the OLED display and vectors for two light sources acting on the ALS (e.g., ALS 14). For example, if the ambient light spectrum is unknown (e.g., if no single light source matches the ambient light measurement characteristics), there may be more than one light source. The control unit (e.g., ECU 16) can convert this data into a linear algebra problem where the number of optical channels equals the number of equations and the number of light types (including the three indicator LEDs) represents the number of unknowns. If the number of equations is equal to or greater than the number of unknowns, a unique solution can be found. However, if there are more light types than equations (e.g., a total of nine different light types), the control unit (e.g., ECU 16) does not necessarily need nine optical channels for a unique solution.This is because it can be assumed that only one or two ambient light types will have non-zero scalars (e.g., halogen, incandescent, and fluorescent lamps probably do not affect the ALS simultaneously). Furthermore, the scalars must be greater than zero (light cannot be negative), and the detected red, green, and blue components (e.g., rs, gs, and bs in Figure 6) can only be this large. As shown in . Fig. As shown, rs, gs, and bs each multiply a vector corresponding to the scalar values ​​for each optical channel for the red, green, and blue components of the screen. The required number of optical channels may also depend on the spectral sensitivity of each optical channel. It may not be necessary to distinguish between two light sources as long as they have similar spectra in the visible light range. This is especially important when determining lux is the primary concern. As used here, the term lux refers to a unit of illuminance equal to one lumen per square meter.

[0024] In some embodiments, the controller may use previously stored knowledge of the display content to improve our approximation. The vector equation 900 of Fig. can be expressed as Ax = y, which has the following Bayesian least squares solution: ((X^T)X + Λ)^-1(Λµ + X^Ty), where µ is a prior estimate of x (e.g., based on knowledge of the frame buffer or the 60 Hz display measurement) and A is evaluated depending on how much one can rely on the prior estimate (see effect when A is very small or very large and A is the mean vector and covariance matrix of a Gaussian conjugate prior). Alternatively, prior estimates can be used to constrain the acceptable values ​​of rs, gs, and bs in a restricted least squares solution.

[0025] In some embodiments, different functions may be performed in response to the type of ambient light(s) detected. In particular, the electronic control unit may modify an output of the display screen based on one or more types and an intensity of the one or more ambient light sources. For example, if only a small amount of ambient light is detected, the electronic control unit may instruct the device (e.g., a smartphone) to adjust the brightness of the screen display. In some embodiments, the electronic control unit modifies a setting of a camera based on one or more types and an intensity of the one or more ambient light sources. For example, the electronic control unit may modify a setting of a flash of a camera based on the type of light to obtain a proper light temperature in an image.

[0026] It is also possible to make these modifications based on the ambient light measurement itself, rather than based on the types of light sources. In particular, in some cases, the electronic control unit modifies a display output based on the ambient light measurement. In some embodiments, the electronic control unit modifies a camera setting based on the ambient light measurement. For example, the host device (e.g., a smartphone) may contain a database of various ambient light measurements and corresponding display screen changes. The electronic control unit may compare the acquired ambient light measurement with various ambient light measurements in the database to determine which function to perform. These functions may be camera modification options, display modification options, and / or other suitable options.

[0027] There are various methods for obtaining reference data for both the display light emissions and the identification of the ambient light source(s). A list of measured light sources can be used, as well as researched PCA-like techniques. This technique generates basis vectors that provide a good description of real light sources. For example, the SPD of almost all typical white LEDs can be almost completely described with just two components. Many formulations of the model rely on "training" the algorithm with specific known light sources.

[0028] OLED displays use only three unique LED colors; therefore, the light emitted by an OLED display can be described in a three-dimensional space (e.g., RGB). Using n photodiodes with unique spectral responses, the light can be described in an n-dimensional space. By definition, any n-dimensional space can be represented by n orthogonal vectors, which can be called basis vectors.

[0029] OLED display light can be described with three vectors orthogonal to n-3 basis vectors. The n-3 basis vectors, which are not affected by the display light, can be used to predict ambient lux. In this n-3-dimensional space, the control unit can calculate lux by taking the dot product between our n-3-dimensional measurement and the n-3-dimensional description of the light source and then scaling it with an empirically derived gain factor.

[0030] In one example, six unique photodiode channels (i.e., C, R, G, B, W, F) can be used. The red, green, and blue photodiodes in an OLED display excite these six photodiodes in a unique way: red:[0.4098, 0.3737, 0.0602, 0.0289, 0.3707, 1.9146] green:[0.4631, 0.1105, 0.2890, 0.1036, 0.1956, 0.8549] blue: [0.1334, 0.0101, 0.0272, 0.1059, 0.0592, 0.1298]

[0031] With three additional (random) vectors, an entire 6-dimensional space can be described with the following matrix. This data can be stored in memory for later processing. [[0.4098, 0.3737, 0.0602, 0.0289, 0.3707, 1.9146], [0.4631, 0.1105, 0.2890, 0.1036, 0.1956, 0.8549], [0.1334, 0.0101, 0.0272, 0.1059, 0.0592, 0.1298], [1, 0, 0, 0, 0, 0], [0, 1, 0, 0, 0, 0], [0, 0, 1, 0, 0, 0]]

[0032] Using a technique such as Gram-Schmidt or PCA, the control unit can find a linear transformation that converts a matrix from the previous step into a set of orthogonal vectors. In the transformed matrix, the first three row vectors describe the OLED display light, while the last three row vectors are orthogonal to the display light: [[5.95170206e−01, 4.52875687e−02, 1.21470132e−01, 4.72515778e−01, 2.64526569e−01,5.79414362e−01], [−1.56372394e−01, 1.36093881e−01, 3.50563304e−01, −6.43355981e−01, −8.95402375e−02, 6.42033588e−01], [−4.14934068e−01, 2.06626662e−01, −7.64134348e−01, 1.25150019e−01, 1.18155348e−01,4.14259129e−01], [6.70186406e−01, 1.19465204e−01, −4.99178066e−01, −4.92253227e−01, −1.82655562e−01, −1.08273773e−01], [−1.44492811e−17, 9.60448322e−01, 1.71080868e−01, 1.03186732e−01, −2.48524360e−03, −1.93950199e−01], [1.91309557e−15, 0.00000000e+00, 1.29214431e−03, 3.0691799e−01, −9.35242387e−01, 1.76412455e−01]]

[0033] The transformed measurements are shown in diagram 1000 of Fig. shown. After taking a raw measurement and projecting it onto a derived basis vector, the data can be analyzed for ambient light. Basis vectors 0, 1, and 2 are initially discarded because they are contaminated by the display light. Basis vectors 3, 4, and 5 can be used to predict the light type and approximate ambient lux. Figure 10 shows an example (not necessarily representative of real data) of how the magnitude of the measurement along the 3rd and 4th basis vectors can be used to determine the light type and lux number. For example, tan^-1 (x / y) gives an indication of the light type, while (x^2 + y^2)^(1 / 2) is proportional to lux.

[0034] One approach to predicting the light type is to compare a measurement vector with all known vectors of the ambient light source (or a subset of such vectors of the ambient light source). The smaller the angle between a measurement vector and a light type vector, the more likely it is that the measurement originates from that light type. As the 1020 diagram in Fig. shows, the measurement likely originates from a fluorescent tube. To predict the lux value of our measurement, the measurement vector can be projected onto the nearest ambient light vector. This technique can be applied in a two-dimensional space, as shown in Figure 10, or in more than two dimensions.

[0035] Another example of the claimed technique is in Fig. Instead of eleven-dimensional vectors as in a previous example, three-dimensional vectors can be used and represented in three-dimensional space. Instead of considering approximately three or four different light types, only two possible light types are considered for this example. Fig. illustrates a theoretical spectral vector for the screen and the ambient light source. As in Fig. As shown, the x-axis, y-axis, and z-axis each represent the output of a hypothetical 3-channel part. If a measurement is taken with the ALS with the screen on and the phone in the dark, the result would be three numbers that are represented as components of the vector s → can be displayed. If the brightness of the screen is changed, the resulting vector would be in the same direction as s →but with a different magnitude. Similarly, a measurement of a particular type of ambient light (e.g., incandescent lamp) with the screen off would yield three numbers that are the components of the vector L → _1 (e.g. as shown in Fig. shown). If the light source is moved to another location, a vector in the same direction as L → _1, but with a different size, can be recognized.

[0036] Fig. shows a measurement with the display switched on while the sensor is under the light source. It can be expressed as a sum of m → =ℓL → _1+sS → where ℓ and s are scalars and L → _1 and S →now represent unit vectors pointing in the direction of the light source and screen measurements. The sum can be visualized (as shown in FIGURE 12) by placing the vectors head to head. If a measurement was taken with the screen switched on and under the light source, the measurement is a vector. It can be represented as the sum m → =ℓL →­ _1+sS → where ℓ and s are scalars and L → _1 and S → Now represent unit vectors pointing in the direction of the light source and screen measurements. Here, too, the sum can be visualized by placing the vectors head to head.

[0037] Fig. shows vectors for two light sources in addition to the screen and how a measurement can be adjusted to a light source. The type of light can be determined by assuming that the host device is only in the presence of one light source. In particular, ℓL→ _2+sS → not a good approximation of the measurement for any value of ℓ and s. However, ℓL → _1+sS → a good approximation of the measurement vector for suitable values ​​of ℓ and s. Therefore, the device is probably under light type 1. With ℓ and our information about light type 1, the control unit (e.g. ECU 16) can estimate the lux without the contribution of the display.

[0038] The design of smartphones and other host computing devices referred to in this disclosure may include one or more processors, one or more memories (e.g., RAM), storage (e.g., a hard drive or flash memory), a user interface (which may include, for example, a keyboard, a TFT-LCD or OLED screen, touch or other gesture sensors, a camera or other optical sensor, a compass sensor, a 3D magnetometer, a 3-axis accelerometer, a 3-axis gyroscope, one or more microphones, etc., along with software instructions to provide a graphical user interface), connections between these elements (e.g., buses), and an interface for communicating with other devices (which may be wireless, such as GSM, 3G, 4G, CDMA, WiFi, WiMax, Zigbee, or Bluetooth, and / or wired, such as via a local Ethernet network, a T-1 internet connection etc.).

[0039] Various aspects of the subject matter and the functional operations described in this disclosure may be implemented in digital electronic circuits or in software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more thereof. The electronic control unit includes a digital control circuit configured to perform the actions required to generate an ambient light measurement. In some embodiments, the electronic control unit may include one or more of software, firmware, or other hardware to facilitate the actions of this disclosure. Furthermore, aspects of the subject matter described in this disclosure may be implemented as one or more computer program products, i.e.as one or more modules of computer program instructions encoded on a computer-readable medium for execution by, or to control the operation of, a data processing device. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable transmitted signal, or a combination of one or more thereof. The device may include, in addition to hardware, code that provides an execution environment for the computer program in question, e.g., code representing processor firmware.

[0040] A computer program (also called a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program is not necessarily a file on a file system. A program may be stored in part of a file that also contains other programs or data (for example, one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in several coordinated files (for example, files that store one or more modules, subroutines, or pieces of code).A computer program may be provided to run on one computer or on multiple computers located at one location or distributed across multiple locations and interconnected by communication.

[0041] The processes and logic flows described in this specification may be executed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating outputs. The processes and logic flows may also be executed by a special-purpose logic circuit, such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit), and the device may also be implemented as such.

[0042] Processors suitable for executing a computer program include, for example, both general-purpose and special-purpose microprocessors, as well as one or more processors of any type of digital computer. Generally, a processor receives instructions and data from read-only memory or random access memory, or both. The essential elements of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, by way of example, semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks, and CD-ROM and DVD-ROM disks.The processor and memory can be supplemented by or integrated into special logic circuits.

[0043] Although this specification contains many specifics, these should not be construed as limitations on the scope of the invention or what may be claimed, but rather as descriptions of features specific to particular embodiments of the invention. Certain features described in this specification in connection with individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in connection with a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination.Furthermore, although features are described above as operating in certain combinations and even originally claimed as such, in some cases one or more features from a claimed combination may be dropped from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0044] Although the operations in the drawings are shown in a particular order, this does not imply that these operations must be performed in the order shown or in sequential order, or that all of the operations shown must be performed to achieve the desired results. Multitasking and parallel processing may be advantageous under certain circumstances.

Claims

[1] Device comprising: a screen (12); a memory; an ambient light sensor (14) arranged behind the screen (12); and an electronic control unit (16) configured to: receive a first data set from the ambient light sensor (14) comprising measurement data from each of a plurality of optical channels; Retrieving a second data set from the memory comprising reference data based on previous measurements for the plurality of optical channels, the previous measurements being for the screen (12); and to generate an ambient light measurement by using the reference data for the multiple optical channels to modify the measurement data from the multiple optical channels in the first data set, wherein the electronic control unit (16) is further configured to generate a first vector from the first data set, the first vector having a number of dimensions corresponding to a number of the plurality of optical channels, and each value within the first vector corresponding to a measurement from one optical channel of the plurality of optical channels. [2] The apparatus of claim 1, wherein the electronic control unit (16) is operable to retrieve the second data set including a second vector, and wherein each value within the second vector corresponds to a stored reference light unit measurement. [3] The apparatus of claim 2, wherein the electronic control unit (16) is operable to generate the ambient light measurement by performing actions including subtracting the second vector, modified by a magnitude value of the light generated by the screen (12), from the first vector. [4] The device according to any one of claims 1 to 3, wherein the electronic control unit (16) is further configured to: Retrieving from the memory a plurality of unit vectors for a plurality of known ambient light sources, the plurality of unit vectors comprising one or more unit vectors for a single light source and one unit vector for a plurality of light sources of one type; and to identify one or more ambient light sources detected by the ambient light sensor (14) using the plurality of unit vectors and the ambient light measurement. [5] The apparatus of claim 4, wherein the electronic control unit (16) is operable to identify the one or more ambient light sources by performing actions including generating a plurality of linear equations corresponding to the plurality of optical channels, wherein a number of unknowns within the plurality of linear equations corresponds to the number of the one or more ambient light sources. [6] The apparatus of claim 5, wherein the electronic control unit (16) is further configured to control an output of the display screen (12) based on one or more of a type of the one or more ambient light sources and a detected signal from the one or more light sources modified. [7] The apparatus of claim 5, wherein the apparatus further comprises a camera, and wherein the electronic control unit (16) is further configured to modify a setting of the camera based on one or more of a type of the one or more ambient light sources and a magnitude of a detected signal from the one or more light sources. [8] The device of any of claims 1-7, wherein the electronic control unit (16) is further configured to modify an output of the display screen (12) based on the ambient light measurement. [9] The apparatus of any one of claims 1 to 8, wherein the apparatus further comprises a camera, and wherein the electronic control unit (16) is further configured to modify a setting of a camera based on the ambient light measurement. [10] Procedure comprising: Receiving a first data set comprising measurements from each of a plurality of optical channels from an ambient light sensor (14) located behind a display screen (12); retrieving a second data set from a memory comprising reference data based on previous measurements for the plurality of optical channels, the previous measurements being for the screen (12); Generating an ambient light measurement using the reference data for the plurality of optical channels to modify the measurement data from the plurality of optical channels in the first data set, and. Generating a first vector from the first data set, the first vector having a number of dimensions corresponding to a number of the plurality of optical channels, and each value within the first vector corresponding to a measurement from one optical channel of the plurality of optical channels. [11] The method of claim 10, wherein retrieving the second data set comprises retrieving a second vector, and wherein each value within the second vector corresponds to a stored reference light unit measurement for a respective optical channel. [12] The method of claim 11, wherein generating the ambient light measurement comprises subtracting the second vector modified by a magnitude of the light generated by the screen (12) from the first vector. [13] A method according to any one of claims 10 to 12, further comprising: Retrieving from the memory a plurality of unit vectors for a plurality of known ambient light sources, the plurality of unit vectors comprising one or more unit vectors for a single light source and one unit vector for a plurality of light sources of one type; and Identifying one or more ambient light sources detected by the ambient light sensor (12) using the plurality of unit vectors and the ambient light measurement. [14] The method of claim 13, wherein identifying the one or more ambient light sources comprises generating a plurality of linear equations corresponding to the plurality of optical channels, wherein a number of unknowns within the plurality of linear equations corresponds to the number of the one or more ambient light sources. [15] The method of claim 14, further comprising modifying an output of the display (12) based on one or more types of the one or more ambient light sources and a magnitude of a detected signal from the one or more light sources. [16] The method of claim 14, further comprising modifying a setting of a camera based on one or more types of the one or more ambient light sources and a magnitude of a detected signal from the one or more light sources. [17] The method of any of claims 10-16, further comprising modifying an output of the display (12) based on the ambient light measurement. [18] The method of any of claims 10-17, further comprising changing a setting of a camera based on the ambient light measurement.

Images