Electronic device with color recognition ambient light sensor

A color detection ambient light sensor with spectral sensitivity profiles and light-directing structures addresses the challenge of adjusting display settings based on ambient lighting, ensuring consistent color appearance across different environments.

DE112016000710B4Active Publication Date: 2024-11-14APPLE INC
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
DE112016000710
Authority / Receiving Office
DE · DE
Patent Type
Patents
Current Assignee / Owner
Priority Date
2015-09-30
Filing Date
2016-01-13
Publication Date
2024-11-14
Estimated Expiration
2036-01-13

AI Technical Summary

Technical Problem

Existing electronic devices struggle to accurately adjust display settings based on ambient lighting conditions, particularly in terms of color temperature, leading to undesirable color casts when moving between different lighting environments.

Method used

Incorporating a color detection ambient light sensor with an array of light detectors on a semiconductor substrate, each with specific spectral sensitivity profiles, and using light-directing structures to measure and calibrate the ambient light, allowing the device to distinguish between different light sources and adjust display settings accordingly.

Benefits of technology

The solution enables precise color measurement and adjustment of display settings to maintain consistent color appearance across varying lighting conditions, improving user experience by reducing color casts and enhancing display quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

Electronic device (10) exposed to ambient light (52) having an ambient light color, comprising: an input / output device (12) that receives input from a user; a display (14) which shows images to the user; a housing (22) in which the input / output device (12) and the display (14) are mounted, and a control circuit (16) in the housing (22) and a color recognition ambient light sensor (26) in the housing (22) with which the control circuit (16) measures an ambient light color, wherein the control circuit (16) adjusts the display (14) at least partially based on the measured ambient light color, wherein the control circuit (16) is configured to identify a light source type associated with the ambient light (52) based on sensor data from the color recognition ambient light sensor (26), to select a color conversion matrix based on the identified light source type, and to apply the color conversion matrix to the sensor data from the color recognition ambient light sensor (26) to produce a color measurement associated with the sensor data from the color recognition ambient light sensor (26).
Need to check novelty before this filing date? Find Prior Art

Description

BACKGROUND

[0001] This applies to electronic devices in general and, in particular, to light sensors for electronic devices.

[0002] Electronic devices such as laptop computers, mobile phones, and other devices are sometimes equipped with light sensors. For example, ambient light sensors can be integrated into a device to provide the device with information about current lighting conditions. Ambient light measurements can be used to control the device. For example, if bright daylight conditions are detected, an electronic device can increase the display brightness to compensate.

[0003] Ambient lighting conditions sometimes exhibit significant color variations. For example, an electronic device may be used in a cool color-temperature environment, such as shaded outdoors, or a warm color-temperature environment, such as an incandescent-lit interior. Content that appears correctly displayed on a display in one of these environments may exhibit an unpleasant color cast in the other. For example, a display correctly adjusted for an outdoor environment may appear excessively cool under incandescent light.

[0004] It would therefore be desirable that the presentation of colored images could be improved or that other appropriate measures could be taken based on the properties of the ambient lighting, such as color information about the ambient light.

[0005] US 2014 / 0132 578 A1 describes an electronic device having a display with a brightness that is adjusted based on ambient light data from one or more ambient light sensors. An ambient light sensor may include at least one silicon-based photosensor. The silicon-based photosensor may generate a corresponding raw sensor reading. Processing circuitry associated with the ambient light sensor may analyze the raw sensor reading to determine the type of light source present by comparing measurements from at least two different photosensors, by determining the color temperature of the light source, and / or by determining the modulation frequency of the light source. A compensation factor may then be selected by reference to a lookup table. The processing circuitry may calculate a compensated sensor reading based on the raw sensor reading and the selected compensation factor.The brightness of the display can be adjusted based on the compensated sensor reading calculated in this way.

[0006] US 5926 282 A describes a multispectral sensor device comprising several optoelectric transducers.

[0007] US 5 739 914 A describes a direct-reading stimulus-based colorimetric instrument having a plurality of photoelectric sensors with different spectral sensitivities.

[0008] DE 20 2011 110 082 U1 describes an optoelectronic measuring unit comprising an optical arrangement for receiving light emanating from a measuring object and a sensor arrangement. SUMMARY

[0009] The invention is defined by the independent claims. Preferred embodiments are described by the independent claims. An electronic device may be provided with a display mounted in a housing. A color-detecting ambient light sensor may measure the color of ambient light. The color-detecting ambient light sensor may be mounted in alignment with an ambient light sensor window formed in an inactive area of ​​the display or elsewhere in the housing.

[0010] The color-detecting ambient light sensor may be formed from an array of light detectors on a semiconductor substrate. Some of the detectors may have spectral sensitivity profiles that fully or partially correspond to those of the color-tuning functions. The color-detecting ambient light sensor may also include an infrared light detector.

[0011] Light-directing structures, such as a diffuser, prism sheet, diverging lens layer, or privacy film, can be used to direct light into the ambient light sensor. A sensor housing can surround the ambient light sensor and have an opening that allows ambient light to reach the ambient light sensor at a restricted viewing angle.

[0012] During device operation, the color-detecting ambient light sensor can take measurements to determine the type of light source currently being used in the vicinity of the device. For example, the color-detecting ambient light sensor can determine whether the device is exposed to an infrared or non-infrared light source and can distinguish between different types of light sources with different corresponding color temperatures. The color-detecting ambient light sensor can be calibrated by exposing the sensor to different types of light sources. During operation, the calibration data can be retrieved and used to calibrate the sensor based on the currently detected light source type. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a schematic diagram of an illustrated electronic device with an ambient light sensor according to one embodiment. Fig. 2 is a perspective view of a portion of a display of an electronic device in which an ambient light sensor has been mounted according to one embodiment. Fig. 3 is a cross-sectional side view of an illustrated light sensor exposed to ambient light according to one embodiment. Fig. 4 is a cross-sectional side view of an illustrated light diffuser for an ambient light sensor according to one embodiment. Fig. 5 is a cross-sectional side view of an illustrated inner enclosure having an opening through which light at a range of desired angular orientations may pass to an ambient light sensor according to one embodiment. Fig. 6 is a cross-sectional side view of an illustrated prism sheet for directing light into an ambient light sensor according to one embodiment. Fig. 7 is a cross-sectional side view of an illustrated microlouvered privacy film that may be placed over an ambient light sensor according to one embodiment. Fig. 8 is a cross-sectional side view of an illustrated optical layer having a pattern of diverging lenses for directing light toward an ambient light sensor according to one embodiment. Fig. 9 is a graph graphically depicting an exemplary ink transfer characteristic of an ambient light sensor window versus wavelength, according to one embodiment. Fig. 10 is a top view of an illustrated multi-channel ambient light sensor according to one embodiment. Fig. 11 is a graph graphically depicting the spectral response of each photodetector in an illustrated color detection ambient light sensor according to one embodiment. Fig. 12 is a matrix equation illustrating the relationship between the color coordinates for an array of light sources and a color conversion matrix and measured color detection ambient light sensor signals for the array of light sources, according to one embodiment. Fig. 13 is a flowchart of illustrated steps in calibrating a color detection ambient light sensor according to one embodiment. Fig. 14 is a flowchart of illustrated steps required in performing color measurements with a color detection ambient light sensor during operation of an electronic device according to one embodiment. Fig. 15 and Fig. 16 are top views of illustrated multi-channel ambient light sensors with circularly arranged photodetectors for improving the angular performance of the ambient light sensors according to one embodiment. DETAILED DESCRIPTION

[0013] An illustrated electronic device of the type provided with one or more light sensors is shown in Fig. 1. The electronic device 10 may be a computing device such as a laptop computer, a computer monitor containing an embedded computer, a tablet computer, a mobile phone, a media player, or other handheld or portable electronic device, a smaller device such as a wristwatch device, a pendant device, a headset, or a hearing aid device, a device embedded in eyeglasses or other equipment worn on a user's head, or other wearable or miniature device, a television, a computer display that does not contain an embedded computer, a gaming device, a navigation device, an embedded system such as a system in which electronic equipment with a display is mounted in a kiosk system or automobile, equipment,which implements the functionality of two or more of these devices, or other electronic equipment.

[0014] As in Fig. 1, the electronic device 10 may include control circuitry 16. The control circuitry 16 may include storage and processing circuitry to support the operation of the device 10. The storage and processing circuitry may include a storage device, e.g., a hard disk drive storage device, non-volatile memory (e.g., flash memory or other electrically programmable read-only memory configured to form a solid-state drive), volatile memory (e.g., static or dynamic random access memory), etc. The processing circuitry in the control circuitry 16 may be used to control the operation of the device 10.The processing circuits may be based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors, power management units, audio chips, application-specific integrated circuits, etc.

[0015] Input / output circuitry may be used in device 10, such as input / output devices 12, to enable data to be provided to device 10 and data from device 10 to be provided to external devices. Input / output devices 12 may include, for example, buttons, joysticks, scroll wheels, touchpads, keypads, keyboards, microphones, speakers, sound generators, inverters, cameras, light-emitting diodes and other status indicators, data ports, etc. A user may control the operation of device 10 by providing commands via input / output devices 12 and may obtain status information and other outputs from device 10 using the output resources of input / output devices 12.

[0016] Input / output devices 12 may include one or more displays, such as display 14. Display 14 may be a touchscreen display that may include a touch sensor for detecting touch input from a user, or display 14 may be touch-insensitive. A touch sensor for display 14 may be based on an array of capacitive touch-sensing electrodes, acoustic touch-sensing structures, resistive touch components, force-based touch-sensing structures, a light-based touch sensor, or other suitable touch-sensing devices.

[0017] Input / output devices 12 may also include sensors 18. Sensors 18 may include an ambient light sensor and other sensors (e.g., a capacitive proximity sensor, a light-based proximity sensor, a magnetic sensor, an accelerometer, a force sensor, a touch sensor, a temperature sensor, a pressure sensor, a compass, a microphone or other sound sensor, or other sensors).

[0018] An ambient light sensor for device 10 may include an array of detectors, each provided with a different corresponding color filter.

[0019] Information from the detectors can be used to measure the total amount of ambient light present around device 10. For example, the ambient light sensor can be used to determine whether device 10 is in a dark or bright environment. Based on this information, control circuitry 16 can adjust the display brightness for display 14 or take other appropriate action.

[0020] The array of colored detectors can also be used to make color measurements (i.e., the ambient light sensor can be a color-detecting ambient light sensor). Color measurements can be captured as color coordinates, color temperature, or correlated color temperature. Processing circuitry can be used, if desired, to convert these various types of color information into other formats (e.g., a group of color coordinates can be processed to generate an associated correlated color temperature, etc.). Configurations in which the color information captured by the ambient light sensor is a group of color coordinates are sometimes described herein as an example. However, this is merely illustrative. The color-detecting ambient light sensor can capture any suitable color information from ambient light. Overall brightness (ambient light intensity) can also be measured.

[0021] Color information from the color-detecting ambient light sensor (and / or brightness information) can be used to adjust the operation of the device 10. For example, the color cast of the display 14 can be adjusted according to the color of the ambient lighting conditions. For example, if a user moves device 10 from a cool lighting environment to a warm lighting environment (e.g., an incandescent lighting environment), the warmth of the display 14 can be increased accordingly so that the user of the device 10 does not perceive the display 14 as excessively cold. If desired, the ambient light sensor can include an infrared light sensor. Generally, based on color measurements and / or measurements of the total light intensity (e.g.,Adjusting display brightness, adjusting display content, changing audio and / or video settings, adjusting sensor measurements from other sensors, setting which screen options are presented to a user of device 10, adjusting wireless circuit settings, etc.) appropriate measures may be taken.

[0022] A perspective view of a portion of an illustrated electronic device is shown in Fig. 2. In the example of Fig. 2, the device 10 includes a display, such as a display 14, mounted within the housing 22. The housing 22, which may sometimes be referred to as a "bezel" or "encapsulation," may be formed from plastic, glass, ceramic, fiber composites, metal (e.g., stainless steel, aluminum, etc.), other suitable materials, or a combination of two or more of these materials. The housing 22 may be formed using a one-piece configuration in which some or all of the housing 22 is machined or formed as a single structure, or may be formed using multiple structures (e.g., an internal frame structure, one or more structures forming external housing surfaces, etc.).

[0023] Display 14 may be covered using a display cover layer, such as a layer of transparent glass, clear plastic, sapphire, or other transparent layer. Openings may be formed in the display cover layer. For example, an opening may be formed in the display cover layer to accommodate a button, a speaker connector, or other components. Openings may be formed in the housing 22 to form communication ports (e.g., an audio jack connector, a digital data connector, etc.), to form openings for buttons, etc.

[0024] The display 14 may include an array of display pixels formed from liquid crystal display (LCD) components, an array of electrophoretic pixels, an array of plasma pixels, an array of organic light-emitting diode or other light-emitting diode pixels, an array of electrowetting pixels, or pixels based on other display technologies. The array of pixels of display 14 forms an active area AA. The active area AA is used to display images to a user of device 10. The active area AA may be rectangular or may have other suitable shapes. The inactive edge area IA may extend along one or more edges of the active area AA. The inactive edge area IA may include circuitry, signal lines, and other structures that do not emit light to form images.To conceal the inactive circuitry and other components in the edge region IA from the view of a user of device 10, the underside of the outermost layer of display 14 (e.g., the display cover layer or another display layer) may be coated with an opaque cover material, such as a layer of black ink. Optical components (e.g., a camera, a light-based proximity sensor, an ambient light sensor, status indicator LEDs, camera flash LEDs, etc.) may be mounted beneath the inactive edge region IA. One or more openings (sometimes referred to as windows) may be formed in the opaque cover layer of the IA to accommodate the optical components. For example, a light component window, such as an ambient light sensor window, may be formed in a peripheral portion of display 14, such as region 20, in the inactive edge region IA.Ambient light from the exterior of device 10 may be measured by an ambient light sensor in device 10 after passing through region 20 and the display cover layer.

[0025] Fig. 3 is a cross-sectional side view of display 14 of Fig. 2 along line 24 and towards 25 from Fig. 2. As in Fig. As shown in Figure 3, the light sensor 26 may be mounted in alignment with window 20. Window 20 may have a circular shape, a rectangular shape, a shape with curved and / or straight edges, a circular ring shape with a central opaque area, or any other suitable shape. The light sensor 26 may be a color-detecting ambient light sensor used to measure the ambient light around device 10. As shown in Fig. As shown in Figure 3, display 14 may include an outermost layer, such as a display cover layer 30. Display cover layer 30 has an outer surface, such as surface 34. The surface normal n is perpendicular to surface 34. The rays of ambient light 28 are characterized by different angles of incidence A (measured with respect to surface normal n).

[0026] Window 20 may be formed from an opening in the opaque cover layer 28 on the inner surface 32 of the display cover layer 30 in the inactive region 1A. Layer 30 may be formed of glass, plastic, ceramic, sapphire, or other transparent materials and may be part of a display module for display 14 or a separate protective layer covering active display structures. The opening connected to window 20 may be filled with optical structures such as ambient light sensor ink 54 and light-redirecting structures 56.

[0027] Ambient light sensor ink 54 may have sufficient transparency at visible and infrared wavelengths to allow sensor 26 to operate while simultaneously enhancing the external appearance of window 20 (e.g., by partially concealing the presence of window 20 from a user of device 10 by giving window 20 a visual appearance not too dissimilar to the portion of layer 30 that includes layer 28). If desired, ambient light sensor ink 54 may be omitted.

[0028] Sensor 26 may include a plurality of light detectors 60 (e.g., photodiodes, phototransistors, or other semiconductor photodetector structures). Light detectors 60 may be formed in an array on a conventional semiconductor substrate, such as substrate 62, or using two or more substrates. Each of the light detectors 60 may be provided with a corresponding color filter 58. To provide sensor 26 with the capability of accurately measuring colors, sensor 26 may include two or more detectors 60 (e.g., 2-10 detectors, 3-8 detectors, 4-7 detectors, 5-7 detectors, only 4 detectors or more than 4 detectors, only 5 detectors or more than 5 detectors, only 6 detectors or more than 6 detectors, only 7 detectors or more than 7 detectors, only 8 detectors or more than 8 detectors, fewer than 8 detectors, or any other suitable number of detectors).Filters 58 may be thin-film interference filters and / or colored layers of polymer or other color filter elements (e.g., color filters formed from dyes and / or pigments).

[0029] Light-redirecting structures 56 may be used to collect light 52 from various incident angles A and effectively direct that light to sensor 26. Light-redirecting structures 56 may include structures such as diffusers and / or patterned lenses to assist in redirecting off-axis ambient light rays into sensor 26 at an angle nearly rectangular to the surface of substrate 62, thereby reducing the dependence of ambient light measurements on the relative alignment between device 10 and the ambient light sources.

[0030] Fig. Figure 4 illustrates how light-directing structures 56 can be formed from a layer of diffuser foil. With this type of arrangement, incident rays, such as ray 52, can be scattered within the diffuser foil to produce scattered rays 52', which are measured by sensor 26. The use of a diffuser structure, such as the diffuser of Fig. 4, helps to collect light by redirecting incoming off-axis light back toward sensor 26. The diffuser film can be a diffuser that has a Lambertian scattering profile (radiance through the diffuser independent of the angle of the incoming light) or a scattering profile that, at any angle of light incidence on the diffuser, deviates by less than 20% from a Lambertian scattering profile at that angle of light incidence (as examples).

[0031] It may be desirable to provide a recess above sensor 26 using an internally recessed sensor bezel. For example, sensor 26 may be mounted within an internal bezel, such as bezel 64, within housing 22. Bezel 64 may be formed from plastic, metal, ceramic, and / or other materials, or combinations of these materials. The opening (recess 66) may be formed in the top of bezel 64 to create a restricted viewing angle for sensor 26 within housing 22. This type of arrangement may help reduce the dependence of sensor 26's sensitivity on its orientation relative to ambient light sources while maintaining a large field of view for collecting incoming ambient light by light-directing structures 56.

[0032] As in Fig. 6, light-directing structures 56 can be formed from prism sheets. Prism sheets utilize triangular edges, pyramidal structures, or other bulges to direct incident off-axis light 52 downward toward the sensor 26 (see, e.g., redirected light beam 52' in the example of Fig. 6). Prism sheets help bend and focus light rays received over a wide range of angles of incidence. Prism sheets can have bulges on one or both sides of a transparent substrate.

[0033] If desired, light-directing structures can be incorporated into 56 privacy films. As shown in Fig. As shown in Figure 7, a privacy film may include structures such as microlouvers 70 that define narrow viewing angle light passage channels, such as channels 72. Microlouvers 70 may be formed from opaque polymers, metal, multiple layers of polymer and / or metal (e.g., double layers of metal with aligned apertures), or other opaque structures. Off-axis light rays, such as light ray 52B, strike microlouvers 70 and are blocked from reaching sensor 26. On-axis rays, such as ray 52A, pass through one of channels 72 and are detected by sensor 26. Privacy filter structures help avoid high angle light rays.

[0034] In the example of Fig. 8, light-redirecting structures 56 were formed from a layer of a one-dimensional or two-dimensional array of diverging lenses 74 (e.g., diverging lens structures on the inner surface of a transparent layer). Lenses 74 can help redirect incoming off-axis light rays, such as ray 52, toward sensor 26, as shown by the illustrated redirected light ray 52'. Diverging lens effects in this type of structure help bend light toward sensor 26, thereby reducing the angle of incidence of the light rays as they strike sensor 26.

[0035] If desired, other types of light-directing structures may be used. Furthermore, two or more different types of light-directing structures may be combined to form light-directing structures 56. For example, a diffuser may be formed as a coating on the surface of a diverging lens layer, a diffuser film may be stacked with a prism film and / or a microlouvre layer, etc.

[0036] Fig. 9 is a graph in which ink light transmission T ink was entered as a function of wavelength for an illustrated ambient light sensor ink, such as ink 54 of Fig. 3. As shown by the illustrated transmission curve 76, ink 54 may exhibit non-zero transmission at visible light (VIS) wavelengths and may optionally exhibit increasing transmission toward infrared (IR) wavelengths. The transmission of ink 54 is preferably sufficient to enable sensor 26 to capture ambient light readings in bright and dim light.

[0037] To enable sensor 26 to make color measurements, sensor 26 may include an array of light detectors 60, each having a different spectral profile for detecting light. There may be any suitable number of visible light detectors in sensor 26 (e.g., two or more, three or more, four or more, five or more, six or more, three or less, four or less, five or less, six or less, less than 10, more than 5, 2-10, 2-15, 4-9, etc.). In the example of Fig. 10, there are six visible light detectors 60 (PD1, PD2, PD3, PD4, PD5, and PD6) and one infrared light detector PD7. This is merely illustrative. For example, there may be fewer than six (e.g., five, four, or three or fewer) or more than six (e.g., seven, eight, or more than eight) visible light detectors. Infrared light detector PD7 may be omitted, or infrared light detection capabilities may be provided by extending the sensitivity of a red light detector in the long wavelength range so that the red light detector has a spectral sensitivity profile that overlaps near-infrared wavelengths. For example, PD7 may be omitted, and PD6 may be a red light detector with an extended spectral profile that is sensitive to infrared wavelengths. In this type of configuration, the IR response of PD6 can be used to help distinguish between different types of light sources (e.g.,IR or non-IR, etc.) and may provide a visible spectral profile contribution (e.g., red sensitivity) to sensor 26, which assists sensor 26 in measuring the color of the ambient light.

[0038] Illustrative spectral sensitivities for detectors 60 of Fig. 10 are in Fig. 11. As shown in Fig. As shown in Figure 11, detector PD1 can have a blue spectral response, detector PD6 can have a red spectral response, and the spectral sensitivities of detectors PD2, PD3, PD4, and PD5 can cover corresponding wavelength ranges between the blue and red ends of the visible spectrum. Detector PD7 can cover infrared wavelengths (e.g., wavelengths that include wavelengths above 700 nm, between 800-900 nm, etc.).

[0039] To improve the accuracy of color detection, it may be appropriate to configure the spectral sensitivities of detectors 60 such that one or more of the detectors 60 have a spectral sensitivity corresponding to a color matching function (e.g., one of the three CIE standard observer color matching functions x, ȳ, and z). The color matching functions represent the spectral sensitivity of a standard observer. In the exemplary configuration of Fig. 11, the spectral sensitivity of detector PD1 corresponds to the spectral shape of the color matching function z, the spectral sensitivity of detector PD3 corresponds to the spectral shape of the color matching function y, and the spectral sensitivity of detector PD4 corresponds to the upper half of the color matching function x (i.e., the upper peak of the response of two peaks for x, which lies between 500 and 700 nm). If desired, one of the detectors 60 can correspond entirely to the spectrum of x. The example of Fig. 11, in which the spectral response profile for PD4 partial color matching function x is merely illustrative.

[0040] If desired, the spectral response profile of one or more of the detectors can resemble a standard observer color matching function without actually corresponding to a standard observer color matching function. For example, a spectral response profile can be the same as that for a standard observer color matching function x, but with different wavelengths (i.e., the spectral response profile curve shape can correspond to that of the color matching function, while the wavelength position of the curve can differ by plus or minus 15 nm or some other suitable amount). As another example, the spectral response profile curve shape can be slightly different from the color matching function (at the same set of wavelengths or a shifted set of wavelengths).It may be appropriate to use one or more spectral response profiles that are within 20% root mean square deviation of a standard observer color matching function (ignoring wavelength differences between the spectral response profile and the standard observer color matching function).

[0041] Optical sensors with filters operating in the visible range (380–780 nm) may have overlapping spectral sensitivities. The shape and positions of some of the channels may (e.g., within 20% root mean square deviation) equal to standard observer color matching functions (using the same wavelengths or regardless of wavelength). The spectral sensitivities can be configured such that linear combinations of these multi-channel spectra (e.g., within 20% error or other appropriate deviation) may equal or match the standard observer color matching functions.

[0042] In addition to the spectral sensitivities of the detectors 60, which correspond to the color matching functions (or, in the case of PD4 and color matching function x, partially correspond to the color matching function due to the correspondence of the upper half of the color matching function), detectors 60 can cover other spectral ranges (e.g., ranges that partially overlap the coverage ranges of other detectors and help provide coverage across the entire visible spectrum) to thereby improve the accuracy of color measurement. At the same time, the use of an excessive number of different detectors can be avoided to avoid excessive cost, complexity, and power consumption. Readings from infrared detector PD7 can be used to improve the accuracy of visible light detection (e.g.,by removing the infrared contributions to the visible light detectors) and / or can be used to assist sensor 26 in distinguishing between different types of light sources. As an example, light sources with little or no infrared light can be characterized as non-IR sources, while light sources containing significant light detected by detector PD7 can be characterized as IR sources.

[0043] The ability to classify light sources by type can improve accuracy when calculating color coordinates for measured light sources. During the calibration process, color sensor 26 can be calibrated using different types of light sources. For example, calibration measurements can be made by training sensor 26 using multiple infrared light sources and multiple non-infrared light sources. When performing color measurements with sensor 26 while a user is using device 10, device 10 can use information about the type of ambient light source present (IR or non-IR in this example) to determine whether calibration data for the IR or non-IR sensor should be used when processing the sensor measurements. If desired, other types of sensor calibration measures can be applied (e.g.,by scanning a single narrowband reference light source across all wavelengths of interest while taking measurements with each of the detectors in sensor 26, etc.). Configurations in which sensor data are processed using light source-dependent calibration techniques are sometimes described herein as an example.

[0044] As in Fig. As shown in Figure 12, a color conversion matrix A can be used in the matrix equation P = AQ by relating the color coordinates (matrix P) to sensor outputs (sensor spectral sensitivity matrix Q). In the equation of Fig. 12, each column of matrix Q refers to the output of sensor 26 when exposed to a different corresponding light source. In the example of Fig. 12, sensor 26 has six visible light sensors, so each column of matrix Q comprises six detector outputs: PD1 to PD6. The first column of matrix Q comprises outputs PD11 to PD61, which were the result of exposing sensor 26 to light from a first light source, the second column of matrix Q comprises outputs PD12 to PD62, which were the result of exposing sensor 26 to light from a second light source, and so on. The outputs of sensor 26 were measured while sensor 26 was illuminated by m different light sources, so matrix Q comprises m columns. Matrix P comprises columns of the color coordinates X, Y, and Z. Each column of matrix P corresponds to the color coordinates of a different measured color.For example, the first column of matrix P comprises the color coordinates X1, Y1, Z1, corresponding to the color measured when sensor 26 is exposed to light from light source number 1 (resulting in outputs PD11 to PD61 of sensor 26), the second column of matrix P comprises the color coordinates X2, Y2, Z2, corresponding to the color measured when sensor 26 is exposed to light from light source number 2 (resulting in outputs PD12 to PD62 of sensor 26), and so on.

[0045] In the example of Fig. 12, matrix P is a 3 × m matrix, matrix Q is a 6 × m matrix, and matrix A is a 3 × 6 matrix. During calibration, different corresponding A matrices can be calculated for different types of light sources. Examples of different types of light sources include infrared (IR) and non-IR. The non-IR type can be divided into highly and low-correlated color temperature types (high CCT and low CCT), if appropriate. With a suitable arrangement, high CCT lighting conditions are characterized by CCT values ​​above 4500 K, and low CCT lighting conditions by CCT values ​​below 4500 K. These light types (or any other suitable light types) can each be associated with a different corresponding version of the color conversion matrix A (i.e.A different color conversion matrix A can be created for each light type to improve color measurement accuracy. Incandescent light and halogen light can be classified as IR light. Sunlight is an example of a high-CCT non-IR light source. Some fluorescent lamps and light from light-emitting diodes (LEDs) can be high-CCT non-IR light sources. Other fluorescent lamps and LED lights can be low-CCT non-IR light sources. The process of calibrating different types of light to create correspondingly different versions of the color conversion matrix A is sometimes referred to as segmented calibration.

[0046] Illustrated steps involved in the calibration of sensor 26 using a segmented calibration approach are shown in Fig. 13 shown.

[0047] At step 110, a light source is selected from a group of light sources of a particular type. For example, an incandescent or halogen light source may be selected if it is appropriate to perform calibration procedures to create a version of the color conversion matrix A that relates to IR light sources.

[0048] After selecting a light source for which calibration measurements are to be taken, a reference spectrometer can be used to perform a color measurement on the selected light source. For example, the spectrometer can measure the color coordinates X, Y, and Z for the light source (step 102). This information can be used to determine the values ​​for a column of matrix P corresponding to the selected light source.

[0049] Once the color coordinates for the currently selected light source have been acquired and stored, sensor 26 may be exposed to the light from the currently selected light source (or a simulated version of that light source generated by a tunable light-emitting diode, a tunable source based on a tunable liquid crystal filter, etc.). In particular, at step 104, sensor 26 may take a measurement on the current light source and may generate corresponding detector signals (i.e., each of the detectors PD1 to PD6 may generate an output signal). This information may be stored to represent one of the columns of matrix Q of Fig. 11 to create.

[0050] As indicated by line 106, processing may return to step 100 if it is appropriate to collect data from one or more additional light sources in the same group of light sources (i.e., additional IR light sources in the illustrated scenario in which sensor 26 is calibrated using a group of IR light sources).

[0051] Once sensor data corresponding to a representative selection of light sources in the current group has been acquired (i.e., once data from all m IR light sources have been used to fill the columns of matrices P and Q in the present example), matrix A can be calculated from matrix Q and P at step 108. By calculating the color conversion matrix A, a matrix inversion technique, such as the pseudoinverse technique, can be applied to invert the sensor spectral response matrix Q. The color conversion matrix A can then be calculated by multiplying matrix P and pseudoinverse matrix Q -1 can be calculated according to equation 1. A=PQ−1

[0052] If it is appropriate to perform further calibration procedures (e.g., to calibrate a group of non-IR high CCT type light sources or a group of non-IR low CCT type light sources), a new light type may be selected, and processing may return to step 100, as indicated by line 110. After data has been acquired for each desired light type, the values ​​of matrix A created for all desired light types may be stored in device 10 to calibrate sensor 26 and device 10 (step 112).

[0053] A device with a sensor calibrated using this type of segmented calibration procedure can be calibrated according to the flowchart of Fig. 14. At step 114, device 10 may be exposed to ambient light having a spectrum of x0. When exposed to input light x0, sensor 26 generates an output x0Q, which may be detected and stored by device 10 (e.g., via control circuits 16).

[0054] After detecting the detector output signals from sensor 16 at step 114, device 10 may determine at step 116 what type of light source is currently being used to illuminate device 10. For example, if IR detector PD7 detects that more than a predetermined amount of infrared light is present relative to the total ambient light measurement, the current lighting conditions may be characterized as falling within the IR light type. If no IR light is present (i.e., if the IR signal is less than a predetermined amount relative to the other light types), the light source may be characterized as being of the non-IR type. A correlated color temperature measurement may then be used to determine whether the non-IR light source illuminating device 10 is a high- or low-correlated color temperature light source.

[0055] After determining the type of light source currently being used to generate ambient light 52 in the vicinity of device 10 and ambient light sensor 26, control circuitry 16 may retrieve an appropriate color conversion matrix A for that light source type from memory (step 118). For example, control circuitry 16 may retrieve the IR light source version of matrix A in response to determining the present lighting conditions as being of the IR type.

[0056] At step 120, the color (i.e., the color coordinates) of the measured ambient light may be determined by multiplying the measured sensor output by the retrieved version of A. If desired, ambient light sensor color information may be captured or converted by sensor 26 to generate color temperature data, correlated color temperature data, or other color information in addition to or instead of the color coordinates. The use of sensor 26 to make color measurements, which are stored as color coordinates, is merely illustrative. All ambient light color information (and intensity information) may be captured and used by device 10, if desired.

[0057] At step 120, device 10 may apply control circuitry 16 to take appropriate actions based on the measured color of the ambient light (and, if desired, the light intensity). For example, device 10 may adjust the color of the images displayed on display 14, may make other adjustments to display 14, etc.

[0058] If desired, ambient light sensor 26 can be calibrated by scanning a tunable light source across a wavelength range of interest for calibration (e.g., the visible spectrum). The tunable light source can be a monochromator, a tunable light-emitting diode, a broadband light source with a tunable filter, such as a tunable liquid crystal filter, or another tunable light source. While the tunable light source is scanned across the wavelength range of interest, the response of sensor 26 can be tracked, and sensor response data can be processed and stored in device 10 for use in calibrating sensor 26. If desired, the tunable sensor can simulate the spectrum of one or more light sources of different types.

[0059] In the example of Fig. 10, photodiodes were arranged in rows and columns. If desired, the photodiodes can be arranged in circular patterns or other centrally symmetric patterns (i.e., in shapes having a common-center structure in which photodiodes are grouped in pairs around a common point and face each other across the central point). These common-center structures can have twofold symmetry (photodiodes face each other in pairs) or can have fourfold symmetry (first, second, third, and fourth photodiodes are arranged around a common point such that the first and second photodiodes face each other across the central point, and the third and fourth photodiodes face each other across the central point).The use of common-center arrays for the photodiodes helps reduce the effect of the angular orientation of incident ambient light on the output of sensor 26. In the illustrated configuration of . Fig. 15, sensor 26 comprises, for example, photodiodes 60 with six different spectral sensitivities (PD1, PD2, PD3, PD4, PD5, and PD6). These photodiodes may be photodiodes for visible light and / or photodiodes with spectral sensitivities extending into the infrared wavelength range. For example, each of the six photodiodes may be assigned to a corresponding one of the visible light spectra in Fig. 11 or other suitable spectra.

[0060] The photodiodes can each be divided into a pair of first and second photodiode sections on opposite sides of the central point 150, thus providing the photodiodes of sensor 36 with central symmetry. For example, photodiode PD1 is divided into a first photodiode section PD1-1 and a corresponding second photodiode section PD1-2 on the opposite side of point 150. This same pattern can be used for each of the photodiodes 60, so that each photodiode has a section on one side of point 150 and on the opposite side of point 150.

[0061] By arranging the photodiode sections in this manner, the photodiodes exhibit central symmetry and reduced angular sensitivity. Due to the scattering profile of the incident ambient light as it passes through the light-redirecting structure, angularly offset ambient light (e.g., light not parallel to the normal surface of sensor 26) will tend to increase the output of one half of each photodiode and decrease the output of the other half of that photodiode. For example, off-axis light might increase the output of the first photodiode section PD1-1 of photodiode PD1, but decrease the output of the second photodiode section PD1-2 of photodiode PD1 by a corresponding amount, so that the total output from each photodiode remains relatively independent of the angle of the incident light. Non-circular patterns can also be used to reduce angular sensitivity in split-photodiode configurations (e.g.,square designs with common centers, etc.). The use of a centrally symmetric, circular photodiode array (with a common center) in the example of . Fig. 15 is merely illustrative. The common-center photodiode array may have twofold symmetry or fourfold symmetry (as examples).

[0062] In the arrangement of Fig. 15, the light sensor processing circuits 26' were mounted on the same mold as the photodiodes 60 at a location offset from the photodiodes 60. Fig. Figure 16 is a diagram illustrating how the annular photodiode area for photodiodes 60 may surround the processing circuits 26' or other control circuits 16 (e.g., to improve the efficiency of the arrangement). Other arrangements may be used if desired.

Claims

[1] An electronic device (10) exposed to ambient light (52) having an ambient light color, comprising: an input / output device (12) that receives inputs from a user; a display (14) which shows images to the user; a housing (22) in which the input / output device (12) and the display (14) are mounted, and a control circuit (16) in the housing (22) and a color detection ambient light sensor (26) in the housing (22) with which the control circuit (16) measures an ambient light color, wherein the control circuit (16) adjusts the display (14) based at least in part on the measured ambient light color, wherein the control circuit (16) is configured to identify a light source type associated with the ambient light (52) based on sensor data from the color detection ambient light sensor (26), select a color conversion matrix based on the identified light source type, and apply the color conversion matrix to the sensor data from the color detection ambient light sensor (26) to generate a color measurement associated with the sensor data from the color detection ambient light sensor (26). [2] The electronic device (10) of claim 1, wherein the color detection ambient light sensor (26) comprises at least five light detectors (60), each having a respective spectral sensitivity profile. [3] The electronic device (10) of claim 1, wherein the color detection ambient light sensor (26) comprises a semiconductor substrate (62) and at least five light detectors (60) in the substrate (62), each having a respective spectral sensitivity profile, wherein at least one of the spectral sensitivity profiles is a standard observer color matching function and at least one of the spectral sensitivity profiles is a visible light spectral sensitivity profile that is not a standard observer color matching function. [4] The electronic device (10) of claim 1, wherein the color detection ambient light sensor (26) comprises a semiconductor substrate (62) and at least five light detectors (60) in the substrate (62), each having a respective spectral response profile, wherein at least one of the spectral response profiles is within a 20% root mean square deviation of a standard observer color matching function, without regard to wavelength differences between the at least one of the spectral response profiles and the standard observer color matching function. [5] The electronic device (10) of claim 1, wherein the color detection ambient light sensor (26) comprises a semiconductor substrate (62) and at least five light detectors (60) in the substrate (62), each having a respective spectral sensitivity profile, one of the spectral sensitivity profiles corresponding to a standard observer color matching function, one of the spectral sensitivity profiles corresponding to a z CIE standard observer color matching function, and one of the spectral sensitivity profiles corresponding to a ȳ CIE standard observer color matching function. [6] The electronic device (10) of claim 1, wherein the color-detecting ambient light sensor (26) comprises a semiconductor substrate (62) and at least five light detectors (60) in the substrate (62), each having a respective spectral response profile, wherein one of the spectral response profiles corresponds to an upper half of a x CIE standard observer color matching function, wherein a linear combination of two or more of the spectral response profiles corresponds to a standard observer color matching function, and wherein the color-detecting ambient light sensor (26) has fewer than seven visible light detectors (60), the electronic device (10) further comprising: an ambient light sensor window in the display (14) and a light deflection structure (56) in the ambient light sensor window, wherein the color detection ambient light sensor (26) receives the ambient light (52) through the ambient light sensor window and the light redirecting structure (56). [7] The electronic device (10) of claim 6, further comprising ink (54) in the ambient light sensor window. [8] The electronic device (10) of claim 6, wherein the light redirecting structure (56) comprises a structure selected from the group consisting of a diffuser sheet, a prism sheet, a privacy sheet, and a diverging lens layer. [9] The electronic device (10) of claim 6, wherein the light redirecting structure (56) comprises a diffuser having a Lambertian scattering profile. [10] The electronic device (10) of claim 6, wherein the light redirecting structure (56) comprises a diffuser having a scattering profile that differs by less than 20% from a Lambertian scattering profile at each angle of light incidence on the diffuser. [11] The electronic device (10) of claim 1, wherein the electronic device (10) is selected from the group consisting of a portable computer, a tablet computer, a mobile phone, and a wristwatch device. [12] The electronic device (10) of claim 1, wherein the color detection ambient light sensor (26) comprises light detectors (60) each having a respective spectral sensitivity profile, the light detectors (60) being arranged in a common center array, and the common center array having four-fold symmetry. [13] The electronic device (10) of claim 1, wherein the color detection ambient light sensor (26) comprises light detectors (60) each having a respective spectral sensitivity profile, the light detectors (60) being arranged in a common center array, and the common center array having two-fold symmetry. [14] The electronic device (10) of claim 1, wherein the color detection ambient light sensor (26) comprises light detectors (60) each having a respective spectral sensitivity profile, each of the light detectors (60) comprising at least a first portion and a second portion on opposite sides of a central point to reduce angular sensitivity for the color detection ambient light sensor (26).

Citation Information

Patent Citations

  • Hand-held color measuring device

    DE202011110082U1

  • Ambient Light Sensors with Infrared Compensation

    US20140132578A1

  • Colorimetric instrument

    US5739914A

  • Multispectral sensor device

    US5926282A