Adaptive light source

Abstract

To provide methods and systems using an adaptive light source.SOLUTION: A method includes capturing a first image of a scene, detecting a face in a section of the scene from the first image, and activating an infrared (IR) light source to selectively illuminate the section of the scene with IR light. The IR light source includes an array of IR light emitting diodes (LEDs). The method includes capturing a second image of the scene under selective IR lighting from the IR light source, detecting the face in the second image, and identifying a person on the basis of the face in the second image.SELECTED DRAWING: Figure 1B

Description

Technical Field

[0001] Embodiments of the present disclosure relate to adaptive light sources.

Background Art

[0002] Among the currently available most efficient light sources, there are semiconductor light emitting devices including light emitting diodes (LEDs), resonant cavity light emitting diodes (RCLEDs), vertical cavity surface emitting lasers (VCSELs), and edge emitting lasers. Material systems currently of interest for the manufacture of high brightness light emitting devices operable in the visible spectrum include III-V semiconductors, in particular binary, ternary, and quaternary alloys of gallium, aluminum, indium, and nitrogen, also referred to as group III nitride materials. Typically, group III nitride light emitting devices are epitaxially grown on substrates such as sapphire, silicon carbide, group III nitride or composite substrates, or other suitable substrates, by metalorganic chemical vapor deposition (MOCVD), molecular beam epitaxy (MBE), or other epitaxial techniques, to form a stack of multiple semiconductor layers of different compositions and dopant concentrations. The stack often includes one or more n-type layers, for example doped with Si, formed on the substrate, one or more light emitting layers within the active region formed on the one or more n-type layers, and one or more p-type layers, for example doped with Mg, formed on the active region. Electrical contacts are formed on these n-type and p-type regions.

[0003] Due to small size and low power requirements, semiconductor light emitting devices are attractive candidates for light sources for portable battery-powered devices such as cameras and mobile phones, such as camera flashes.

Summary of the Invention

[0004] According to an embodiment of the present invention, there is provided a light source that can be used, for example, as a flash for a camera or for other suitable applications. This light source is configured such that the illumination pattern emitted by the light source can be changed. For example, when used as a camera flash, for a given scene within the field of view of the camera, this light source can supply more light to portions of the scene that are not well illuminated by ambient light and less light to portions of the scene that are well illuminated by ambient light.

[0005] In an example of the present disclosure, the method includes capturing a first image of a scene, detecting a face within a section of the scene from the first image, and activating an infrared (IR) light source to selectively illuminate the section of the scene with IR light. The IR light source includes an array of IR light-emitting diodes (LEDs). The method includes capturing a second image of the scene under selective IR illumination from the IR light source, detecting the face within the second image, and identifying a person based on the face within the second image.

Brief Description of the Drawings

[0006]

Figure 1A

Figure 1B

Figure 1C

Figure 2A

Figure 2B

Figure 2C

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10A

Figure 10B

Figure 11A

Figure 11B

Figure 12A

Figure 12B

Figure 13A

Figure 13B

Figure 14A

Figure 14B

Figure 15A

Figure 15B

Figure 16

Figure 17A

Figure 17B

Figure 18A

Figure 18B

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

[0007] In the following description, embodiments of the present invention will be described as a camera flash, but other applications are contemplated and are within the scope of the present invention.

[0008] One problem associated with all camera flashes is that subjects near the camera often become overexposed while subjects far from the camera do not receive enough light. Embodiments of the present invention include a light source such as a camera flash for, for example, a portable device or a battery-powered device, or for a larger non-battery-powered photographic studio flash. A light source according to an embodiment of the present invention can adapt its illumination profile to the scene and deliver an appropriate amount of light to all subjects on the scene. An adaptive illumination system according to an embodiment of the present invention may include a semiconductor light source such as a semiconductor light emitting device, but any suitable light may be used.

[0009] FIG. 1A shows an example of an adaptive lighting system 1 according to an embodiment of the present invention. The system 1 can be a smartphone, a tablet computer, a laptop computer, a desktop computer, a computer monitor, a digital camera, or any suitable device. The system 1 can also be a camera module for any suitable device. The system 1 includes a light source 10 connected to a driver 12. The driver 12 supplies power to the light source 10 as will be described later. The light source 10 can be an array of one or more infrared (IR) light emitting diodes (LEDs). The driver 12 is connected to a microprocessor 14. The microprocessor 14 can include an application processor and a baseband processor, which can be connected to a radio frequency (RF) transceiver 13 to make and receive calls. A non-volatile memory 15 stores applications such as, for example, a face detection / recognition application 17, and the microprocessor 14 loads those applications into a volatile memory 19 for execution. The microprocessor 14 receives inputs from an input device 18 and a camera 11. The system 1 can also include a 3D sensor 16. The input device 18 can be a user-activated input device such as, for example, a button that the user presses to take a photo. The input device 18 may not require user input in some embodiments, such as when a photo is automatically taken. The input device 18 may be omitted in some embodiments.

[0010] The 3D sensor 16 can be any suitable sensor capable of creating a 3D profile of a scene prior to taking a photograph. In some embodiments, the 3D sensor 16 can be a time-of-flight (ToF) camera. The ToF camera measures the time it takes for light reflected from a subject to return to the ToF camera. This time can be used to calculate the distance to each subject within the scene. In some embodiments, the 3D sensor 16 can be a structured light sensor. The structured light sensor includes a projection device that projects a specially designed pattern of light onto the scene. The camera also included in the structured light sensor measures the position of each part of the light pattern reflected from the subjects in the scene and determines the distance to those subjects by triangulation. In some embodiments, the 3D sensor 16 can be one or more auxiliary cameras positioned at a distance from each other within the body of the device. By comparing the positions of the subjects seen by the auxiliary cameras, the distance to each subject can be determined by triangulation. In some embodiments, the 3D sensor 16 is the autofocus signal of the main camera within the device. While scanning the focal position of the camera lens, the system can detect which part of the scene is in focus at those positions. Then, by converting the corresponding lens position to the distance to the subject in focus at those positions, a 3D profile of the scene is constructed. A suitable autofocus signal can be obtained by conventional methods, for example, by measuring contrast or by utilizing a phase detection sensor within the camera sensor. When a phase detection sensor is used, in some embodiments, the position of each individual phase detection sensor can correspond to the area illuminated by a separate segment of the light source 10 so that the adaptive flash functions optimally, as will be described later.

[0011] Figure 1B is a flowchart of a method 300B of using the system 1 (Figure 1A) in some embodiments of the present disclosure. The method 300B is implemented by the microprocessor 14 loading the instructions of the application 17 from the non-volatile memory 15 into the volatile memory 19 and then executing those instructions. The method 300B, and any method described herein, may include one or more processes, functions, or actions illustrated by one or more blocks. Although those blocks are illustrated in order, those blocks may also be executed in parallel and / or in an order different from that described herein. Also, depending on the desired implementation, various blocks may be combined into fewer blocks, divided into additional blocks, and / or eliminated. The method 300B may begin at block 304.

[0012] At block 304, the microprocessor 14 causes the camera 11 to capture a first image of a scene (corresponding to the field of view of the camera). Block 304 may be followed by block 306.

[0013] At block 306, the microprocessor 14 detects a face within a section of the scene from the first image. The microprocessor 14 may use a face detection algorithm, such as a face detector within the Open Source Computer Vision (OpenCV) library, to locate the face within the scene. Block 306 may be followed by block 318.

[0014] At block 318, the microprocessor 14 activates the light source 10 to selectively illuminate the section of the scene with IR light. For example, the microprocessor 14 may select a group of IR LEDs within the light source 10 based on the position of the face within the scene (the section of the scene) and then cause the driver 12 to drive only the selected IR LEDs. Block 318 may be followed by block 320.

[0015] At block 320, the microprocessor 14 causes the camera 11 to capture a second image of the scene under the selective IR illumination generated at block 318. Block 320 may be followed by block 322.

[0016] At block 322, the microprocessor 14 detects the face in the second image. The face in the second image may have optimal exposure. Block 322 may be followed by block 324.

[0017] At block 324, the microprocessor 14 identifies or matches a person based on the face in the second image. The microprocessor 14 may use a face recognition algorithm, such as a face recognizer in the OpenCV library, for example, to identify or match a person based on the face in the second image. Method 300B may end at block 324.

[0018] Figure 1C is a flowchart of a method 300C of using the system 1 (Figure 1A) in some embodiments of the present disclosure. Method 300C may be a variation of method 300B (Figure 1B). Method 300C may start at block 302.

[0019] At block 302, the microprocessor 14 activates the light source 10 to uniformly illuminate the scene with IR light. For example, the microprocessor 14 causes the driver 12 to drive each of the IR LEDs in the light source 10 with a first individual current. Block 302 may be followed by block 304.

[0020] At block 304, the microprocessor 14 causes the camera 11 to capture a first image of the scene (corresponding to the field of view of the camera). Block 304 may be followed by block 306.

[0021] At block 306, the microprocessor 14 detects a face in a section of the scene from the first image. Block 306 may be followed by block 318.

[0022] At block 318, the microprocessor 14 activates the light source 10 to selectively illuminate the above section of the scene with IR light. For example, based on the position of the face within the scene (the above section of the scene), the microprocessor 14 selects a group of IR LEDs within the light source 10 and causes the driver 12 to drive each of the selected IR LEDs with a second individual current. The second individual current can be made larger than the first individual current, and the sum of the second individual currents (“second total current”) used to drive the light source 10 can be equal to the sum of the first individual currents (“first total current”) used to drive the light source 10. In other examples, the second individual current can be made smaller than the first individual current, and the second total current can be made smaller than the first total current. Block 318 can be followed by block 320.

[0023] At block 320, the microprocessor 14 causes the camera 11 to capture a second image of the scene under the selective IR illumination generated at block 318. Block 320 can be followed by block 322.

[0024] At block 322, the microprocessor 14 detects the above face within the second image. The face within the second image can have optimal exposure. For example, when the face within the scene is under-exposed in the first image, the face within the second image is illuminated with more IR light from the selected IR LEDs (aimed at that face and driven with a larger current without increasing the total current usage). In another example, when the face within the scene is over-exposed in the first image, the face within the second image is illuminated with less IR light from the selected IR LEDs (aimed at that face and driven with a smaller current while reducing the total current usage). Block 322 can be followed by block 324.

[0025] At block 324, the microprocessor 14 identifies or matches a person based on the face in the second image. Method 300C may end at block 324.

[0026] FIG. 2A is a flowchart of a method 200A of using the system 1 (FIG. 1A) in some embodiments of the present disclosure. Method 200A may be a variation of method 300B (FIG. 1B). Method 200A may start at block 202.

[0027] At block 202, an input is generated, such as an instruction to take a photo. At block 204, the camera 11 takes a first image of the scene (corresponding to the camera's field of view) with the flash (light source 10) turned off. At block 206, a face is detected within a section of the scene from the first image. At block 210, the light source 10 is turned on in a low light output mode (typically referred to as "torch mode"). At this time, the illumination profile of the light source 10 is kept uniform. "Uniform" means that all parts of the scene are illuminated with a known illumination profile. While the light source 10 remains on with a uniform illumination profile and low luminance at block 212, an intermediate image is captured. At block 214, the face is detected within the section of the scene from the intermediate image. At block 216A, the system (e.g., the microprocessor 14) calculates the optimal luminance for all parts of the face (or the entire scene) to achieve optimal exposure. This can be done by subtracting the pixel luminance values of the face (or scene) in the first image from the respective pixel luminance values of the face (or scene) in the intermediate image and scaling the difference to achieve the optimal exposure level. At block 218, the light source 10 is operated according to the illumination profile calculated at block 216A. While the light source 10 is being operated according to the illumination profile calculated at block 216A at block 220, a second image is taken by the camera 11. At block 222, the face is detected within the second image. At block 224, a person is identified or matched based on the face in the second image.

[0028] Figure 2B is a flowchart of a method 200B of using the system 1 (FIG. 1A) in some embodiments of the present disclosure. The method 200B may be a variation of the method 300B (FIG. 1B). The method 200B may start at block 202.

[0029] At block 202, an input is generated, such as an instruction to take a photo. At block 204, the camera 11 takes a first image of the scene (corresponding to the camera's field of view) with the flash (light source 10) turned off. At block 206, a face is detected within a section of the scene from the first image. At block 208, a 3D profile of the face (or the entire scene) is generated. For example, the 3D sensor 16 may generate a 3D profile of the face (or the scene), or alternatively, the 3D sensor 16 may detect data regarding the face (or the scene) and transmit the data to the microprocessor 14, and the microprocessor 14 may generate a 3D profile of the face (or the scene). At block 216B, the system (e.g., the microprocessor 14) calculates an optimal luminance for all parts of the face (or the scene) to achieve optimal exposure based on the 3D profile. For example, the microprocessor may determine to provide additional IR illumination to parts of the face (or the scene) that are far from the camera 11. At block 218, based on the calculations performed at block 216B, the scene is illuminated by the light source 10. At block 220, while the light source 10 is operating according to the illuminance profile calculated at block 216B, a second image is taken by the camera 11. At block 222, the face is detected within the second image. At block 224, a person is identified or verified based on the face within the second image.

[0030] Figure 2C is a flowchart of a method 200C of using the system 1 (FIG. 1A) in some embodiments of the present disclosure. The method 200C may be a variation of the method 300B (FIG. 1B). The method 200C may start at block 202.

[0031] At block 202, an input is generated to, for example, instruct taking a photo. At block 204, camera 11 takes a first image of the scene (corresponding to the field of view of the camera) with the flash (light source 10) turned off. At block 206, a face is detected within a section of the scene from the first image. At block 208, a 3D profile of the face (or the entire scene) is generated. At block 210, light source 10 is turned on in a low light output mode (typically referred to as "torch mode"). At this time, the illuminance profile of light source 10 is kept uniform. "Uniform" means that all parts of the scene are illuminated. At block 212, with light source 10 in torch mode, an intermediate image is captured. At block 214, the face is detected within the above section of the scene from the intermediate image. At block 216C, the system (e.g., microprocessor 14) calculates the optimal luminance for all parts of the face (or scene) to achieve optimal exposure based on the input of the captured first image and intermediate image and the 3D profile as described in the text regarding FIGS. 2A and 2B. At block 218, based on the calculation performed at block 216C, the scene is illuminated by light source 10. At block 220, while light source 10 is operating according to the illuminance profile calculated at block 216C, a second image is taken by camera 11. At block 222, the face is detected within the second image. At block 224, a person is identified or verified based on the face within the second image.

[0032] In each of FIGS. 2A, 2B, and 2C, the input can be, for example, a user input such as the user pressing a button, an input generated by the microprocessor 14 (e.g., when the microprocessor 14 is programmed to take a photo at a predetermined time or at predetermined intervals), or other suitable input. FIG. 3 illustrates a scene that will be captured in the photo when the input is generated. The scene shown in FIG. 3 includes a first person 30 in the foreground and a second person 32 in the background. This scene is merely selected for illustrative purposes. Other scenes having a plurality of objects or people at various distances from the camera are also suitable for use in the present invention.

[0033] FIG. 4 illustrates a 3D profile related to the scene shown in FIG. 3. In FIG. 4, lighter color tones correspond to shorter distances from the camera, and darker color tones correspond to greater distances from the camera. Thus, the person 30 in the foreground has the lightest color tone, indicating that the person 30 is closest to the camera. The person 32 in the background has a darker color tone, indicating that the person 32 is farther from the camera. The background is black, indicating the greatest distance from the camera.

[0034] A subject located far from the flash may receive a higher light intensity, and a subject located close to the flash may receive less light. As is well known, the illuminance of light follows the inverse square law of distance (illuminance ~ 1 / distance 2) decreases according to. Therefore, using the 3D profile of the scene, the required amount of light to be distributed to each part of the scene can be calculated. The algorithm for calculating the required intensity profile also takes into account the illuminance received by each subject in the scene, which is the information collected in the capture of the first image, from the ambient light, and adjusts the amount of flash light accordingly. For example, subjects 30 that are already well illuminated (e.g., because they are of bright colors or reflective) may receive less light than that which can be calculated based only on the distance from the light source determined by the 3D profile, and subjects 32 that are not well illuminated (e.g., because they are dark or non-reflective) may receive more light than that which can be calculated above.

[0035] Digital cameras and their image processors typically include a face recognition algorithm. In some embodiments, using the information from the face recognition algorithm, the face can be better illuminated compared to other subjects. If there is not enough light to properly expose the entire photo, the face receives the benefit of more light. If a person is too close and there is a risk of overexposure, this function should be turned off so that more light is not directed at the face. In some embodiments, the calculation of the relative light from the 3D profile may reduce the amount of light sent towards a person's eyes so as to minimize "red-eye" in the photo.

[0036] In some embodiments, the calculation of the relative light from the 3D profile may identify parts of the scene that are very far from the flash and cannot be properly illuminated (e.g., faces within the background). To maximize the amount of light sent towards the beneficial parts of the scene (e.g., faces within the foreground), and thus provide a better use of the available drive current capacity, those parts of the scene receive a minimal amount of light.

[0037] In some embodiments, a user interface (e.g., a touch screen of a smartphone) may enable a user to control the relative amount of light sent to each part of a scene. For example, the user can turn the flash's adaptive function on and off, turn various parts of the algorithm (described above) used to calculate relative light from a 3D profile on and off, and also create a flash accent on the scene manually.

[0038] Several lighting modes are contemplated by embodiments of the present invention.

[0039] In some embodiments, in a first group of lighting modes, the illumination from light source 10 is distributed across the scene so as to achieve a useful photograph that is very evenly lit. In particular, in some embodiments, when overexposure is minimized and the foreground is well lit by ambient light, all of the light from light source 10 is directed towards the background. In some embodiments, light source 10 functions as a fill-in flash and when the background is well lit by ambient light, all of the light from light source 10 is directed towards the foreground. In some embodiments, when the foreground and background are evenly lit by ambient illumination, most of the light from light source 10 is sent to the background. In some embodiments, in the case of a dark foreground, the light from light source 10 illuminates the foreground only enough for a good photograph and the remainder of the light from light source 10 is sent to the background.

[0040] In some embodiments, in the second group of lighting modes, the selected subject is illuminated. In particular, in some embodiments, in combination with face recognition, the face can be weighted most highly for optimal lighting. In some embodiments, in combination with face recognition, for example, to enhance the contrast between the illuminated face and the background closest to the face, the background around the face (or other subject) can receive less light. In some embodiments, the selected zone of the scene is identified, for example, by user input. The light from the light source 10 may be directed only within the selected zone. Examples of the selected zone include the zoomed-in image or the portion of the scene identified in another way. In some embodiments, for example, in a photo of a business card, the light from the light source 10 is emitted with a very high degree of uniformity.

[0041] FIG. 5 illustrates the light provided to the scene of FIG. 3 based on the calculations shown in FIG. 4. In FIG. 5, the lighter the color tone, the more light from the light source corresponds, and the darker the color tone, the less light from the light source corresponds. As illustrated in FIG. 5, more light is provided to the area 42 corresponding to the person 32 in the background, while less light is provided to the area 40 corresponding to the person 30 in the foreground. Additional light is provided to the face 52 of the person in the background. The least amount of light can be provided to the background where neither person 30 nor person 32 appears (not shown).

[0042] FIGS. 6, 7, and 8 show an example of the light source 10 that can be used in the system 1 illustrated in FIG. 1A. Any suitable light source may be used, and the embodiments of the present invention are not limited to the structures shown in FIGS. 6, 7, and 8.

[0043] FIG. 7 is a top view of a square array 60 of LEDs 62. The LEDs 62 can be white LEDs, IR LEDs, or a combination thereof. The LEDs 62 can be grown monolithically on a single substrate. In other examples, the LEDs 62 need not be grown monolithically on a single substrate and may be arranged on a mount after dicing such that adjacent LEDs are very close to each other. In some embodiments, the gap between the LEDs 62 is less than 1 / 3 of the dimension (e.g., width) of an individual LED 62. A 3×3 square array is shown, but any suitable number of LEDs may be used and the array need not be square and may be rectangular or of any suitable shape. The size of an individual LED may depend on several design parameters such as, for example, the build volume including an optical lens, the field of view of a camera, and the number of LEDs in the array. For example, the array must include sufficient LEDs to illuminate the entire field of view of the camera (i.e., the entire scene). For smartphone applications, the overall width of the array can be 2 mm or less in some embodiments. For larger cameras, the width of the array can be 10 mm or less in some embodiments. These individual LEDs are square, but this is not necessary and rectangular LEDs or LEDs of any suitable shape may be used.

[0044] FIG. 6 is a cross-sectional view of the light source 10. The array 60 of LEDs 62 is positioned such that most of the light extracted from the array 60 is emitted toward the optical system 64. In the illustrated example, the optical system 64 is spaced apart from the array 60. In other examples, the optical system 64 may be placed on top of the array 60. The optical system 64 can be any suitable structure that collimates the light and directs the light toward an appropriate region of the scene. The optical system 64 can be, for example, a lens, a plurality of lenses, one or more Fresnel lenses, one or more refractive lenses, one or more total internal reflection lens elements, one or more reflectors, one or more collimators, or other suitable optical systems. In the following example, the optical system 64 is a Fresnel lens. The light source 10 may be in the shape of a box 66, with the array 60 disposed at the bottom of the box and the optical system 64 forming the top of the box. The inner side walls 68 of the box, the portion of the bottom not occupied by the array 60, and the portion of the lid not occupied by the optical system 64 are part of the optical design and can thus be made reflectant or light-absorbent as appropriate.

[0045] FIG. 8 is a cross-sectional view of an example of a single LED 62 within the array shown in FIGS. 6 and 7. Any suitable LED may be used, and embodiments of the present invention are not limited to the structure illustrated in FIG. 8. In the device of FIG. 8, most of the light is extracted from the LED through the growth substrate. Such a device may be referred to as a flip-chip device. The LED of FIG. 8 is formed by growing a semiconductor structure of a group III nitride (e.g., gallium nitride for a blue LED or UV LED) or a group III arsenide (e.g., gallium arsenide for an IR LED) on a growth substrate 70, as is technically known. The growth substrate 70 is often sapphire, but may be any suitable substrate, such as, for example, a non-group III nitride material, SiC, Si, GaN, or a composite substrate. The surface of the growth substrate on which the group III nitride or group III arsenide semiconductor structure is grown may be patterned, roughened, or textured prior to growth, which can improve light extraction from the device. The surface of the growth substrate on the side opposite the growth surface (i.e., the surface through which most of the light is extracted in the flip-chip configuration) may be patterned, roughened, or textured before or after growth, which can improve light extraction from the device.

[0046] The semiconductor structure includes a light-emitting region or an active region sandwiched between an n-type region and a p-type region. First, the n-type region 72 can be grown. The n-type region 72 can include a plurality of layers with different compositions and dopant concentrations. The plurality of layers can include, for example, a preparation layer such as a buffer layer or a nucleation layer that can be n-type or not intentionally doped, and a device layer designed to meet specific optical, material, or electrical characteristics desirable for efficient light emission in the light-emitting region, which can be n-type or p-type. On top of the n-type region 72, the light-emitting region or the active region 74 is grown. Examples of suitable light-emitting regions include a single thick or thin light-emitting layer, or a multi-quantum well light-emitting region including a plurality of thin or thick light-emitting layers separated by barrier layers. Next, on top of the light-emitting region 74, the p-type region 76 can be grown. Similar to the n-type region 72, the p-type region 76 can include a plurality of layers with different compositions, thicknesses, and dopant concentrations, and the plurality of layers can include layers that are not intentionally doped or n-type layers.

[0047] After the growth of the semiconductor structure, a reflective p-contact 78 is formed on the surface of the p-type region 76. The p-contact 78 often includes a plurality of conductive layers such as, for example, a reflective metal and a guard metal. The guard metal can prevent or suppress the electromigration of the reflective metal. The reflective metal is often silver, but any suitable one or more materials can be used. After forming the p-contact 78, a portion of the n-type region 72 where the n-contact 80 is to be formed on top is exposed by removing a portion of the p-contact 78, the p-type region 76, and the active region 74. The n-contact 80 and the p-contact 78 are electrically isolated from each other by a gap 82 that can be filled with a dielectric such as, for example, silicon oxide or other suitable materials. A plurality of n-contact vias can be formed, and the n-contact 80 and the p-contact 78 are not limited to the configuration illustrated in FIG. 8. The n and p contacts can be re-wired to form bond pads having a dielectric / metal stack, as is technically known (not shown).

[0048] As described above, the LEDs 62 within the array 60 are formed on a single wafer and can then be diced from the wafer as an array 60 in which the individual LEDs 62 in the array are still attached to a single growth substrate portion. In other examples, a number of LEDs 62 are formed on a single wafer and then diced from the wafer so that the diced individual LEDs can be arranged on a mount to form the array 60.

[0049] The substrate 70 can be thinned after the growth of the semiconductor structure or after forming the individual devices. In some embodiments, the substrate is removed from the device of FIG. 8. Most of the light extracted from the device of FIG. 8 is extracted through the substrate 70 (or the surface of the semiconductor structure exposed by removing the substrate 70). Embodiments of the present invention are not limited to flip-chip LEDs, and any suitable device may be used.

[0050] A wavelength conversion structure 84 can be disposed within the path of light extracted from the light emitting device. The wavelength conversion structure can include one or more wavelength conversion materials that can be, for example, a conventional phosphor, an organic phosphor, quantum dots, an organic semiconductor, a II-VI or III-V semiconductor, a II-VI or III-V semiconductor quantum dot or nanocrystal, a dye, a polymer, or other light emitting material. The wavelength conversion material absorbs the light emitted by the LED and emits light of one or more different wavelengths. The un-converted light emitted by the LED often forms part of the final spectrum of the light extracted from this structure, but this is not necessarily the case. The final spectrum of the light extracted from the structure can be white, multi-color, or single-color. Examples of common combinations include a blue-emitting LED combined with a yellow-emitting wavelength conversion material, a blue-emitting LED combined with green-emitting and red-emitting wavelength conversion materials, a UV-emitting LED combined with blue-emitting and yellow-emitting wavelength conversion materials, and a UV-emitting LED combined with blue-emitting, green-emitting and red-emitting wavelength conversion materials. To adjust the spectrum of the light extracted from the structure, wavelength conversion materials emitting light of other colors may be added. The wavelength conversion structure 84 may include a light scattering element or a light diffusing element, such as TiO 2 etc.

[0051] In some embodiments, the wavelength conversion structure 84 is a structure that is manufactured separately from the LED and attached to the LED by a suitable adhesive such as wafer bonding or silicone or epoxy. An example of such a pre-manufactured wavelength conversion element is a ceramic phosphor, which is formed, for example, by sintering a powder phosphor or a precursor of the phosphor into a ceramic slab that can be later diced into individual wavelength conversion elements. The ceramic phosphor may also be formed, for example, by tape casting, in which case the ceramic is manufactured in an exact shape without the need for dicing or cutting. Examples of suitable non-ceramic preform wavelength conversion elements are powder phosphors dispersed in a transparent material such as silicone or glass that is formed into a sheet by rolling, casting, or other methods and then singulated into individual wavelength conversion elements, powder phosphors placed within a transparent material such as silicone and laminated over a wafer of LEDs or individual LEDs, and phosphors mixed with silicone and placed on a transparent substrate. The wavelength conversion element need not be pre-formed and may be, for example, a wavelength conversion material mixed with a transparent binder that is laminated, dispensed, deposited, screen printed, electrophoretically deposited, or otherwise positioned within the path of light emitted by the LED.

[0052] The wavelength conversion structure 84 need not be disposed in direct contact with the LED as illustrated in FIG. 8, and in some embodiments, the wavelength conversion structure 84 is spaced apart from the LED.

[0053] The wavelength conversion structure 84 may be a monolithic element that covers multiple LEDs or all of the LEDs in an array, or it may be structured into separate segments, each of which is attached to a corresponding LED. The gaps between those separate segments of the wavelength conversion structure 84 may be filled with an optically reflective material so as to confine the light emission from each segment to that segment only.

[0054] To electrically and physically connect the LEDs 62 within the array 60 to a structure such as, for example, a mount, a printed circuit board, or other suitable structure, an interconnect (not shown) such as, for example, solder, stud bumps, a gold layer, or other suitable structure may be used. The mount may be configured such that individual LEDs 62 can be individually controlled by the driver 12 of FIG. 1A. The light emitted by the individual LEDs 62 illuminates different parts of the scene. By varying the current to the individual LEDs, the light provided to the corresponding parts of the scene can be changed. By supplying each LED 62 with an appropriate level of current, the optimal illuminance profile for the scene calculated as described above is obtained.

[0055] In some devices, such as, for example, mobile devices or battery-powered devices, the maximum amount of current available to the adaptive lighting system of FIG. 1A is often limited by the capacity of the device battery. When determining the drive current levels for all the LEDs 62, the system 1 typically takes into account the maximum available current budget such that the correct intensity ratio between the LEDs is maintained and the total drive current does not exceed the maximum value while maximizing the total light output.

[0056] FIG. 9 shows a scene illuminated in the examples shown below in FIGS. 10A, 11A, 12A, 13A, 14A, and 15A. The amount of current supplied to each LED 62 in each example is shown in FIGS. 10B, 11B, 12B, 13B, 14B, and 15B. A target 88, which can be a face and is identified by a dashed line in FIG. 9, requires more light than the rest of the scene according to calculations from a 3D profile as described above. In each of FIGS. 10A, 11A, 12A, 13A, 14A, and 15A, the amount of light supplied to the area decreases as the darkness of the color tone increases. The light distribution shown in each figure can be considered relative.

[0057] Figure 10A shows how the scene is illuminated when all the LEDs 62 are supplied with the same amount of current, as shown in Figure 10B. While the center of the scene is brightly illuminated, the outer edges of the scene are not well illuminated. Therefore, the part of the target closer to the center of the scene is illuminated more than the part of the target closer to the edge (end) of the scene.

[0058] Figure 11A shows how the scene is illuminated when only three LEDs are supplied with current, each of the three receiving the same amount of current and the other six LEDs receiving no current. The three LEDs 91, 92, and 93 that are supplied with current are, as shown in Figure 11B, the central LED and the two lower LEDs in the leftmost column. As shown in Figure 11A, the right side of the scene that generally corresponds to the target is illuminated brighter than the rest of the scene. The current density of the LEDs 91, 92, and 93 in Figure 11B can be three times as high as that shown in Figure 10B where all the LEDs are supplied with equal current. The illuminance of the target in Figure 11A is approximately 1.6 times as high as the illuminance of the target in Figure 10A.

[0059] To obtain a higher illuminance, as shown in two examples shown in Figures 12A, 12B, 13A, and 13B, fewer segments can be switched on.

[0060] Figure 12A shows how the scene is illuminated when only two LEDs are supplied with current, each receiving the same amount of current and the other seven LEDs receiving no current. The two LEDs 94 and 95 that are supplied with current are, as shown in Figure 12B, the two lower LEDs in the leftmost column. As shown in Figure 12A, the right side of the scene that generally corresponds to the target is illuminated brighter than the rest of the scene. The illuminance of the target in Figure 12A is higher than the illuminance of the target in Figure 11A.

[0061] FIG. 13A shows how the scene is illuminated when only a single LED is supplied with current and the other eight LEDs receive no current. The LED 96 supplied with current is the central LED in the leftmost column, as shown in FIG. 13B. As shown in FIG. 13A, the right side of the scene generally corresponding to the target is illuminated brighter than the rest of the scene, but the bright spot is smaller than in FIGS. 12A and 11A. The illuminance of the target in FIG. 13A is higher than the illuminance of the target in FIG. 11A.

[0062] To improve the uniformity of the illuminance across the entire target, as shown in two examples shown in FIGS. 14A, 14B, 15A, and 15B, the currents supplied to different LEDs can be made different.

[0063] FIG. 14A shows how the scene is illuminated when six LEDs are supplied with variable levels of current and three LEDs receive no current. The central LED 96 in the left column is supplied with five times as much current as the five LEDs 97, 98, 99, 100, and 101 surrounding the LED 96. As shown in FIG. 14B, the three LEDs in the right column receive no current. As shown in FIG. 14A, the right side of the scene generally corresponding to the target is illuminated brighter than the rest of the scene. The illuminance of the target is more uniform, for example, than in FIG. 13A.

[0064] FIG. 15A shows how the scene is illuminated when four LEDs are supplied with variable levels of current and five LEDs receive no current. The central LED 102 in the left column is supplied with a current that is four times that of the lowermost LED 105 in the central column and twice that of the central LED 104 and the lowermost LED 103 in the left column. As shown in FIG. 15B, the LEDs in the top row and the right column receive no current. As shown in FIG. 15A, the right side of the scene generally corresponding to the target is illuminated brighter than the rest of the scene. The illuminance of the target is more uniform, for example, than in FIG. 13A.

[0065] Figures 16, 17B, and 18B show how current can be applied to the array 60 of LEDs 62 in FIG. 6 for zoom and wide-angle applications. When a command to zoom in the camera lens is received, as shown in FIGS. 16 and 17B, the LEDs near the center of the array receive more current. FIG. 17A shows how the scene is illuminated when the LEDs are supplied with variable levels of current as shown in FIG. 17B.

[0066] When a command to zoom out the camera lens is received, as shown in FIG. 18B, the LEDs near the edge of the array receive more current. FIG. 18A shows how the scene is illuminated when the LEDs are supplied with variable levels of current as shown in FIG. 18B.

[0067] In FIG. 16, for the zoom application, only the central LED 110 is supplied with current, and the eight LEDs surrounding the central LED receive no current. The center of the scene will be brightly illuminated, and the edges of the scene will receive less light. The illuminance at the center of the scene can be increased by a factor of 2.2 compared to the center of the scene in FIG. 10A where all nine LEDs receive equal current.

[0068] In FIG. 17B, for the zoom application, the central LED 111 is supplied with a current that is twice as much as LED 112 and four times as much as LED 114. The center of the scene is illuminated more than the edges of the scene. The illuminance at the center of the scene can be increased by a factor of 1.15 compared to the center of the scene in FIG. 10A where all nine LEDs receive equal current.

[0069] In FIG. 18B, for the wide-angle application, the eight LEDs 118 at the edge of the array receive equal current, and the central LED 116 receives no current. The illuminance at the center of the scene can be reduced to 0.85 times the illuminance at the center of the scene in FIG. 10A where all nine LEDs receive equal current.

[0070] The adaptive illumination system 1 in FIG. 1A can be used to illuminate a plurality of targets (e.g., faces) by supplying current only to the LEDs corresponding to each target or by supplying more current to the LEDs corresponding to each target. Using the adaptive illumination system 1, overexposure in a scene including elements close to the camera and elements far from the camera can be suppressed by supplying current only to the LEDs corresponding to elements far from the camera or by supplying more current to the LEDs corresponding to elements far from the camera.

[0071] The illuminance values given for the above examples are calculated for the illustrated 3×3 array with a single Fresnel lens. The light output of each LED in the above examples can be controlled by the driver current of the LED or by the pulse time with a constant current.

[0072] FIGS. 19, 20, 21, 22, and 23 show alternative light sources.

[0073] In the light source of FIG. 19, each LED 62 in the array has an individual optical system 122 instead of a single optical system for the entire array as shown in FIG. 6. Each optical system 122 directs the light from its LED towards a specific portion of the scene. The optical system 122 can be any suitable optical system including, for example, a lens, a dome lens, a Fresnel lens, a reflector, a total internal reflection lens, or other suitable structures. The optical systems 122 do not have to be the same, and different optical systems may be used for different LEDs 62 in the array.

[0074] The light source of FIG. 20 includes a plurality of LED arrays each having a plurality of optical elements. For example, FIG. 20 shows two 3×3 arrays each having a single corresponding Fresnel lens. More or fewer arrays may be used, and the arrays are not limited to the illustrated devices. In some embodiments, each array illuminates a portion of the scene. Array 124 in FIG. 20 illuminates the upper portion 128 of the scene, and array 126 illuminates the lower portion 130 of the scene. In some embodiments, these arrays illuminate a portion of the scene with an overlap to provide more light in the overlapping portions. For example, these arrays may overlap at the center of the scene, which may be a portion of the scene that often requires more light than the edges.

[0075] The light source of FIG. 21 uses a narrow-beam emitting device such as a laser. The light source of FIG. 21 includes a laser 140, and a wavelength conversion element 142 is disposed in the path of the light from the laser. A focusing optical system 144 can create a light beam of a desired size. The beam is incident on a first scanning mirror 146 and a second scanning mirror 148 before being incident on the scene 150. These scanning mirrors can be moved so that the light beam scans the entire scene, during which a driver controls the intensity of the light source so that different portions of the scene can receive different amounts of light. When the beam scans a portion of the scene that requires a higher intensity, the current supplied to the laser increases, and when the beam scans a portion of the scene that requires only a lower intensity, the current supplied to the laser decreases.

[0076] The light source of FIG. 22 includes a matrix control element, such as a digital micromirror switching device or a multi-segment liquid crystal display. Light from the LED or laser 152 illuminates the matrix control element 154. The intensity of the reflected or transmitted light is changed according to the calculated illumination profile. The reflected or transmitted light from the matrix switching element 154 is projected onto the scene 156. The matrix switching element 154 may have a number of small mirrors as pixels. By changing the orientation of each mirror, the intensity of each pixel can be adjusted. The orientation of the mirrors can also be used to create brighter regions by overlapping beams from multiple different mirrors.

[0077] The light source in FIG. 23 is color tunable. The light source in FIG. 23 includes two arrays 160 and 162 arranged to emit overlapping beams 166 and 168 respectively when illuminating scene 164. Two arrays such as the arrays shown in FIG. 6 are illustrated, but other suitable light emitters may be used. This system may include three or more arrays having different emission spectra. Arrays 160 and 162 emit light of different colors. For example, both arrays 160 and 162 may emit white light, but array 160 may emit white light having a color temperature different from that of array 162. That is, one of arrays 160 and 162 emits warm white light. For example, an array that emits warm white light can emit light having a color temperature as low as about 1700K, and an array that emits cool white light can emit light having a color temperature as high as about 10000K. The difference in color temperature between these two arrays can be at least 1000K in some embodiments, at least 2000K in some embodiments, at least 3000K in some embodiments, and at least 4000K in some embodiments. Alternatively, arrays 160 and 162 may emit different monochromatic lights. The appropriate current supplied to each LED within each array is calculated such that the sum of the lights from arrays 160 and 162 has an appropriate illuminance and color temperature for each part of the scene. An array (or other light emitter) that emits additional light of a different color or color temperature may be added.

[0078] In some embodiments, multiple LEDs that emit multiple spectra can be combined in a single interleaved array with a single optical system as shown in FIG. 6, or individual optical systems as shown in FIG. 19. LEDs of different colors are grouped, each group illuminating a part of the scene, and each group includes at least one LED of each different color.

[0079] The above color-tunable light source can be used to illuminate different parts of a scene with light of different correlated color temperatures (CCTs). For example, the color-tunable light source can be used to equalize the CCTs of different ambient light sources. A section of a scene having low CCT ambient light can be illuminated with higher CCT light, and a section of a scene having high CCT ambient light can be illuminated with lower CCT light.

[0080] In some embodiments, the light source 10 can be used with a plurality of different cameras. For example, a single smartphone may have multiple cameras, and different smartphone models may use different cameras. Each of those cameras can have a specific field of view, for which the flash for that camera is adjusted (e.g., adjusted to provide a minimum level of illumination at the corners of the field of view). Thus, with conventional flashes, each camera requires a separate flash adjusted to the field of view of that camera. Using an adaptive illumination system according to embodiments of the present invention, the default current distribution for each camera can be defined and selected when that camera is selected, and thus a single light source can be used for multiple cameras. The default for each camera can be changed according to the scene being photographed, as described in the above embodiments.

[0081] In the above examples, the semiconductor light-emitting device is a group III nitride LED that emits blue light or UV light and a group III arsenide LED that emits IR light, but semiconductor light-emitting devices other than LEDs, such as laser diodes, or semiconductor light-emitting devices made of other material systems, such as other III-V materials, group III phosphides, II-VI materials, ZnO, or Si-based materials, may be used.

[0082] Although the present invention has been described in detail, as will be appreciated by those skilled in the art, given the present disclosure, modifications may be made to the present invention without departing from the spirit of the inventive concept described herein. In particular, various elements from a plurality of different examples or embodiments may be combined, and the scope of the present invention is not limited to the specific embodiments illustrated and described.

Claims

1. A camera having a field of view, A light emitting diode (LED) array configured to generate infrared light, An array of lenses coupled to the LED array, each lens configured to direct the infrared light from a corresponding LED to a specific portion within the field of view of the camera, A controller configured to selectively power the LED array to selectively illuminate the field of view such that infrared light from each LED is directed to respective portions within the field of view, At least one processor, A memory coupled to the at least one processor, the memory configured to store instructions that, when executed by the at least one processor, cause the at least one processor to, Control the camera to capture a first image of a scene located within the field of view of the camera, Determine, with the at least one processor, from the first image, the presence of a human face within a first portion of the field of view, With the controller, selectively power a subset of the LEDs of the LED array, the subset configured to direct infrared light to the first portion of the field of view to illuminate the human face with the infrared light, A memory that causes an operation having this to be executed, A system having this.

2. The operation further, With the at least one processor, select the subset of the LEDs of the LED array, the LEDs of the subset being located within the LED array such that infrared light emitted from the LEDs of the subset is directed to the first portion of the field of view to illuminate the human face with the infrared light, The system according to claim 1, having this.

3. The operation further, Power the subset of LEDs and, with the camera, capture a second image of the scene, the second image including the human face illuminated with the infrared light, The system according to claim 1, having this.

4. The operation further, With the at least one processor, determine the identity of the human in the first image and the second image from the second image, The system according to claim 3, having this.

5. The system of claim 3, wherein when the second image is captured, the controller supplies power only to the subset of LEDs within the LED array.

6. The operation further receiving, by the at least one processor, instructions for capturing the first image of the scene The system of claim 1, comprising:

7. The system of claim 6, further comprising an input device configured to provide the instructions in response to an input.

8. The system of claim 7, wherein the input device is a button located on the camera, and the input is a depression of the button.

9. The system of claim 6, wherein the camera is controlled to capture the first image of the scene in response to the instructions being received.

10. The operation further controlling the controller to capture the first image by supplying power to each LED in the LED array such that at least two LEDs in the LED array are supplied with different current values. The system of claim 1, comprising:

11. The operation further controlling the controller to capture the first image by supplying power to each LED in the LED array with the same first current value. The system of claim 1, comprising:

12. The operation further determining, by the at least one processor, from the first image that the face of the person is under-exposed in the first image, determining, by the at least one processor, from the first image, a second current value greater than the first current value, comprising: wherein the controller supplies power to the subset of LEDs with the second current value. The system of claim 11.

13. The system of claim 12, wherein the product of the number of LEDs in the LED array and the first current value is substantially equal to the product of the number of LEDs in the subset and the second current value.

14. The operation further determining, by the at least one processor, from the first image that the face of the person is over-exposed in the first image, determining, by the at least one processor, from the first image, a second current value less than the first current value. having the controller supplies power to the subset of LEDs with the second current value The system according to claim 11

15. The system according to claim 14, wherein the product of the number of LEDs in the LED array and the first current value is substantially equal to the product of the number of LEDs in the subset and the second current value

16. A camera captures a first image of a scene located within the field of view of the camera At least one processor coupled to the camera determines from the first image that a human face is present within a first portion of the field of view A controller coupled to an array of light-emitting diodes (LEDs) selectively supplies power to a subset of the LEDs within the array, and the subset of LEDs is located within the array such that infrared light emitted from the subset of LEDs is directed toward the first portion of the field of view and illuminates the human face with the infrared light having the LED array is coupled to a lens array, and each lens is configured to direct the infrared light from a corresponding LED to a specific portion within the field of view of the camera Method

17. Power the subset of LEDs and capture, with the camera, a second image of the scene, the second image including the human face illuminated by the infrared light Determine, with the at least one processor, the identity of the human in the first image and the second image from the second image The method according to claim 16, further having

18. The method further includes determining, with the at least one processor, from the first image that the human face is under-exposed in the first image determining, with the at least one processor, from the first image a second current value that is greater than the first current value having the controller supplies power to the subset of LEDs with the second current value the product of the number of LEDs in the array and the first current value is substantially equal to the product of the number of LEDs in the subset and the second current value The method according to claim 17

19. The method further includes determining, with the at least one processor, from the first image that the human face is over-exposed in the first image With the at least one processor, determining, from the first image, a second current value that is less than the first current value comprising The controller supplies power to the subset of LEDs with the second current value The product of the number of LEDs in the array and the first current value is substantially equal to the product of the number of LEDs in the subset and the second current value The method according to claim 17 **Claim 20** A camera having a field of view, A light emitting diode (LED) array configured to generate infrared light, An array of lenses coupled to the LED array, each lens configured to direct the infrared light from a corresponding LED to a specific portion within the field of view of the camera A controller configured to selectively power the LED array to selectively illuminate the field of view such that infrared light from each LED is directed to respective portions within the field of view At least one processor, A memory coupled to the at least one processor, the memory configured to store instructions that, when executed by the at least one processor, cause the at least one processor to With the controller, supply power to each LED in the LED array with a first current value Supply power to the LEDs of the LED array with the first current value and control the camera to capture a first image of a scene located within the field of view of the camera With the at least one processor, determine from the first image that a human face is present within a first portion of the field of view With the at least one processor, select a subset of the LEDs of the LED array, the LEDs of the subset being located within the LED array such that infrared light emitted from the LEDs of the subset is directed to the first portion of the field of view and illuminates the human face with the infrared light With the controller, selectively supply power to each LED in the subset with a second current value different from the first current value, the subset being configured to direct infrared light to the first portion of the field of view and illuminate the human face with the infrared light Supply power to the LEDs of the subset with the second current value, and capture, with the camera, a second image of the scene, the second image including the face of the person illuminated by the infrared light. A memory that causes an operation having this to be executed. A system having this.

Images