Wide-angle white balance

Abstract

A device configured to: detect activation of an image sensor; automatically capture at least one intermediate image of a scene via the image sensor; detect capture of a primary image; detect at least one light source from visual information of the scene associated with the captured at least one intermediate image; determine a white balance coefficient associated with the detected at least one light source; and apply the determined white balance coefficient to the captured primary image. The intermediate image of the scene creates a field of view of the scene that is greater than the field of view of the captured primary image. The intermediate image can be used to construct a three-dimensional model of the scene, providing additional information for calculating the light source.

Description

Technical Field

[0001] Aspects of the present invention relate generally to imaging devices and, more particularly, to detecting light sources in a scene. Background Art

[0002] Automatic white balance (AWB) algorithms are used in imaging devices such as cameras to modify pixel values ​​to obtain color-correct images. More precisely, these algorithms are used to correctly set the white point of the image. The first step in white balancing is to identify the light source, then calculate and apply the appropriate correction based on the identified light source. Typically, white balance correction is performed using a global algorithm, where an offset and a gain are calculated for each color component. These offsets and gains are then used to map these pixel values ​​to the correct pixel values, which ideally will match the true scene color. While this color correction is a trivial task, robustly identifying the correct light source remains a challenging task even under the most challenging conditions. Failure to correctly identify the light source will lead to incorrect estimation of the color correction parameters, resulting in an image with incorrect colors.

[0003] Automatic white balance algorithms often have difficulty identifying monochromatic objects in a scene. In scenes containing only monochromatic objects, such as uniformly colored walls or blades of grass, the correct color cannot be recognized. Many combinations of light source spectra and object colors can produce the same pixel value on the camera sensor. For example, if a flat, monochromatic object appears reddish, it is impossible to tell whether it is red under illuminant 1 or gray under illuminant 2. Understanding whether the light source is illuminant 1 or 2 is essential for proper color processing. Monochromatic objects in scenes are more common if the camera has a narrow field of view (FOV).

[0004] In existing solutions, automatic white balance algorithms can use historical information. For example, if the light source is not correctly identified, the last known reliable result is used. However, this is merely a guess, as there is no guarantee that the last reliable light source was estimated based on the current scene. For example, changing the scene's imaging perspective may cause the light source to change.

[0005] Therefore, it is necessary to provide a more robust way to identify light sources to at least solve some of the above-mentioned identification problems. Summary of the Invention

[0006] The various aspects of the disclosed embodiments are directed to detecting light sources. This object is achieved by the subject matter of the independent claims. Further advantageous modifications are provided in the dependent claims.

[0007] According to a first aspect, the above and other objects and advantages are obtained by an apparatus. In one embodiment, the apparatus includes a processor configured to: detect activation of an image sensor; automatically capture at least one intermediate image of a scene via the image sensor; detect capture of a main image of at least a portion of the scene via the image sensor; detect at least one light source from visual information of the scene associated with the captured at least one intermediate image; determine a white balance coefficient associated with the detected at least one light source; and apply the determined white balance coefficient to the captured main image. Aspects of the disclosed embodiments improve automatic white balance performance by using a wider field of view for light source detection. The automatic white balance algorithm uses more information obtained from the intermediate images to improve the robustness of light source detection.

[0008] In one possible implementation of the apparatus, the processor is configured to construct the visual information of the scene by stitching the primary image and at least one captured intermediate image together to create a stitched image, wherein the stitched image has a field of view greater than that of the captured primary image. Aspects of the disclosed embodiments create a three-dimensional model using the adjacent scene and detect the temperature of the light source of the captured image from the three-dimensional model.

[0009] In one possible implementation of the device, the processor is configured to detect activation of the image sensor from one or more of the following: activation of a camera or camera application of the device; movement of the device; a change in the image sensor's frame view; framing of the primary image; or an inability to detect a light source associated with the capture of the primary image. Aspects of the disclosed embodiments are directed to automatically constructing a three-dimensional view of a scene and then using the three-dimensional view to detect or determine the light source of an image captured from the scene.

[0010] In one possible implementation of the apparatus, the processor is configured to: create a three-dimensional model of the scene from the at least one captured intermediate image; calculate at least one light source from the three-dimensional model of the scene; and provide the calculated light source as the detected light source. Aspects of the disclosed embodiments utilize adjacent scenes to detect the temperature of the light source of the captured image.

[0011] In one possible implementation of the apparatus, the processor is further configured to: delete the three-dimensional model of the scene; and, after detecting the light source, begin capturing a new intermediate image of the scene. In the system of the disclosed embodiment, the wider field of view is based on current information rather than random historical information.

[0012] In one possible implementation of the apparatus, the processor is further configured to: detect a change in orientation of the image sensor; delete the at least one captured intermediate image; capture at least one new intermediate image; and create the three-dimensional model of the scene from the at least one new captured intermediate image. Aspects of the disclosed embodiments use current information about the scene, rather than historical information, to calculate the light source. A change in orientation may indicate a new scene.

[0013] In one possible implementation of the apparatus, automatically capturing the at least one intermediate image of the scene using the image sensor includes creating a field of view of the scene that is greater than a field of view of the image sensor. The intermediate image of the scene creates a field of view that is greater than a field of view of the image sensor, which provides more information for detecting and calculating the light source.

[0014] In one possible implementation of the apparatus, the initial image is automatically captured, regardless of whether the user starts the camera application. Aspects of the disclosed embodiments are used to collect information about the scene before capturing the main image.

[0015] In one possible implementation of the apparatus, automatic capture of the initial image is triggered by a failure in light source detection. Aspects of the disclosed embodiments use current information of the scene rather than historical information to calculate the light source.

[0016] In one possible implementation of the apparatus, any of a variety of known methods is used to construct the three-dimensional model of the scene. Aspects of the disclosed embodiments create a three-dimensional model using an adjacent scene and detect the temperature of the light source of the captured image from the three-dimensional model.

[0017] In one possible implementation of the apparatus, the three-dimensional model contains only information related to the orientation of the camera relative to the scene. Aspects of the disclosed embodiments create a three-dimensional model using the adjacent scene and detect the temperature of the light source of the captured image from the three-dimensional model.

[0018] In one possible implementation of the apparatus, the captured intermediate image is used to calculate a light source. Aspects of the disclosed embodiments use current information of the scene rather than historical information to calculate the light source.

[0019] In one possible implementation of the apparatus, the three-dimensional scene data is used to select which intermediate images to use to select the light source. Aspects of the disclosed embodiments use current information of the scene rather than historical information to calculate the light source.

[0020] In one possible implementation of the apparatus, the processor is configured to detect mixed light sources. Aspects of the disclosed embodiments use current information of the scene rather than historical information to calculate the light sources.

[0021] In one possible implementation of the device, the processor is configured to construct an image with a wider field of view by stitching the intermediate images. The intermediate images of the scene create a field of view of the scene that is greater than the field of view of the image sensor, providing more information for calculating the light source. Aspects of the disclosed embodiments improve auto-white balance performance by using a wider field of view for light source detection. The auto-white balance algorithm uses the additional information obtained from the intermediate images to improve the robustness of light source detection.

[0022] In one possible implementation of the apparatus, the stitched image is used to calculate a light source. The intermediate image of the scene creates a field of view of the scene that is larger than the field of view of the image sensor, which provides more information for calculating the light source.

[0023] In one possible implementation of the apparatus, the stitched images are used to calculate the mixed light source. The intermediate image of the scene creates a field of view of the scene that is larger than the field of view of the image sensor, which provides more information for calculating the light source.

[0024] In one possible implementation of the apparatus, the calculated 3D scene model is deleted, and a new intermediate image is captured immediately after the primary image is processed. In the system of the disclosed embodiment, the wider field of view is based on current information rather than random historical information.

[0025] In one possible implementation of the apparatus, after processing the main image, the created image with a wider field of view is deleted. In the system of the disclosed embodiment, the wider field of view is based on current information rather than random historical information.

[0026] In one possible implementation of the apparatus, after processing the main image, the captured intermediate image is deleted. In the system of the disclosed embodiment, the wider field of view is based on current information rather than random historical information.

[0027] In one possible implementation of the apparatus, a three-dimensional model of the scene is constructed based on the newly captured intermediate image. In the system of the disclosed embodiment, the wider field of view is based on current information rather than random historical information.

[0028] In one possible implementation of the apparatus, a stitched image with a wider field of view is constructed based on a set of newly captured intermediate images. In the system of the disclosed embodiment, the wider field of view is based on current information rather than random historical information.

[0029] In one possible implementation of the apparatus, the three-dimensional model of the scene and the intermediate image are used to calculate a light source. The intermediate image creates a field of view of the scene that is larger than a field of view of the image sensor, which provides more information for calculating the light source.

[0030] In one possible implementation of the apparatus, the three-dimensional model of the scene and the intermediate image are used to calculate a mixed light source. The intermediate image creates a field of view of the scene that is larger than the field of view of the image sensor, which provides more information for calculating the light source.

[0031] In one possible implementation of the apparatus, the image obtained with the larger field of view is used to calculate a light source. The intermediate image creates a field of view of the scene that is larger than the field of view of the image sensor, which provides more information for calculating the light source.

[0032] In one possible implementation of the apparatus, the obtained image with the larger field of view is used to calculate the mixed light source. The intermediate image creates a field of view of the scene that is larger than the field of view of the image sensor, which provides more information for calculating the light source.

[0033] In one possible implementation of the apparatus, the detected light source is used to calculate a white balance coefficient for a next captured main image. Aspects of the disclosed embodiments use current information of the scene rather than historical information to calculate the light source.

[0034] In one possible implementation of the apparatus, the user may stop capturing the background of the intermediate image.Aspects of the disclosed embodiments may control the capture of the intermediate image used to create the three-dimensional model of the scene.

[0035] In one possible implementation of the apparatus, the user may select a scene of which the intermediate image is captured. Aspects of the disclosed embodiments may control the capture of intermediate images used to create the three-dimensional model of the scene.

[0036] In one possible implementation of the apparatus, the light source is calculated only from the primary image.Aspects of the disclosed embodiments may control the capture of intermediate images used to create the three-dimensional model of the scene.

[0037] In one possible implementation of the apparatus, detecting that the light source cannot be estimated from the primary image.Aspects of the disclosed embodiments may control when intermediate images are captured to create the three-dimensional model of the scene.

[0038] In one possible implementation of the apparatus, the inability to detect a light source is communicated to a user who may begin collecting intermediate images.Aspects of the disclosed embodiments may control when intermediate images are captured to create the three-dimensional model of the scene.

[0039] According to a second aspect, the foregoing and other objects and advantages are achieved by a method. In one embodiment, the method comprises: detecting activation of an image sensor of a device; automatically capturing at least one intermediate image of a scene via the image sensor; detecting capture of a primary image of at least a portion of the scene via the image sensor; detecting at least one light source from visual information of the scene associated with the captured at least one intermediate image; determining a white balance coefficient associated with the detected at least one light source; and applying the determined white balance coefficient to the captured primary image.

[0040] In one possible implementation of the method, the method further includes constructing the visual information of the scene by stitching the primary image and at least one captured intermediate image together to create a stitched image, wherein the stitched image has a field of view greater than the field of view of the captured primary image. The intermediate images of the scene create a field of view of the scene that is greater than the field of view of the image sensor, which provides more information for detecting or calculating the light source.

[0041] In one possible implementation of the method, the method further includes detecting activation of the image sensor from one or more of: activation of a camera or camera application of the device; movement of the device; a change in the frame view of the image sensor; framing of the primary image; or an inability to detect a light source associated with the capture of the primary image. Aspects of the disclosed embodiments are directed to automatically constructing a three-dimensional view of a scene and then using the three-dimensional view to detect or determine the light source of an image captured from the scene.

[0042] In one possible implementation of the method, the method further includes: creating a three-dimensional model of the scene from the captured at least one intermediate image; calculating at least one light source from the three-dimensional model of the scene; and providing the calculated light source as the detected light source. The intermediate image of the scene creates a field of view of the scene that is larger than the field of view of the image sensor, which provides more information for calculating the light source.

[0043] In one possible implementation of the method, the method further includes: deleting the three-dimensional model of the scene; and, after detecting the light source, starting to capture a new intermediate image of the scene. Aspects of the disclosed embodiments rely on current information rather than historical information to calculate the light source.

[0044] In one possible implementation of the method, the method further includes: detecting a change in orientation of the image sensor; deleting the at least one captured intermediate image; capturing at least one new intermediate image; and creating the three-dimensional model of the scene from the at least one new captured intermediate image. Aspects of the disclosed embodiments use current information about the scene, rather than historical information, to calculate the light source. A change in orientation may indicate a new scene.

[0045] In one possible implementation of the method, automatically capturing the at least one intermediate image of the scene using the image sensor further includes creating a field of view of the scene that is greater than the field of view of the image sensor. The intermediate image of the scene creates a field of view of the scene that is greater than the field of view of the image sensor, which provides more information for calculating the light source. Aspects of the disclosed embodiments improve auto-white balancing performance by using a wider field of view for light source detection. The auto-white balancing algorithm uses the additional information obtained from the intermediate image to improve the robustness of light source detection.

[0046] These and other aspects, implementations, and advantages of the exemplary embodiments will become apparent from the embodiments described herein when considered in conjunction with the accompanying drawings. It should be understood, however, that such description and drawings are for illustrative purposes only and are not intended to limit the disclosed invention; reference should be made to the appended claims for any limitations thereto. Additional aspects and advantages of the invention will be set forth in the description that follows, and in part will be apparent from the description or may be learned by practice of the invention. Furthermore, aspects and advantages of the invention may be realized and obtained by means of the instrumentalities or combinations particularly pointed out in the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In the following detailed description of the present invention, the present invention will be explained in detail with reference to exemplary embodiments shown in the accompanying drawings, in which:

[0048] Figure 1 A schematic block diagram illustrating exemplary apparatuses provided by various aspects of the disclosed embodiments is shown;

[0049] Figure 2 Flowcharts illustrating aspects of exemplary methods provided for aspects of the disclosed embodiments;

[0050] Figure 3Flowcharts illustrating aspects of exemplary methods provided for aspects of the disclosed embodiments;

[0051] Figure 4 Flowcharts illustrating aspects of exemplary methods provided for aspects of the disclosed embodiments;

[0052] Figure 5 Flowcharts illustrating aspects of exemplary methods provided for aspects of the disclosed embodiments;

[0053] Figure 6 Flowcharts illustrating aspects of exemplary methods provided for aspects of the disclosed embodiments;

[0054] Figure 7 A schematic block diagram is shown of an exemplary apparatus that can be used to implement aspects of the disclosed embodiments. DETAILED DESCRIPTION

[0055] Figure 1 A schematic block diagram of an exemplary apparatus 100 provided by aspects of the disclosed embodiments is shown. The apparatus 100 is used to capture at least one intermediate image of a scene 110 and to determine a white balance coefficient to be applied to a captured main image based on a light source associated with the at least one intermediate image of the scene 110. Aspects of the disclosed embodiments improve automatic white balance (AWB) performance by using a wider field of view (FOV) for light source detection. The intermediate images of the scene create a field of view of the scene that is larger than the field of view of the image sensor, which provides more information for calculating the light source. The automatic white balance algorithm uses the more information obtained from the intermediate images to improve the robustness of light source detection. In the system of the disclosed embodiment, the wider field of view is based on current information rather than random historical information.

[0056] like Figure 1 As shown, apparatus 100 includes a processor 102. In one embodiment, processor 102 is configured to detect activation of image sensor 104 and automatically capture at least one intermediate image of a scene via image sensor 104. Activation of image sensor 104 may occur automatically or manually.

[0057] For example, in one embodiment, the apparatus 100 is or includes a camera device or camera application 106. For the purposes described herein, the camera device or camera application 106 is generally referred to herein as "camera 106." The terms "camera device" or "camera application" are generally used herein to refer to any apparatus or device that includes a camera or imaging device, such as a mobile communication device, a smartphone, or other computing device. A user starts or otherwise initializes or activates the camera 106 of the apparatus 100. In one embodiment, initializing the camera 106 may cause the image sensor 104 to automatically capture an image (referred to herein as an "intermediate image"). The intermediate image may be an image of a scene that appears within the field of view of the viewfinder of the camera 106.

[0058] For example, in one embodiment, a user initializes and aims the camera 106. While the camera 106 is being aimed, the image sensor 104 is used to capture an intermediate image.

[0059] For another example, while camera 106 is being aimed, the framing may change, or the user may zoom in on object 112 in scene 110. When processor 102 detects an action, such as a framing change or a zoom change, image sensor 104 is configured to automatically capture the intermediate image. As will be further described herein, the intermediate image is used to form or create a three-dimensional model or image of scene 110. The three-dimensional model can then be used to detect the light source of a captured primary image of portion 112 of scene 110.

[0060] Processor 102 is configured to detect the capture of a primary image of at least a portion of a scene via image sensor 104, and to detect at least one light source from visual information of scene 110 associated with the captured intermediate image. In one embodiment, processor 102 is configured to execute an automatic white balance algorithm while processing the captured primary image. For example, the automatic white balance algorithm may be executed based on panoramic statistics of the created three-dimensional image of scene 110. When capturing image 112, the panoramic statistics include information outside the camera's field of view, so the determination of the light source is more reliable.

[0061] For another example, in one embodiment, when processing the captured primary image, the processor 102 can be configured to run the algorithm using only camera field of view statistics. However, the results may be unreliable. By using a 3D model of the scene created from the intermediate image as disclosed herein, reliable automatic white balance results can be obtained even when the intermediate image is completely or partially outside the field of view of the captured primary image.

[0062] In one embodiment, the processor 102 is configured to specify which areas of the three-dimensional model outside the current camera field of view are relevant to the current camera field of view. The processor 102 may also be configured to use additional information of the created three-dimensional model to understand which light sources provide light to the camera field of view directly or via reflective surfaces.

[0063] The processor 102 is configured to determine a white balance coefficient associated with the detected at least one light source and apply the determined white balance coefficient to the captured primary image. Being able to understand scenes outside the camera's field of view improves automatic white balance performance, thereby increasing robustness and providing better color.

[0064] Various aspects of the disclosed embodiments are used to create a three-dimensional map or model of a scene using adjacent scenes, and to detect the temperature of the light source of the captured image from the three-dimensional model. A system for building a three-dimensional model is called Simultaneous Localization and Mapping (SLAM). SLAM is a general term for an exemplary system that is used to build a three-dimensional model of an environment and simultaneously locate the device running the SLAM algorithm in the mapped scene. Although SLAM is referenced herein, various aspects of the disclosed embodiments can use any suitable three-dimensional mapping algorithm. SLAM can be used in many applications from three-dimensional modeling to augmented reality (AR) and virtual reality (VR). In one embodiment, a SLAM element can be used with device 100 to improve the accuracy of light source detection.

[0065] Aspects of the disclosed embodiments provide for running a simplified 3D mapping process upon initialization of the camera application 106 of the device 100. For example, as a user frames a scene to be taken, a mapping process such as SLAM will calculate and update a simplified version of the scene using the viewfinder image. The scene to be taken, referred to herein as a captured primary image, is one aspect of the scene.

[0066] In one embodiment, each viewfinder image can also be checked to determine whether the user has changed the orientation of camera 106 or device 100 so that a new scene not previously seen is now being imaged. This can be referred to as a reset point. Processor 102 is configured to construct a simplified three-dimensional model of scene 110 for each position and / or orientation of camera 106 or the camera viewfinder. In this way, more information than just the current frame can be provided that can be used by the automatic white balance algorithm.

[0067] In one embodiment, statistical information and / or light source information can be extracted and stored from previous frames used to construct the 3D model of the current scene 110. Thus, whenever a frame is detected in which a light source cannot be identified, the statistical information and / or light source information previously detected from the same scene is used. In this way, the light source is always detected from the same or current scene, rather than from random historical data.

[0068] Furthermore, historical data is not used when a scene change is detected, as the user may have changed the position and orientation of camera 106 or device 100 toward a completely different scene. Furthermore, the approximate spatial positions of the pixels and their detected light sources are known for each intermediate frame used to construct the three-dimensional model of scene 110. In this way, a more accurate light source model can be constructed, particularly in mixed light source scenes.

[0069] refer to Figure 2 , shows an example of a method 200 incorporating aspects of the disclosed embodiments. In this example, initialization of an image sensor of a device is detected (202). The device may include or comprise a camera device. In one embodiment, as Figure 3 As shown, initialization 202 of the image sensor may include one or more of the following: initialization 231 or opening of a camera application; manual activation 232 of the camera or camera application; movement 233 of the camera; a change in framing 234 of the camera; a change in focus or zoom 235; or a failure (236) to detect a light source associated with capture of a primary image. In alternative embodiments, any suitable mechanism may be used to initiate capture of an intermediate image, as described herein.

[0070] For example, aspects of the disclosed embodiments provide for enabling capture of intermediate images while moving from the camera 106. Alternatively, the intermediate images may be captured after turning on the camera 106 or activating a camera application of the device 100.

[0071] In one embodiment, the apparatus 100 is a smartphone having a camera application 106 and an image sensor 104. It is detected that the user picks up or otherwise moves the smartphone. When the smartphone is moved, or when the smartphone's camera application 106 is initialized or opened, the capture of intermediate images can begin.

[0072] At least one intermediate image of the scene is captured (204). An intermediate image is an image of the scene within the field of view of the viewfinder of camera 106 at the time the image is captured. Aspects of the disclosed embodiments enable capturing one or more intermediate images of the scene. For example, in one embodiment, the primary image is stitched together with a first intermediate image, and the light source is detected. If the light source is not detected, another intermediate image is captured and stitched together with the combined image. The combined image is the result of the previous stitching.

[0073] In one embodiment, a primary image capturing the scene is detected (206). The primary image of the scene is typically the image that the user wishes to capture.

[0074] At least one light source is detected (208) from visual information of the scene associated with or determined from the intermediate image. As will be further described below, this may also include determining one or more light sources based on a three-dimensional map or model of the scene created from the captured intermediate image. White balance coefficients associated with the detected at least one light source are determined (210). The determined white balance coefficients are then applied (212) to the captured primary image.

[0075] In one embodiment, when the intermediate image is captured, a three-dimensional model or map of the scene is created (220). Figure 4 In one embodiment, creating (220) the three-dimensional model or map of the scene may include, for example, stitching (224) at least two of the captured intermediate images together to create (226) a stitched image. The stitched image has a field of view that is greater than the field of view of the captured primary image. In this example, the light source calculated (222) may be determined based on the stitched image.

[0076] For example, in one embodiment, the 3D model is constructed using a SLAM 3D mapping process. This allows the camera to have a larger field of view than would otherwise be possible. The 3D model can provide reliable current and historical information to meet the needs of the automatic white balance system.

[0077] Various aspects of the disclosed embodiments can be used to improve the reliability of detecting the light source of a scene. For example, if the light source detection method of the disclosed embodiments is always used, the reliability of light source detection will be improved because the three-dimensional model provides more information. Alternatively, various aspects of the disclosed embodiments can only be used when a difficult scene is detected. A difficult scene can be a scene where the light source cannot be detected from the captured main image. An example of this is capturing a picture of a unique object with a constant color. In this case, the light source detection of the main image fails. By capturing intermediate images and creating a three-dimensional model, as described herein, the field of view is expanded and the reliability of light source detection is improved.

[0078] Aspects of the disclosed embodiments rely on current information related to the scenario rather than historical information. In one embodiment, reference Figure 5 The method includes: deleting (242) the three-dimensional model of the scene; and starting (244) capturing a new intermediate image of the scene after detecting the light source. In this way, timely updating of scene information is ensured.

[0079] In one embodiment, reference Figure 6 After creating the three-dimensional model and detecting the light source, detecting a change in the orientation of the image sensor. Deleting (254) any captured intermediate images. Capturing (256) at least one new intermediate image and creating a three-dimensional model of the scene from the at least one new intermediate image. In this way, timely updating of scene information is ensured, and old information is discarded.

[0080] Figure 7 A block diagram of an exemplary apparatus 1000 is shown that may be suitable for implementing aspects of the disclosed embodiments. Figure 1 The apparatus 100 shown in FIG. 1 may be Figure 7 A portion of the apparatus 1000, or comprising Figure 7 The device 1000. Figure 7 In the example of FIG, the apparatus 1000 includes or is coupled to a processor or computing hardware 102, a memory 1004, a radio frequency (RF) unit 1006, and a user interface (UI) 1008. An image sensor device 104 is also connected to the processor 102. Figure 7 Not shown, image sensor device 104 may include, comprise, or otherwise be connected to camera 106 .

[0081] Processor 1002 can be a single processing device or can include multiple processing devices, including specialized devices such as a digital signal processing (DSP) device, a microprocessor, a graphics processing unit (GPU), a dedicated processing device, or a general-purpose computer processing unit (CPU). Processor 1002 typically includes a CPU that works in conjunction with the DSP to handle signal processing tasks. Processor 102 is configured to implement any one or more of the methods described herein.

[0082] exist Figure 7 In the example of FIG. 1 , the processor 102 is coupled to a memory 1004. The memory 1004 can be a combination of various types of volatile and non-volatile computer memory, such as read-only memory (ROM), random access memory (RAM), a magnetic or optical disk, or other types of computer memory. The memory 1004 is used to store computer program instructions that can be accessed and executed by the processor 102 to cause the processor 102 to perform various desired computer-implemented processes or methods, such as the methods described herein.

[0083] The program instructions stored in memory 1004 are organized into sets or groups of program instructions, which are referred to variously in the industry as programs, software components, software modules, units, etc. Each module may include a set of functions designed to support a particular purpose. Memory 1004 also includes program data and data files that can be stored and processed by processor 102 when executing a set of computer program instructions.

[0084] The apparatus 1000 may also include or be coupled to an RF unit 1006, such as a transceiver, coupled to the processor 1002 for transmitting and receiving RF signals based on digital data 1012 exchanged with the processor 102, and may be configured to transmit and receive wireless signals with other nodes in a wireless network. In some embodiments, the RF unit 1006 includes a receiver capable of receiving and interpreting messages transmitted from satellites in the Global Positioning System (GPS), which are combined with information received from other transmitters to obtain positioning information related to the location of the computing device 1000. To facilitate the transmission and reception of RF signals, the RF unit 1006 includes an antenna unit 1010, which in some embodiments may include multiple antenna elements.

[0085] The UI 1008 may include one or more user interface elements, such as a touch screen, a keyboard, buttons, a voice command processor, and other elements suitable for exchanging information with a user. The UI 1008 may also include a display unit for displaying various information suitable for a computing device or mobile user device, and may be implemented by any suitable display type, such as an organic light emitting diode (OLED), a liquid crystal display (LCD), and less complex elements such as LEDs or indicator lights. For example, in one embodiment, an icon for the camera application 106 may be presented on the UI 1008.

[0086] Thus, while the basic novel features of the invention as applied to exemplary embodiments of the invention have been shown, described, and pointed out herein, it will be understood that various omissions, substitutions, and changes may be made to the form and details of the apparatus and methods shown, as well as to the configuration operations, by those skilled in the art without departing from the spirit and scope of the invention. Further, it is expressly intended that all combinations of those elements which perform substantially the same function in substantially the same manner to achieve the same results are within the scope of the invention. Furthermore, it will be appreciated that the structures and / or elements shown and / or described in conjunction with any form or embodiment of the invention disclosed may be incorporated into any other form or embodiment disclosed, described, or suggested as a general matter of design choice. It is therefore intended that the invention be limited only as indicated by the scope of the appended claims.

Claims

1. A device (100), characterized in that The invention comprises a processor (102), wherein the processor (102) is configured to: detecting activation of an image sensor (104) when a light source associated with capture of a primary image cannot be detected; automatically capturing at least one intermediate image of a scene (110) via the image sensor (104); detecting, by the image sensor (104), the capture of a primary image (112) of at least a portion of the scene (110); detecting at least one light source from visual information of the scene (110) associated with the captured at least one intermediate image, comprising: creating a three-dimensional model of the scene (110) based on the at least one intermediate image; calculating the at least one light source from the three-dimensional model; providing the calculated light source as the detected light source; and determining a white balance coefficient associated with the detected at least one light source; The determined white balance coefficients are applied to the captured main image.

2. The device (100) according to claim 1, characterized in that The processor (102) is configured to construct the visual information of the scene (110) by stitching together the main image and at least one captured intermediate image to create a stitched image, wherein the stitched image has a field of view that is greater than the field of view of the captured main image.

3. The device (100) according to any one of claims 1 or 2, characterized in that The processor (102) is configured to detect activation of the image sensor (104) from one or more of: activation of a camera application (106) of the device (100); movement of the device (100); a change in a frame view (108) of the image sensor (104); or framing of the primary image.

4. The device (100) according to claim 3, characterized in that The processor (102) is further configured to: delete the three-dimensional model of the scene (110); and start capturing a new intermediate image of the scene (110) after detecting the light source.

5. The device (100) according to any one of claims 3 or 4, characterized in that The processor (102) is further configured to: detect a change in orientation of the image sensor (104); delete the captured at least one intermediate image; capture at least one new intermediate image; and create the three-dimensional model of the scene (110) from the captured at least one new intermediate image.

6. The device (100) according to any one of the preceding claims, characterized in that By the image sensor (104) Automatically capturing the at least one intermediate image of the scene (110) includes creating a field of view of the scene that is greater than a field of view of the image sensor (104).

7. A method (200), characterized in that include: detecting (202) activation of an image sensor of the device when a light source associated with capture of a primary image is unable (236) to be detected; automatically capturing (204) at least one intermediate image of a scene via the image sensor; detecting (206) capture of a primary image of at least a portion of the scene by the image sensor; Detecting (208) at least one light source from visual information of the scene associated with the captured at least one intermediate image, comprising: creating a three-dimensional model or map of the scene based on the at least one intermediate image; calculating the at least one light source from the three-dimensional model or map; using the calculated light source as the light source for the detection (208); determining (210) white balance coefficients associated with said detected at least one light source; The determined white balance coefficients are applied (212) to the captured main image.

8. The method (200) according to claim 7, characterized in that Creating (220) the three-dimensional model or map further comprises: creating (226) a primary image and at least one captured intermediate image by stitching (224) the primary image and at least one captured intermediate image together. The images are stitched together to construct the visual information of the scene, wherein the field of view of the stitched image is greater than the field of view of the captured main image.

9. The method (200) according to claim 7 or 8, characterized in that Also includes: The activation of the image sensor is detected (202) from one or more of: initialization of a camera application of the device (231); manual activation of a camera (232); movement of the device (233); a change in the frame view of the image sensor (234); framing of the main image (235).

10. The method (200) according to claim 9, characterized in that Also includes: The three-dimensional model or map of the scene is deleted (242); and after detecting the light source, capturing a new intermediate image of the scene is started (244).

11. The method (200) according to any one of claims 7 to 10, characterized in that Also includes: detecting (252) a change in orientation of the image sensor; Deleting (254) the at least one captured intermediate image; capturing (256) at least one new intermediate image; and creating (258) the three-dimensional model or map of the scene from the at least one new captured intermediate image.

12. The method (200) according to any one of claims 7 to 11, characterized in that Automatically capturing the at least one intermediate image of the scene with the image sensor further comprises creating a field of view of the scene that is greater than a field of view of the image sensor.

Images