Method and electronic device for enhancing image frame

Abstract

Embodiments of the disclosure describe a method for enhancing the image frame. The method includes obtaining a plurality of image frames through one or more image sensors of an electronic device. The method includes extracting foreground objects from each of the obtained plurality of image frames. The method includes performing a noise reduction on the obtained plurality of image frames based on the extracted foreground objects. The method includes identifying one or more light sources in each of the plurality of noise-reduced image frames. The method includes obtaining one or more characteristics of each of the identified light sources for the extracted foreground objects. The method includes generating at least one of encapsulated light models for adjusting a dynamics of light within an image based on the one or more identified light sources and the one or more obtained characteristics. The method includes adjusting illumination of the one or more identified light sources using at least one of the generated encapsulated light models to enhance the image frame.

Description

METHOD AND ELECTRONIC DEVICE FOR ENHANCING IMAGE FRAME

[0001] The present invention generally relates to the field of image processing, and more specifically relates to a method and an electronic device for enhancing an image frame.

[0002] Image processing refers to manipulation of images using digital techniques to enhance a quality of an image or extract useful information from the image. In the context of electronic devices, the image processing is used to improve the quality of images taken by a camera of the electronic device (e.g., smartphone). The image processing may involve processes such as noise reduction, color correction, sharpening, and other adjustments to provide an enhanced image. The electronic device may utilize the image processing in various camera modes to cater to different photography scenarios. Examples of the various camera modes may include a pro mode, a panorama mode, a food mode, a night mode, a macro mode, and a portrait mode. These camera modes demonstrate how the electronic devices use the image processing to cater to various photography needs, providing users with a tool to capture high-quality images in different scenarios.

[0003] In general, the night mode is designed to capture images / photos in low-light situations by taking multiple shots at different exposure levels and combining the captured images to reduce noise and improve image characteristics. In addition, the night mode also requires a larger camera sensor to gather more light over a longer period to brighten up darker areas for a better user experience. However, several problems are encountered in the existing night mode of the electronic device.

[0004] One significant issue arises when the images are captured using the night mode in environments with inadequate illumination. An exposure from multiple light sources may adversely impact a background of the image, resulting in an artificial appearance as the processed image loses its original lighting dynamics. This may lead to the processed image resembling a daytime scene, despite being captured in low-light conditions or at night. Furthermore, the processed image may exhibit one or more random patches of colour that affect the overall background quality. Another significant issue is the presence of random noise in the images captured during video calls in the environments with insufficient illumination. This random noise may degrade the user experience by introducing visual disturbances during the video calls, impacting the overall quality of the communication.

[0005] This summary is provided to introduce a selection of concepts, in a simplified format, that are further described in the detailed description of the invention. This summary is neither intended to identify key or essential inventive concepts of the invention nor is it intended for determining the scope of the invention.

[0006] According to an embodiment of the present disclosure, a method for enhancing an image frame. The method includes obtaining a plurality of image frames through one or more image sensors of an electronic device. The method further includes extracting foreground objects from each of the obtained image frames. The method further includes performing a noise reduction on the obtained plurality of image frames. The method further includes identifying one or more light sources in each of the plurality of noise-reduced image frames. The method further includes obtaining one or more characteristics of each of the identified light sources for the extracted foreground objects. The method further includes generating at least one of the encapsulated light models for adjusting a dynamics of light with an image based on the one or more identified light sources and the one or more obtained characteristics. The method further includes adjusting an illumination of the one or more identified light sources using at least one of the generated encapsulated light models to enhance the image frame.

[0007] According to an embodiment of the present disclosure, an electronic device for enhancing the image frame is disclosed herein. The system may a memory storing one or more instructions, a communicator, a camera module, a display module and at least one processor configure to execute the one or more instructions stored in the memory. The processor is configured to execute the one or more instructions to obtain the plurality of image frames through one or more image sensors of the electronic device. The processor is further configured to execute the one or more instructions to perform a noise reduction the noise on the obtained plurality of image frames based on the extracted foreground objects. The processor is further configured to execute the one or more instructions to identify one or more light sources in each of the plurality of noise-reduced image frames. The processor is further configured to execute the one or more instructions to obtain one or more characteristics of each of the identified light sources for the extracted foreground objects. The processor is further configured to execute the one or more instructions to generate at least one of the encapsulated light models for adjusting a dynamics of light within an image based on the one or more identified light sources and the one or more obtained characteristics. The processor is further configured to execute the one or more instructions to adjust the illumination of one or more identified light sources using the at least one generated encapsulated light models to enhance the image frame.

[0008] According to an aspect of the present disclosure, a computer-readable storage medium storing instruction is provided. The instructions, when executed by at least one processor, may cause the at least one processor to perform the method corresponding.

[0009] To further clarify the advantages and features of the present invention, a more particular description of the invention will be rendered by reference to an embodiment thereof, which are illustrated in the appended drawings. It is appreciated that these drawings depict only an embodiment of the invention and are therefore not to be considered limiting of its scope. The invention will be described and explained with additional specificity and detail in the accompanying drawings.

[0010] These and other features, aspects, and advantages of the present invention will become better understood when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:

[0011] FIG. 1A illustrates a problematic scenario within the existing electronic device, wherein a user utilizes the night mode functionality to capture the image under low-light conditions, according to prior art;

[0012] FIG. 1B illustrates a graphical representation of a noise sampling process used in image processing, according to prior art;

[0013] FIG. 2 illustrates a block diagram of an electronic device for enhancing an image frame, according to an embodiment as disclosed herein;

[0014] FIG. 3 illustrates an example scenario where the electronic device aggregates one or more latent codes of each foreground object associated with a captured image frame, according to an embodiment as disclosed herein;

[0015] FIG. 4 illustrates an example scenario where the electronic device eliminates noise information from the captured image frame upon analyzing the aggregated latent code, according to an embodiment as disclosed herein;

[0016] FIGS. 5A-5B illustrates one or more operations associated with a training pipeline and a data collection procedure to eliminate the noise information from the captured image frame, according to an embodiment as disclosed herein;

[0017] FIG. 6 illustrates an example scenario where the electronic device performs one or more operations on the captured image frame to eliminate the noise information from the captured image frame, according to an embodiment as disclosed herein;

[0018] FIGS. 7A-7B example scenarios where the electronic device generates at least one encapsulated light model based on one or more light sources or one or more characteristics of the one or more light sources present in the captured image frame, according to an embodiment as disclosed herein;

[0019] FIGS. 8A, 8B, and 8C example scenarios where the electronic device adjusts illumination of each light source using the at least one generated encapsulated light model, according to an embodiment as disclosed herein; and

[0020] FIG. 9 is a flow diagram illustrating a method for enhancing the image frame, according to an embodiment as disclosed herein.

[0021] Further, skilled artisans will appreciate that elements in the drawings are illustrated for simplicity and may not have necessarily been drawn to scale. For example, the flow charts illustrate the method in terms of the most prominent steps involved to help to improve understanding of aspects of the present invention. In an embodiment, in terms of the construction of the device, one or more components of the device may have been represented in the drawings by conventional symbols, and the drawings may show only those specific details that are pertinent to understanding an embodiment of the present invention so as not to obscure the drawings with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.

[0022] For the purpose of promoting an understanding of the principles of the invention, reference will now be made to the embodiment illustrated in the drawings and specific language will be used to describe the same. It will nevertheless be understood that no limitation of the scope of the invention is thereby intended, such alterations and further modifications in the illustrated system, and such further applications of the principles of the invention as illustrated therein being contemplated as would normally occur to one skilled in the art to which the invention relates.

[0023] It will be understood by those skilled in the art that the foregoing general description and the following detailed description are explanatory of the invention and are not intended to be restrictive thereof.

[0024] Reference throughout this specification to “an aspect”, “another aspect” or similar language means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, appearances of the phrase “in an embodiment”, “in one embodiment”, “in another embodiment”, and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment.

[0025] The terms “comprise”, “comprising”, or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process or method that comprises a list of steps does not include only those steps but may include other steps not expressly listed or inherent to such process or method. Similarly, one or more devices or sub-systems or elements or structures or components proceeded by “comprises... a” does not, without more constraints, preclude the existence of other devices or other sub-systems or other elements or other structures or other components or additional devices or additional sub-systems or additional elements or additional structures or additional components.

[0026] The embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein. Also, the various embodiments described herein are not necessarily mutually exclusive, as some embodiments may be combined with one or more other embodiments to form new embodiments. The term “or” as used herein, refers to a non-exclusive or unless otherwise indicated. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein may be practiced and to further enable those skilled in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.

[0027] As is traditional in the field, an embodiment may be described and illustrated in terms of blocks that carry out a described function or functions. These blocks, which may be referred to herein as units or modules or the like, are physically implemented by analog or digital circuits such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hardwired circuits, or the like, and may optionally be driven by firmware and software. The circuits may, for example, be embodied in one or more semiconductor chips, or on substrate supports such as printed circuit boards and the like. The circuits constituting a block may be implemented by dedicated hardware, or by a processor (e.g., one or more programmed microprocessors and associated circuitry), or by a combination of dedicated hardware to perform some functions of the block and a processor to perform other functions of the block. Each block of an embodiment may be physically separated into two or more interacting and discrete blocks without departing from the scope of the invention. Likewise, the blocks of an embodiment may be physically combined into more complex blocks without departing from the scope of the invention.

[0028] The accompanying drawings are used to help easily understand various technical features and it should be understood that an embodiment presented herein are not limited by the accompanying drawings. As such, the present disclosure should be construed to extend to any alterations, equivalents, and substitutes in addition to those which are particularly set out in the accompanying drawings. Although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are generally only used to distinguish one element from another.

[0029] Referring now to the drawings, and more particularly to FIGS. 1 to 9, where similar reference characters denote corresponding features consistently throughout the figures, there are shown preferred an embodiment.

[0030] FIG. 1A illustrates a problematic scenario within the existing electronic device, wherein a user utilizes the night mode functionality to capture the image under low-light conditions, according to prior art.

[0031] Referring to FIG. 1A, the electronic device may display images taken in various camera modes. In an embodiment, the camera modes may include a preview mode and a night mode. In an embodiment, the preview mode 11 may be an image capturing mode that displays the captured images in real time. For example, when the camera application is executed, the electronic device may determine the execution mode of the camera application to be the preview mode 11. In the preview mode 11, images captured through the camera module of the electronic device may be displayed through the display module of the electronic device. In an embodiment, the preview mode may display images obtained through the camera module in real time or with a short delay by setting a short exposure time for the image sensor of the camera module or by performing minimal image processing, compared to the night mode described below.

[0032] In an embodiment, the images displayed in the night mode 13 may differ when compared to the images displayed in the preview mode 11, which is similar to the actual real-world environment where the images were taken. For example, the images displayed in the night mode 13 may appear brighter than the actual environment due to the loss of lighting dynamics within the image caused by the long exposure time. Additionally, the images displayed in the night mode 13 may include random noise generated during the image processing. That is, although the images displayed in the night mode 13 have the advantage of improved brightness and detail compared to the images displayed in the preview mode 11, several problems may arise. The electronic device according to an embodiment of this disclosure may solve the above problems through a method for enhancing the image frame. Hereafter, a detailed description will be provided with reference to the drawings.FIG. 1B illustrates a graphical representation of a noise sampling process used in image processing, according to prior art.

[0033] Referring to FIG. 1B, the electronic device may obtain an image 14 that includes noise. In an embodiment, the image 14 that includes noise may be an image captured in a low-light environment, thereby including noise. In an embodiment, the electronic device may sample the noise components of the image 14 that includes noise to obtain a complex noise model 15. In this case, the obtained complex noise model 15 may include different noise characteristics compared to the previously known common noise models 16. In other words, the image 14 that includes noise, obtained by capturing images in a low-light environment with the electronic device, may include noise components generated according to the specifications of the camera module of the electronic device or the individual conditions of the low-light environment. Therefore, it may have different noise characteristics compared to the noise of the common noise models 16. Accordingly, the electronic device according to an embodiment of this disclosure may reduce the noise included in the image 14 that includes noise using a method according to an embodiment of this disclosure, rather than using the common noise models 16 described later.

[0034] FIG. 2 illustrates a block diagram of an electronic device 200 for enhancing an image frame, according to an embodiment as disclosed herein. Examples of the electronic device 200 include, but are not limited to a smartphone, a tablet computer, a Personal Digital Assistance (PDA), an Internet of Things (IoT) device, a wearable device, etc.

[0035] In an embodiment, the electronic device 200 comprises a system 201. The system 201 may include a memory 210, a processor 220, a communicator 230, a camera module 240, and a display module 250. In an embodiment, the system 201 may be implemented on one or multiple electronic devices (not shown in FIG.2). The electronic device 200, the memory 210, the processor 220, the communicator 230, the camera module 240, and the display module 250 may be electrically and / or physically connected to each other.

[0036] In an embodiment, the memory 210 stores one or more instructions to be executed by the processor 220 for enhancing the image frame, as discussed throughout the disclosure. The memory 210 may include non-volatile storage elements. Examples of such non-volatile storage elements may include magnetic hard discs, optical discs, floppy discs, flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable (EEPROM) memories. In addition, the memory 210 may, in some examples, be considered a non-transitory storage medium. The term “non-transitory” may indicate that the storage medium is not embodied in a carrier wave or a propagated signal. However, the term “non-transitory” should not be interpreted that the memory 210 is non-movable. In some examples, the memory 210 may be configured to store larger amounts of information than the memory. In certain examples, a non-transitory storage medium may store data that can, over time, change (e.g., in Random Access Memory (RAM) or cache). The memory 210 may be an internal storage unit, or it may be an external storage unit of the electronic device 200, a cloud storage, or any other type of external storage.

[0037] The processor 220 communicates with the memory 210, the communicator 230, the camera module 240, and the display module 250. The processor 220 is configured to execute instructions stored in the memory 210 and to perform various processes for enhancing the image frame, as discussed throughout the disclosure. The processor 220 may include one or a plurality of processors, maybe a general-purpose processor, such as a central processing unit (CPU), an application processor (AP), or the like, a graphics-only processing unit such as a graphics processing unit (GPU), a visual processing unit (VPU), and / or an artificial intelligence (AI) dedicated processor such as a neural processing unit (NPU).

[0038] The processor may include various processing circuitry and / or multiple processors.　 For example, as used herein, including the claims, the term “processor” may include various processing circuitry, including at least one processor, wherein one or more of at least one processor, individually and / or collectively in a distributed manner, may be configured to perform various functions described herein. As used herein, when “a processor”, “at least one processor”, and “one or more processors” are described as being configured to perform numerous functions, these terms cover situations, for example and without limitation, in which one processor performs some of recited functions and another processor(s) performs other of recited functions, and also situations in which a single processor may perform all recited functions. Additionally, the at least one processor may include a combination of processors performing various of the recited / disclosed functions, e.g., in a distributed manner.　 At least one processor may execute program instructions to achieve or perform various functions.

[0039] In an embodiment, the processor 220 may include an object extraction module 221, a latent code estimation module 222, a graininess identification and correction module 223, an encapsulation generation module 224, and a scene adjustment module 225. However, it is not limited to these modules, and at least one of the object extraction module 221, latent code estimation module 222, graininess identification and correction module 223, encapsulation generation module 224, and scene adjustment module 225 may be stored in the memory 210, and the processor 220 may execute at least one of the modules stored in the memory 210. The processor 220 is implemented by processing circuitry such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hardwired circuits, or the like, and may optionally be driven by firmware. The circuits may, for example, be embodied in one or more semiconductor chips, or on substrate supports such as printed circuit boards and the like. Hereinafter, the operations and functions of the object extraction module 221, latent code estimation module 222, graininess identification and correction module 223, encapsulation generation module 224, and scene adjustment module 225 may be understood as operations and functions performed by the processor 220.

[0040] In an embodiment, the object extraction module 221 is configured to obtain a plurality of image frames through one or more image sensors (i.e., camera module 240) of the electronic device 200. The plurality of image frames is received in an image-capturing mode associated with the electronic device 200. The image-capturing mode may include, but is not limited to, an auto mode, a portrait mode, a landscape mode, a macro mode, a night mode, a sport mode, and a normal mode. In an embodiment, the object extraction module 221 is configure to extract foreground objects from the plurality of image frames. In other words, the object extraction module 221 is further configured to extract, in response to obtaining the plurality of image frames, one or more foreground objects associated with each captured image frame, as described in conjunction with FIG. 3.

[0041] For instance, in a smartphone, the object extraction module 221 may use the front and rear cameras to capture multiple image frames and then apply image processing techniques (e.g., edge detection, region-based segmentation, etc.) to isolate and extract foreground objects, such as people, vehicles, or other prominent elements within the image.

[0042] In an embodiment, the latent code estimation module 222 is configured to obtain latent codes for each of the extracted foreground objects. The latent code serves as compact representations of one or more visual features and characteristics of each foreground object, facilitating efficient storage and further processing. The latent code estimation module 222 is further configured to aggregate the determined latent code of each foreground object of each captured image frame into a unified representation and stored in the memory 210 for further processing, as described in conjunction with FIG. 3.

[0043] For instance, in an autonomous driving system, the latent code estimation module 222 may process input from multiple onboard cameras to calculate latent codes for different types of objects, such as vehicles, pedestrians, and cyclists. These latent codes are then utilized to create a unified representation of the surrounding environment, enabling a vehicle's perception system to make informed decisions based on the aggregated information.

[0044] In one embodiment, the latent code is generated by utilizing the at least one Artificial Intelligence (AI) model.

[0045] In one embodiment, the latent code is utilized to manipulate and generate one or more new image frames with the one or more foreground objects while preserving other aspects of each captured image frame.

[0046] In an embodiment, the graininess identification and correction module 223 is configured to performing a noise reduction on the obtained plurality of image frames based on the extracted foreground objects. In an embodiment, the obtained plurality of image frames may include graininess, such as fine dots or patterns that appear as irregular brightness or color variations, which may be referred to as noise or noise information. In an embodiment, the graininess identification and correction module 223 may perform noise reduction by eliminating the graininess present in the image. In an embodiment, the graininess identification and correction module 223 is configure to analyze the aggregated latent code of each foreground object of each captured image frame. The graininess identification and correction module 223 is further configured to determine whether the captured plurality of image frames comprises the noise based on a result of analysis of the aggregated latent code. The at least one AI model determines patterns that differentiate between the image frame affected by the noise or the image frame that is free from the noise. The graininess identification and correction module 223 is further configured to reduce in response to determining that the captured plurality of image frames comprises the noise, the noise from the captured plurality of image frames by performing one or more operations on the captured plurality of image frames, as described in conjunction with FIG. 4, FIGS. 5A-5B, and FIG. 6.

[0047] In an embodiment, the encapsulation generation module 224 is configured to identify one or more light sources in each of the plurality of noise-reduced image frames. In an embodiment, the encapsulation generation module 224 is further configured to obtain and one or more characteristics of each of the identified light source for the extracted foreground objects. In an embodiment, the one or more characteristics may include, but are not limited to, orientation information, reflection information, and absorbance information. The encapsulation generation module 224 is further configured to generate at least one encapsulated light model for adjusting the dynamics of light within an image based on the at least one of one or more determined light sources and one or more determined characteristics, as described in conjunction with FIGS. 7A-7B.

[0048] In an embodiment, the encapsulated light model may include information related to the light dynamics of the scene within the image. In an embodiment, light dynamics include information about the interaction between the light source and the foreground object within the image. In other words, light dynamics include information about how the characteristics of the light source (e.g., direction, absorbance, and reflectance) for the foreground object are applied within the image. In an embodiment, the encapsulated light model may obtain information related to the light dynamics based on the identified foreground object and the characteristics of the lighting obtained within the image. In an embodiment, the encapsulated light model may adjust the illumination of the lighting within the image by changing he obtained information related to the light dynamics and reflecting the changed information in the image.

[0049] In an embodiment, the scene adjustment module 225 is configured to adjust the illumination of the one or more identified light sources using at least one of generated encapsulated light models to enhance the image frame, as described in conjunction with FIGS. 8A, 8B and 8C.

[0050] The communicator 230 is configured for communicating internally between internal hardware components and with external devices (e.g., server) via one or more networks (e.g., radio technology). The communicator 230 includes an electronic circuit specific to a standard that enables wired or wireless communication.

[0051] The camera module 240 includes one or more image sensors (e.g., Charged Coupled Device (CCD), Complementary Metal-Oxide Semiconductor (CMOS)) to capture one or more images / image frames / video to be processed for enhancing the image frame. In an alternative embodiment, the camera module 240 may not be present, and the system 201 may process an image / video received from an external device or process a pre-stored image / video displayed at the display module 250.

[0052] The display module 250 is configured to accept user inputs and is made of a Liquid Crystal Display (LCD), a Light Emitting Diode (LED), an Organic Light Emitting Diode (OLED), or another type of display. The user inputs may include, but are not limited to, touch, swipe, drag, gesture, and so on.

[0053] A function associated with the various components of the electronic device 200 may be performed through the non-volatile memory, the volatile memory, and the processor 220. One or a plurality of processors controls the processing of the input data in accordance with a predefined operating rule or AI model stored in the non-volatile memory and the volatile memory. The predefined operating rule or AI model is provided through training or learning to enhance the image frame. Here, being provided through learning means that, by applying a learning algorithm to a plurality of learning data, a predefined operating rule or AI model of the desired characteristic is made. The learning may be performed in a device itself in which AI according to an embodiment is performed, and / or may be implemented through a separate server / system. The learning algorithm is a method for training a predetermined target device (for example, a camera device) using a plurality of learning data to cause, allow, or control the target device to decide or predict. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0054] The AI model may consist of a plurality of neural network layers. Each layer has a plurality of weight values and performs a layer operation through a calculation of a previous layer and an operation of a plurality of weights. Examples of neural networks include, but are not limited to, convolutional neural network (CNN), deep neural network (DNN), recurrent neural network (RNN), restricted Boltzmann Machine (RBM), deep belief network (DBN), bidirectional recurrent deep neural network (BRDNN), generative adversarial networks (GAN), and deep Q-networks.

[0055] Although FIG. 2 shows various hardware components of the electronic device 200, but it is to be understood that an embodiment is not limited thereon. In an embodiment, the electronic device 200 may include less or more number of components. Further, the labels or names of the components are used only for illustrative purposes and do not limit the scope of the invention. One or more components may be combined to perform the same or substantially similar functions to enhance the image frame.

[0056] In an embodiment, the electronic device 200 is configured to identify a presence of at least one image frame stored within the electronic device 200. The electronic device 200 is further configured to receive at least one user input (e.g., click command, voice command, etc.) to enhance the at least one identified image frame. The electronic device 200 is further configured to reduce, upon reception of the at least one user input, the noise from at least one of the identified image frame. The electronic device 200 is further configured to identify one or more light sources in at least one of the noise-reduced image frame. In an embodiment, the electronic device 200 is further configured to obtain one or more characteristics of each of the identified light sources for the extracted foreground objects. The electronic device 200 is further configured to generate at least one of the encapsulated light models for adjusting the dynamics of light within an image based on the one or more identified light sources and the one or more obtained characteristics. The electronic device 200 is further configured to adjust the illumination of the one or more identified light sources using at least one of the generated encapsulated light models to enhance the at least one image frame.

[0057] In an embodiment, the electronic device 200 is configured to identify a presence of at least one image frame displayed on a screen of the electronic device 200 (e.g., display module 250). The electronic device 200 is further configured to receive the at least one user input to enhance the at least one displayed image frame. The electronic device 200 is further configured to reduce, upon reception of the at least one user input, the noise from at least one of the displayed image frame. The electronic device 200 is further configured to identify one or more light sources in at least one of the noise-reduced image frame. In an embodiment, the electronic device 200 is further configured to obtain the one or more characteristics of each of the identified light sources for the extracted foreground objects. The electronic device 200 is further configured to generate at least one of encapsulated light models for adjusting the dynamics of light within an image based on the one or more identified light sources and one or more obtained characteristics. The electronic device 200 is further configured to adjust the illumination of the one or more identified light sources using at least one of the generated encapsulated light models to enhance the at least one image frame.

[0058] FIG. 3 illustrates an example scenario where the electronic device 200 aggregates one or more latent codes of each foreground object associated with the image, according to an embodiment as disclosed herein, by performing several operations outlined as follows.

[0059] At operations 301-302, the electronic device 200 is configured to capture multiple image frames using one or more image sensors (i.e., the camera module 240), such as telephoto, wide-angle, and ultra-wide-angle lenses. Each frame may contain various objects, such as a person sitting on a rooftop, a building, a tree, and so on. Each image frame captured by a specific lens exhibits distinct characteristics. For instance, the image frame taken by the telephoto lens displays a bright effect on the object, the image frame taken by the wide-angle lens shows a less pronounced dark effect on the object, and the image frame taken by the ultra-wide-angle lens exhibits a very pronounced dark effect on the object.

[0060] The electronic device 200 is further configured to extract the one or more foreground objects associated with the frame. Extraction of one or more foreground objects may involve employing advanced image processing techniques such as semantic segmentation or instance segmentation. The semantic segmentation entails classifying each pixel in the image frame to determine the foreground object, while the instance segmentation further distinguishes individual objects of the same class. These techniques leverage neural network models, such as convolutional neural networks (CNNs) or more advanced architectures like Mask R-CNN, to accurately delineate and extract the one or more foreground objects from the captured image frames. In other words, the aforementioned process requires the electronic device 200 to analyze and interpret visual data, effectively isolating the one or more foreground objects from a background within each frame.

[0061] At operations 303-304, subsequently, the electronic device 200 is configured to employ a neural network model to determine a latent code for each foreground object within the captured image frames, utilizing image reconstruction for latent code estimation. The determined latent codes representing the essential characteristics of the person across all captured frames are aggregated into a unified representation, enabling comprehensive encoding and analysis of the person's visual attributes and movements within the captured image frames.

[0062] At operation 305, the electronic device 200 is further configured to aggregate the determined latent codes of each foreground object from every captured image frame into a unified representation. The unified representation serves as a comprehensive encoding of the foreground objects across all captured image frames, enabling efficient storage, retrieval, and analysis of the underlying visual information. The aggregation process encompasses mathematical operations or data fusion techniques to integrate the latent codes. This process involves aggregating the latent codes of individual objects obtained from a multi-lens setup, accounting for variations in luminous intensities and colors across the different lenses.

[0063] FIG. 4 illustrates an example scenario where the electronic device 200 performing a noise reduction from the image upon analyzing the aggregated latent code, according to an embodiment as disclosed herein, by performing several operations outlined as follows.

[0064] At operations 401-402, the electronic device 200 is configured to conduct an analysis of the aggregated latent code corresponding to each foreground object within the captured image frames to ascertain the presence of noise information. This analysis involves the utilization of at least one Artificial Intelligence (AI) model functioning as a “classifier”. The AI model is configured to discern patterns that differentiate between image frames affected by noise information (referred to as “noisy images”) and those that are devoid of such noise (referred to as “non-noisy images”).

[0065] At operations 403-404, the electronic device 200 is configured to implement a noise elimination process upon determining the presence of noise within the captured plurality of image frames. This process involves the application of a luminance channel-based de-noising procedure and a chroma channel-based de-noising procedure, both of which are customized to provide adaptive denoising through the incorporation of a strength parameter.

[0066] A luminance channel may include important structural details like edges and textures in the image. By focusing on the luminance channel, the luminance channel-based de-noising procedure aims to keep the image's essential structure intact while reducing the impact of noise (noise information); and

[0067] A chroma channel may include vital color information such as hues and saturation., The noise in the chroma channel is more visible and may affect the image's color quality. By focusing on the chroma channel, the chroma channel-based de-noising procedure aims to keep the image's essential color accuracy or to maintain accurate and natural colors while reducing the impact of noise (noise information).

[0068] By addressing noise separately in both the luminance and chroma channels, the electronic device 200 may effectively reduce unwanted noise while preserving the image's structure and color accuracy or maintain accurate and natural colors.

[0069] At operation 405, after processing both channels, the electronic device 200 may combine the results to create a final noise-free image, ensuring the visual data remains clear and free from interference / noise information.

[0070] FIGS. 5A-5B illustrates one or more operations associated with a training pipeline 502 and a data collection procedure 501 to reduce the noise from the image, according to an embodiment as disclosed herein.

[0071] Referring to FIG. 5A: In order to mitigate noise in the image and determine the impact of noise on the captured plurality of image frames, the AI model undergoes training using one or more datasets. Each dataset encompasses the plurality of image frames with distinct illumination characteristics within the same environment, spanning low, mid, and high illumination ranges.

[0072] For instance, the process of creating these datasets (data collection procedure 501) involves several operations, including setting up 200 different environments with controlled lighting conditions, gradually adjusting the brightness levels in the environment to capture image frames, and labeling the images as noisy or noise-free. In an embodiment, the data collection procedure 501entails changing flagship electronic devices and capturing images for 200 different subjects (e.g., person), resulting in a total of 400,000 images (200 environments * 200 subjects * 10 images per subject). This comprehensive training approach ensures that the AI model is adept at recognizing and distinguishing noise patterns across varying illumination conditions, leading to robust noise elimination and noise impact assessment capabilities.

[0073] Referring to FIG. 5B: in one example scenario 502, the training pipeline incorporates a lightweight MobileNet-based classifier, which may relate to the at least one AI model, designed to classify images as either noisy or non-noisy. The classifier receives input images with dimensions of 1024x1024 and processes them through a series of fully connected nodes, specifically N=32, N=96, and N=1280, culminating in a SoftMax layer with two classes: c1 for noisy images and c2 for non-noisy images.

[0074] The training pipeline further specifies the utilization of a loss function to minimize errors, as shown in the below equation-1, with a batch size of 4 and a total of 1000 epochs for training. In an embodiment, the base learning rate is set at 0.0001, and the training optimization is facilitated through the tf.keras.optimizers. RMSprop optimizer. The dataset is divided into training, testing, and validation subsets, with the training dataset accounting for 80% of the data, the testing dataset for 10%, and the validation dataset for the remaining 10%. This meticulous training configuration ensures the effective training of the classifier to accurately discern between noisy and non-noisy images while optimizing performance and generalization capabilities.

[0075] (1)

[0076] After the image has been classified as a noisy image, similar patches are determined. Those similar patches are sent for the denoising step with an added strength parameter, as described in conjunction with FIG. 6.

[0077] FIG. 6 illustrates an example scenario where the electronic device 200 performs one or more operations on the image to reduce the noise from the image, according to an embodiment as disclosed herein, by performing several operations outlined as follows.

[0078] At operation 601, the electronic device 200 is configured to implement the noise elimination process upon determining the presence of noise within the captured plurality of image frames. This process involves the application of a luminance channel-based de-noising procedure 601a and a chroma channel-based de-noising procedure 601b, both of which are customized to provide adaptive denoising through the incorporation of an adaptive strength parameter.

[0079] At operations 602-605, the electronic device 200 is configured to identify a plurality of pixels in each image frame that is affected by the noise information in each of the plurality of channels, each of the plurality of channels comprises one of the luminance channel and the chroma channel. The plurality of pixels has the same value. The electronic device 200 is configured to group the identified plurality of pixels together based on a common value or according to intensity.

[0080] At operations, 603-604 and 606-607, the electronic device 200 is configured to apply the adaptive strength parameter for levelling process on each group. The adaptive strength parameter is determined based on the values of the pixels within the group. The electronic device 200 is configured to eliminate the noise information from each image frame based on the levelling process applied to each group (e.g., levelling out the same patches using local means).

[0081] In the illustrated example scenario, when a user takes a selfie using a smartphone camera (i.e., electronic device 200) in a low-light environment, resulting in noticeable noise effects near the user's face in the captured image. The electronic device 200 identifies numerous pixels affected by noise in the luminance and chroma channels, particularly near the user's face. Upon detection of these noisy pixels, the electronic device 200 groups them together based on their common value or intensity. Subsequently, the electronic device 200 applies the adaptive strength parameter for a leveling process on each group, tailoring the noise mitigation process to the specific characteristics of the noise near the user's face. Following this adaptive strength parameter approach, the electronic device 200 effectively eliminates the noise effects from the selfie image by applying the leveling process to each group. For example, the electronic device 200 may utilize advanced techniques such as local means to equalize the noisy patches, resulting in a significant reduction of noise effects and an overall enhancement of the visual quality of the selfie image, particularly near the user's face. At operations 608-609, after processing both channels, the electronic device 200 may combine the results and perform Red, Green, and Blue (RGB) conversion to create a final noise-free image, ensuring the visual data remains clear and free from interference / noise information.

[0082] FIGS. 7A-7B example scenarios where the electronic device 200 generates the at least one encapsulated light model based on the one or more light sources or the one or more characteristics of the one or more light sources present in the image, according to an embodiment as disclosed herein, by performing several operations outlined as follows.

[0083] Referring to FIG. 7A: at operation 701, consider a situation where the user of the electronic device 200 intends to capture the image of a person walking on a street at night, with one or more street lamps providing illumination. This captured image, referred to as a “truth image” or preview image, contains authentic data regarding the dynamics of the light source. In an embodiment, the “truth image” or preview image may correspond to an image displayed when the camera's execution mode of the electronic device is in preview mode. The electronic device 200 is configured to analyze the impact of each light source on various objects (e.g., a person) within the preview image. At operation 702, subsequently, the electronic device 200 may isolate one or more foreground objects present in the preview image, captured in the preview mode. At operations 703-703a and 704, once the electronic device 200 has comprehended the influence of each light source and extracted the foreground objects, the electronic device 200 may ascertain the characteristics of each light source, including orientation, reflection, and absorbance information. At operations 705, leveraging these determined characteristics, the electronic device 200 may determine the contribution and strength of each light source that affects the foreground objects in the preview mode. At operations 706, ultimately, the electronic device 200 may generate a first encapsulated light model based on the determined contribution and strength values of each light source on each foreground object in the preview mode, encapsulating crucial information about the light sources and their impact on the objects within the image.

[0084] Referring to FIG. 7B: at operation 707, consider a situation where the user of the electronic device 200 intends to capture the image of the person walking on the street at night, with one or more street lamps providing illumination. This image is captured in the night mode, which has artificial values of light dynamics. The electronic device 200 is configured to analyze the impact of each light source on various objects (e.g., the person) within the image. At operation 708, subsequently, the electronic device 200 may isolate one or more foreground objects present in the captured image. At operations 709-709a and 710, once the electronic device 200 has comprehended the influence of each light source and extracted the foreground objects, the electronic device 200 may ascertain the characteristics of each light source, including orientation, reflection, and absorbance information. At operations 711, leveraging these determined characteristics, the electronic device 200 may determine the contribution and strength of each light source that affects the foreground objects in the preview mode. At operations 712-713, further, the electronic device 200 may determine an aesthetic score for each light source to evaluate whether each light source is perceived as artificial or real from an illumination perspective. At operations 714, ultimately, the electronic device 200 may generate a second encapsulated light model based on the determined contribution value, the determined strength value of each light source on each foreground object, and the determined aesthetic score, encapsulating crucial information about the light sources and their impact on the objects within the image.

[0085] In an embodiment, the electronic device 200 may adjust the illumination of each light source using the at least one generated encapsulated light model (e.g., 706 and 714) to enhance the image frame, as described in conjunction with FIGS. 8A, 8B, and 8C.

[0086] FIGS. 8A, 8B, and 8C example scenarios where the electronic device 200 adjusts the illumination of each light source using the at least one generated encapsulated light model, according to an embodiment as disclosed herein.

[0087] Referring to FIG. 8A: after generating the encapsulated light model (e.g., 706 and 714) of the input image 801 captured in low-light conditions, the electronic device 200 may conduct an analysis 802 to assess the aesthetic score of each detected light source in the input image 801. Subsequently, the electronic device 200 may compare 803 the determined aesthetic score of each light source with a predefined threshold value, such as 0.7. In the event that the aesthetic score of any light source falls below the predefined threshold value, the electronic device 200 may identify the light source 804 as exhibiting an artificial appearance similar to that of a daytime capture, thereby compromising the overall user experience. To address this, the electronic device 200 must make adjustments to at least one value associated with the identified light source 804 (outlier detection), as described in conjunction with FIGS. 8B.

[0088] Referring to FIG. 8B: based on the identified light source 804, the electronic device 200 may vary at least one of a temperature value for each light source (805 and 805a), a contrast value for each light source (806 and 806a), and a saturation value for each light source (807 and 807a) that has the aesthetic score lower than the pre-defined threshold value to adjust the illumination (outlier adjustment).

[0089] Referring to FIG. 8C: for light sources with low aesthetic scores, they are categorized as outliers, by the electronic device 200, and undergo adjustments 807a, by the electronic device 200, using temperature, contrast, and saturation controls. Subsequently, the adjusted outliers are reintegrated into the original image 707. In order to achieve a realistic appearance, the remaining elements of the scene are also replaced / modified (808 and 809). Following the replacement of the outliers, it is imperative to make further adjustments 810, by the electronic device 200, to a final image 811 to avoid an artificial appearance. This involves modifying the illumination of each light source using the encapsulated model 714.

[0090] As a result, there are several advantages of the disclosed method, the final image 811 appears more realistic and authentic, resembling a natural low-light environment rather than an artificial or overly manipulated one, and the overall visual experience (e.g., video call) for users is enhanced, resulting in more pleasing images.

[0091] FIG. 9 is a flow diagram illustrating a method 900 for enhancing the image frame, according to an embodiment as disclosed herein. The method 900 may execute multiple operations to enhance the image, which are given below.

[0092] At operation 901, the method 900 includes obtaining the plurality of image frames through the one or more image sensors of the electronic device 200. At operation 902, the method 900 includes extracting foreground objects from each of the obtained plurality of image frames. At operation 903, the method 900 includes performing the noise reduction on the obtained plurality of image frames. At operation 904, the method 900 includes identifying the one or more light sources in each of the plurality of noise-reduced image frames. At operation 905, the method 900 includes obtaining the one or more characteristics of each light source for the extracted foreground obeject. At operation 906, the method 900 includes generating at least one of the encapsulated light models for adjusting the dynamics of light within the image based on the one or more identified light sources and the one or more obtained characteristics. At operation 907, the method 900 includes adjusting the illumination of the one or more identified light sources using at least one of generated encapsulated light models to enhance the image frame. Further, a detailed description related to the various operations of FIG. 9 is covered in the description related to FIG. 3 to FIG. 8C, and is omitted herein for the sake of brevity.

[0093] According to an embodiment of the disclosure, wherein the one or more characteristics comprise orientation information, reflection information, and absorbance information.

[0094] According to an embodiment of the disclosure, the method may include obtaining a latent code for the extracted foreground objects. The method may include aggregating the obtained latent codes into an unified representation. The method may include determining whether the obtained plurality of image frames comprises the noise based on the aggregated latent code. The method may include reducing, in response to determining that the plurality of obtained image frames comprises the noise, the noise from the obtained plurality of images frames.

[0095] According to an embodiment of the disclosure, the method may include identifying a plurality of pixels in each of the obtained plurality of image frames that is affected by the noise in each of a plurality of channels, wherein each of the plurality of channels comprising one of a luminance channel and a chroma channel, wherein the plurality of pixels have same value. The method may include grouping the identified plurality of pixels together based on a common value. The method may include applying an adaptive strength parameter for levelling process on the each group, wherein the adaptive strength parameter is determined based on the values of the pixels within the group. The method may include reducing the noise from each of the plurality of image frames based on the levelling process applied to each group.

[0096] According to an embodiment of the disclosure, the method may include determining a contribution value and a strength value of each of the identified light sources affecting the extracted foreground objects in a first camera mode of the electronic device (200) based on the one or more obtained characteristics, wherein the first camera mode is an image-capturing mode for displaying captured images in real-time. The method may include generating a first encapsulated light model based on the determined contribution value and the determined strength value.

[0097] According to an embodiment of the disclosure, the method may include determining a contribution value and a strength value of each of the identified light sources affecting the extracted foreground objects in a second camera mode of the electronic device (200) based on the one or more obtained characteristics, wherein the second camera mode is an image-capturing mode for displaying a capture image processed through at least one of long exposure times or multi-frame composition. The method may include determining an aesthetic score for each of the identified light sources to evaluate whether each of the identified light sources is perceived as artificial or real from an illumination perspective. The method may include generating a second encapsulated light model based on the determined contribution value, the determined strength value, and the determined aesthetic score.

[0098] According to an embodiment of the disclosure, the method may include detecting that the aesthetic score is lower than a pre-defined threshold value. The method may include varying at least one of a temperature value, a contrast value, and a saturation value for each of the detected light sources that has the aesthetic score lower than the pre-defined threshold value to adjust the illumination.

[0099] According to an embodiment of the disclosure, the electronic device for enhancing an image frame may include a memory storing one or more instructions, a communicator, a camera module, a display module and at least one processor configure to execute the one or more instructions stored in the memory. The at least one processor may be configured to execute the one or more instructions to obtain a plurality of image frames through one or more image sensors of an electronic device. The at least one processor may be configured to perform a noise reduction on the obtained plurality of image frames based on the extracted foreground objects. The at least one processor may be configured to execute the one or more instructions to extract foreground objects from each of the obtained plurality of image frames. The at least one processor may be configured to execute the one or more instructions to identify one or more light sources in each of the plurality of noise-reduced image frames. The at least one processor may be configured to execute the one or more instructions to obtain one or more characteristics of each of the identified light sources for the extracted foreground objects. The at least one processor may be configured to execute the one or more instructions to generate at least one of encapsulated light models for adjusting a dynamics of light within an image based on the one or more identified light sources and the one or more obtained characteristics. The at least one processor may be configured to execute the one or more instructions to adjust illumination of the one or more identified light sources using at least one of the generated encapsulated light models to enhance the image frame

[0100] According to an embodiment of the disclosure, wherein the one or more characteristics comprise orientation information, reflection information, and absorbance information.

[0101] According to an embodiment of the disclosure, the at least one processor may be configured to execute the one or more instructions to obtain a latent code of the extracted foreground objects. The at least one processor may be configured to execute the one or more instructions to aggregate the obtained latent codes into an unified representation. The at least one processor may be configured to execute the one or more instructions to determine whether the obtained plurality of image frames comprises the noise based on the aggregated latent code. The at least one processor may be configured to execute the one or more instructions to reduce, in response to determine that the obtained plurality of image frames comprises the noise, the noise from the obtained plurality of image frames.

[0102] According to an embodiment of the disclosure, the at least one processor may be configured to execute the one or more instructions to identify a plurality of pixels in each of the obtained plurality of image frames that is affected by the noise in each channel, wherein each channel comprising one of a luminance channel and a chroma channel, wherein the plurality of pixels have same value. The at least one processor may be configured to execute the one or more instructions to group the identified plurality of pixels together based on a common value. The at least one processor may be configured to execute the one or more instructions to apply an adaptive strength parameter for levelling process on each group, wherein the adaptive strength parameter is determined based on the values of the pixels within the group. The at least one processor may be configured to execute the one or more instructions to reducing the noise information from each of the plurality of image frames based on the levelling process applied to each group.

[0103] According to an embodiment of the disclosure, the at least one processor may be configured to execute the one or more instructions to determine a contribution value and a strength value of each of the identified light source affecting the extracted foreground objects in a first camera mode of the electronic device (200) based on the one or more obtained characteristics, wherein the first camera mode is a shooting mode for displaying captured images in real-time. The at least one processor may be configured to execute the one or more instructions to generate a first encapsulated light model based on the determined contribution value and the determined strength value.

[0104] According to an embodiment of the disclosure, the at least one processor may be configured to execute the one or more instructions to determine a contribution value and a strength value of each of the identified light sources affecting the extracted foreground objects in a second camera mode of the electronic device (200) based on the one or more obtained characteristics, wherein the second camera mode is a shooting mode for displaying a capture image processed through at least one of long exposure times or multi-frame composition. The at least one processor may be configured to execute the one or more instructions to determine an aesthetic score for each of the identified light sources to evaluate whether each of the identified light sources is perceived as artificial or real from an illumination perspective. The at least one processor may be configured to execute the one or more instructions to generate a second encapsulated light model based on the determined contribution value, the determined strength value, and the determined aesthetic score.

[0105] According to an embodiment of the disclosure, the at least one processor may be configured to execute the one or more instructions to detect that the aesthetic score is lower than a pre-defined threshold value. The at least one processor may be configured to execute the one or more instructions to vary at least one of a temperature value, a contrast value, and a saturation value for each of the detected light sources that has the aesthetic score lower than the pre-defined threshold value to adjust the illumination.

[0106] The various actions, acts, blocks, steps, or the like in the flow diagrams may be performed in the order presented, in a different order, or simultaneously. Further, in an embodiment, some of the actions, acts, blocks, steps, or the like may be omitted, added, modified, skipped, or the like without departing from the scope of the invention.

[0107] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one ordinary skilled in the art to which this invention belongs. The system, methods, and examples provided herein are illustrative only and not intended to be limiting.

[0108] While specific language has been used to describe the present subject matter, any limitations arising on account thereto, are not intended. As would be apparent to a person in the art, various working modifications may be made to the method to implement the inventive concept as taught herein. The drawings and the forgoing description give examples of an embodiment. Those skilled in the art will appreciate that one or more of the described elements may well be combined into a single functional element. In an embodiment, certain elements may be split into multiple functional elements. Elements from one embodiment may be added to another embodiment.

[0109] An embodiment disclosed herein may be implemented using at least one hardware device and performing network management functions to control the elements.

[0110] An embodiment disclosed herein may be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer. A Computer-readable medium may be any available medium that may be accessed by a computer and include both volatile and non-volatile media, removable and non-removable media. Also, computer-readable media may include computer storage media and communication media.

[0111] The computer storage media includes both volatile and non-volatile media and removable and non-removable media implemented by any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other types of data. Communication media may include computer readable instructions, data structures, or other types of data of modulated data signals, such as program modules. Also, computer-readable storage media may be provided in the form of non-transitory storage media. Here, "non-transitory storage media" is a tangible device and simply means not including signals (for example, electromagnetic waves), and the term does not distinguish between a case where data is semi-permanently stored in a storage medium and a case where data is temporarily stored in a storage medium. For example, "non-transitory storage media" may include a buffer where data is temporarily stored.

[0112] According to one embodiment, methods according to various embodiments disclosed in the disclosure may be included in a computer program product. The computer program product is commodity and may be traded between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (for example, a compact disc read only memory (CD-ROM)) or may be distributed (for example, downloaded or uploaded) directly or online through an application store or between two user devices (for example, smartphones). In the case of online distribution, at least a part of the computer program product (for example, a downloadable app) may be temporarily stored on a machine-readable storage medium, such as a server of an application store or a memory of a relay server or may be generated temporarily.

[0113] The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and / or adapt for various applications such specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the scope of the embodiments as described herein.

Claims

1.A method (900) for enhancing an image frame, the method (900) comprising:obtaining (901) a plurality of image frames through one or more image sensors of an electronic device (200);extracting (902) foreground objects from each of the obtained plurality of image frames;performing (903) a noise reduction on the obtained plurality of image frames based on the extracted foreground objects;identifying (904) one or more light sources in each of the plurality of noise-reduced image frames;obtaining (905) one or more characteristics of each of the identified light sources for the extracted foreground objects;generating (906) at least one of encapsulated light models for adjusting a dynamics of light within an image based on the one or more identified light sources and the one or more obtained characteristics; andadjusting (907) illumination of the one or more identified light sources using at least one of the generated encapsulated light models to enhance the image frame.2.The method (900) of claim 1, wherein the one or more characteristics comprise orientation information, reflection information, and absorbance information.3.The method (900) any one of claims 1 to 2, wherein performing the noise reduction comprises:obtaining latent codes for each of the extracted foreground objects,aggregating the obtained latent codes into an unified representation;determining whether the obtained plurality of image frames comprises the noise based on the aggregated latent code, andreducing, in response to determining that the plurality of obtained image frames comprises the noise, the noise from the obtained plurality of images frames.4.The method (900) any one of claims 1 to 3, wherein the reducing the noise from the obtained plurality of image frames comprises:identifying a plurality of pixels in each of the obtained plurality of image frames that is affected by the noise in each of a plurality of channels, wherein each of the plurality of channels comprising one of a luminance channel and a chroma channel, wherein the plurality of pixels have same value;grouping the identified plurality of pixels together based on a common value;applying an adaptive strength parameter for levelling process on the each group, wherein the adaptive strength parameter is determined based on the values of the pixels within the group; andreducing the noise from each of the plurality of image frames based on the levelling process applied to the each group.5.The method (900) any one of claims 1 to 4, wherein generating at least one of the encapsulated light models comprises:determining a contribution value and a strength value of each of the identified light sources affecting the extracted foreground objects in a first camera mode of the electronic device (200) based on the one or more obtained characteristics, wherein the first camera mode is an image capturing mode for displaying captured images in real time; andgenerating a first encapsulated light model based on the determined contribution value and the determined strength value.6.The method (900) any one of claims 1 to 5, wherein generating at least one of the encapsulated light models comprises:determining a contribution value and a strength value of each of the identified light sources affecting the extracted foreground objects in a second camera mode of the electronic device (200) based on the one or more obtained characteristics, wherein the second camera mode is an image capturing mode for displaying a capture image processed through at least one of long exposure times or multi frame composition; anddetermining an aesthetic score for each of the identified light sources to evaluate whether each of the identified light sources is perceived as artificial or real from an illumination perspective; andgenerating a second encapsulated light model based on the determined contribution value, the determined strength value, and the determined aesthetic score.7.The method (900) any one of claims 1 to 6, wherein adjusting the illumination of the one or more identified light sources comprises:detecting that the aesthetic score is lower than a pre-defined threshold value; andvarying at least one of a temperature value, a contrast value, and a saturation value for each of the detected light sources that has the aesthetic score lower than the pre-defined threshold value to adjust the illumination.8.An electronic device (200) for enhancing an image frame, the electronic device (200) comprising:a memory (210) storing one or more instructions;a communicator (230);a camera module (240);a display module (250); andat least one processor (220) configure to execute the one or more instructions stored in the memory,wherein the at least one processor is configured to execute the one or more instructions to :obtain a plurality of image frames through one or more image sensors of an electronic device (200);extract foreground objects from each of the obtained plurality of image framesperform a noise reduction on the obtained plurality of image frames based on the extracted foreground objects;identify one or more light sources in each of the plurality of noise-reduced image framesobtain one or more characteristics of each of the identified light sources for the extracted foreground objects;generate at least one of encapsulated light models for adjusting a dynamics of light within an image based on the one or more identified light sources and the one or more obtained characteristics; andadjust illumination of the one or more identified light sources using at least one of the generated encapsulated light models to enhance the image frame.9.The electronic device (200) of claim 8, wherein the one or more characteristics comprise orientation information, reflection information, and absorbance information.10.The electronic device (200) any one of claims 8 to 9, wherein to perform the noise reduction on the obtained plurality of image frames, the processor (220) is configured to execute the one or more instructions to:obtain latent codes of the extracted foreground objects,aggregate the obtained latent codes into an unified representation;determine whether the obtained plurality of image frames comprises the noise based on the aggregated latent code, andreduce, in response to determine that the obtained plurality of image frames comprises the noise, the noise from the obtained plurality of image frames.11.The electronic device (200) any one of claims 8 to 10, wherein to reduce the noise from the obtained plurality of image frames, the processor (220) is configured to execute the one or more instructions to:identify a plurality of pixels in each of the obtained plurality of image frames that is affected by the noise in each of a plurality of channels, wherein each of the plurality of channels comprising one of a luminance channel and a chroma channel, wherein the plurality of pixels have same value;group the identified plurality of pixels together based on a common value;apply an adaptive strength parameter for levelling process on each group, wherein the adaptive strength parameter is determined based on the values of the pixels within the group; andreducing the noise information from each of the plurality of image frames based on the levelling process applied to the each group.12.The electronic device (200) any one of claims 8 to 11, wherein to generate at least one of encapsulated light models, the processor (220) is configured to execute the one or more instructions to:determine a contribution value and a strength value of each of the identified light source affecting the extracted foreground objects in a first camera mode of the electronic device (200) based on the one or more obtained characteristics, wherein the first camera mode is a shooting mode for displaying captured images in real-time; andgenerate a first encapsulated light model based on the determined contribution value and the determined strength value.13.The electronic device (200) any one of claims 8 to 12, wherein to generate at least one of the encapsulated light models, the processor (220) is configured to execute the one or more instructions to:determine a contribution value and a strength value of each of the identified light sources affecting the extracted foreground objects in a second camera mode of the electronic device (200) based on the one or more obtained characteristics, wherein the second camera mode is a shooting mode for displaying a capture image processed through at least one of long exposure times or multi-frame composition;determine an aesthetic score for each of the identified light sources to evaluate whether each of the identified light sources is perceived as artificial or real from an illumination perspective; andgenerate a second encapsulated light model based on the determined contribution value, the determined strength value, and the determined aesthetic score.14.The electronic device (200) any one of claims 8 to 13, wherein to adjust the illumination of the one or more identified light sources, the processor (220) is configured to execute the one or more instructions to:detect that the aesthetic score is lower than a pre-defined threshold value; andvary at least one of a temperature value, a contrast value, and a saturation value for each of the detected light sources that has the aesthetic score lower than the pre-defined threshold value to adjust the illumination.15.A computer-readable recording medium having recorded thereon a program for performing the control method of any one of claims 1 to 7, on a computer.

Images