Electronic device, method, and non-transitory computer-readable storage medium for performing tone mapping on basis of local histogram information

Abstract

This electronic device may comprise: a memory for storing instructions; a camera; and at least one processor. The instructions may cause the electronic device to: obtain, through the camera, a first set of local gain tables corresponding to multiple regions of a source image, respectively, by using an identified source image, wherein each of the local gain tables includes gains to be applied to brightness values of pixels of a corresponding region of the source image; obtain a set of pieces of local histogram information corresponding to the multiple regions, respectively, wherein the local histogram information indicates a distribution of brightness values of pixels changed by the gains; obtain a global gain table including gains to be applied to pixels of the entire region of the source image, by using the set of pieces of local histogram information; obtain a second set of local gain tables corresponding to the multiple regions, respectively, by combining the global gain table and the local gain tables included in the first set; and generate a result image, which is a result of tone mapping, by applying the second set of local gain tables to the multiple regions of the source image.

Description

Electronic device, method, and non-transient computer-readable storage medium for performing tone mapping based on local histogram information

[0001] The present disclosure relates to an electronic device, a method, and a non-transient computer-readable storage medium for performing tone mapping based on local histogram information.

[0002] The electronic device may include a camera. The electronic device may perform tone mapping on an image identified through the camera. The electronic device may acquire histogram information representing the distribution of brightness values ​​of pixels of the identified image. The electronic device may perform histogram equalization to change the tone of the identified image. By performing histogram equalization on the image, the electronic device may generate a different image.

[0003] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure.

[0004] No claim or determination is made as to whether any of the foregoing can be applied as prior art related to the present disclosure.

[0005] An electronic device is described. The electronic device may include a memory comprising one or more storage media for storing instructions. The electronic device may include a camera. The electronic device may include at least one processor comprising a processing circuit. The instructions may cause the electronic device to identify a source image for performing tone mapping through the camera when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to obtain a first set of local gain tables corresponding to each of a plurality of regions of the source image using the identified source image when executed individually or collectively by the at least one processor. Each local gain table included in the first set may include gains to be applied to the brightness values ​​of the pixels of the corresponding region of the source image. The instructions may cause the electronic device to obtain a set of local histogram information corresponding to each of the plurality of regions when executed individually or collectively by the at least one processor. The local histogram information within the set may represent the distribution of brightness values ​​of pixels modified by the gains. When the instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain a global gain table representing gains to be applied to pixels of the entire area of ​​the source image using the set of local histogram information. When the instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain a second set of local gain tables corresponding to each of the plurality of areas by combining the global gain table and the local gain tables included in the first set.The above instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to generate a result image, which is the result of the tone mapping for the source image, by applying the second set of the local gain tables to the plurality of regions of the source image.

[0006] A method is provided. The method may be executed within an electronic device having a camera. The method may include the operation of identifying a source image for performing tone mapping through the camera. The method may include the operation of obtaining a first set of local gain tables corresponding to each of a plurality of regions of the source image using the identified source image. Each local gain table included in the first set may include gains to be applied to the brightness values ​​of pixels in the corresponding region of the source image. The method may include the operation of obtaining a set of local histogram information corresponding to each of the plurality of regions. The local histogram information in the set may represent the distribution of brightness values ​​of pixels changed by the gains. The method may include the operation of obtaining a global gain table representing gains to be applied to pixels in the entire region of the source image using the set of local histogram information. The method may include the operation of obtaining a second set of local gain tables corresponding to each of the plurality of regions by combining the global gain table and the local gain tables included in the first set. The above method may include the operation of generating a result image, which is the result of the tone mapping for the source image, by applying the second set of local gain tables to the plurality of regions of the source image.

[0007] A non-transient computer-readable storage medium is provided. The non-transient computer-readable storage medium may store one or more programs. The one or more programs may include instructions that, when executed by an electronic device having a camera, cause the electronic device to identify a source image for performing tone mapping through the camera. The one or more programs may include instructions that, when executed by the electronic device, cause the electronic device to obtain a first set of local gain tables corresponding to each of a plurality of regions of the source image using the identified source image. Each local gain table included in the first set may include gains to be applied to the brightness values ​​of pixels in the corresponding region of the source image. The one or more programs may include instructions that, when executed by the electronic device, cause the electronic device to obtain a set of local histogram information corresponding to each of the plurality of regions. The local histogram information in the set may represent the distribution of brightness values ​​of pixels changed by the gains. The above one or more programs may include instructions that cause the electronic device to obtain a global gain table, which indicates gains to be applied to pixels of the entire area of ​​the source image, by using the set of local histogram information when executed by the electronic device. The above one or more programs may include instructions that cause the electronic device to obtain a second set of local gain tables corresponding to each of the plurality of areas by combining the global gain table and the local gain tables included in the first set when executed by the electronic device.The above one or more programs may include instructions that cause the electronic device to generate a result image, which is the result of the tone mapping for the source image, by applying the second set of local gain tables to the plurality of regions of the source image when executed by the electronic device.

[0008] Figure 1 illustrates an example of an electronic device that performs tone mapping.

[0009] Figure 2 is a simplified block diagram of an exemplary electronic device.

[0010] Figure 3 is a flowchart illustrating the operation of an electronic device that generates a result image using a set of local gain tables.

[0011] FIG. 4 illustrates an exemplary operation of an electronic device that provides an image when performing tone mapping.

[0012] FIGS. 5a through 5d illustrate an exemplary operation of an electronic device for acquiring a second set of local gain tables for each of a plurality of regions of a source image.

[0013] FIG. 6 illustrates an exemplary operation of an electronic device that performs interpolation using a second set of local gain tables.

[0014] FIG. 7 is a block diagram of an electronic device in a network environment according to various embodiments.

[0015] Figure 1 illustrates an example of an electronic device that performs tone mapping.

[0016] Referring to FIG. 1, an electronic device (100) may be used to perform image processing. For example, the electronic device (100) may include a camera (e.g., camera (209) of FIG. 2). For example, the electronic device (100) may perform image processing (e.g., tone mapping) on ​​an image acquired through the camera. For example, the electronic device (100) may perform tone mapping on a source image identified through the camera (e.g., source image (415) of FIG. 4). For example, the tone mapping may be described as a technique for changing the dynamic range of an image. For example, global tone mapping may be described as a technique for changing the tone of an image collectively by applying a single tone mapping curve to the entire image. For example, local tone mapping can be described as a technique for changing the tone of each of the regions differently by applying tone mapping curves to each of the regions of the image. For example, the electronic device (100) can change the distribution of brightness values ​​of the pixels of the source image by performing the tone mapping. The electronic device (100) can improve the partial (or overall) gradation (or gradation range) of the image (or video) by performing the tone mapping.

[0017] For example, the electronic device (100) can perform equalization on a histogram representing the distribution of brightness values ​​of pixels of the source image. For example, the electronic device (100) can perform histogram equalization on the source image identified through the camera. For example, the histogram equalization can be described as a technique for changing the brightness values ​​of pixels to improve the contrast of the image. For example, the histogram equalization can be described as a technique for changing the brightness values ​​of each pixel within the range of brightness values ​​that a pixel can have. For example, the electronic device (100) can widen the dynamic range of the image by performing histogram equalization.

[0018] The reference dynamic range (or maximum dynamic range) can be described as the range between the maximum and minimum brightness values ​​that can be represented within a single image frame. For example, the reference dynamic range can be defined through the camera (or sensor) used to acquire the image. For example, the reference dynamic range can be defined through the display used to display the image. For example, the reference dynamic range of an image frame on which histogram equalization has been performed can be maintained. For example, histogram equalization can be described as a technique that modifies only the dynamic range represented by the image frame, among the reference dynamic range and the dynamic range represented by the image frame.

[0019] For example, the state (140) may be described as a state that displays an image acquired through the camera of the electronic device (100). For example, the image of the state (140) may be described as an image before tone mapping is performed. For example, the image of the state (160) may be described as a state that displays an image after tone mapping has been performed. For example, when the electronic device (100) acquires the image of the state (140) through the camera, it may convert the image of the state (140) into an image of the state (160). For example, the electronic device (100) may improve the quality of the image acquired through the camera by performing tone mapping. For example, the electronic device (100) may generate an image of the state (160) (in real time) using the image of the state (140) based on acquiring the image of the state (140) through the camera. For example, the image of state (160) may be described as an image from which local tone mapping and / or global tone mapping has been performed on the image of state (140). For example, the electronic device (100) may perform local tone mapping and global tone mapping to improve the quality of the image of state (140). For example, the electronic device (100) may be required to complete image processing within a short time to perform image processing in real time.

[0020] For example, an electronic device (100) may transfer a source image (e.g., source image (415) of FIG. 4) from a memory (e.g., memory (206) of FIG. 2) to at least one processor (e.g., at least one processor (207) of FIG. 2) to perform tone mapping. For example, the electronic device (100) may perform tone mapping within the at least one processor. For example, the electronic device (100) may store at least a portion of the data for the source image in a line memory included in the at least one processor. For example, time may be required for the electronic device (100) to transfer the data of the source image from the memory to the at least one processor. For example, if the electronic device (100) acquires an image through a camera (e.g., camera (209) of FIG. 2), transferring the data of the image stored in the memory to the at least one processor may take a relatively long time. For example, the electronic device (100) may be required to reduce the time required for image processing (e.g., tone mapping) in order to process the image acquired through the camera in real time. For example, the time required to perform tone mapping using the image data stored in the at least one processor may be shorter than the time required to perform tone mapping using the image data stored in the memory. For example, when processing the image acquired through the camera in real time, it may be required to reduce the time required for image processing. For example, the electronic device (100) may be required to transmit the image data from the memory to the line memory of the at least one processor whenever tone mapping is performed.For example, the electronic device (100) may transfer data of the image from the memory to the at least one processor to perform local tone mapping, and then transfer data of another image on which local tone mapping has been performed from the at least one processor to the memory. For example, the electronic device (100) may transfer data of the image on which local tone mapping has been performed from the memory to the at least one processor to perform global tone mapping, and then transfer data of the image on which global tone mapping has been performed from the at least one processor to the memory.

[0021] For example, the electronic device (100) may store at least a portion of the image in a line memory (not shown) within the at least one processor. For example, the electronic device (100) may perform tone mapping using the data of the image stored in the line memory. For example, the line memory may be described as a memory that operates by processing image data one line at a time. For example, the line memory may be included within at least one processor (e.g., at least one processor (207) of FIG. 2). However, it is not limited thereto. The larger the capacity of the line memory, the larger the size of the data that the line memory can store. For example, in the case of the electronic device (100) having a large capacity line memory, high-complexity tone mapping may be performed. For example, because the size of the electronic device (100) is limited, the line memory may be required to be miniaturized. For example, the size of the line memory for performing tone mapping may be preferred to be smaller.

[0022] For example, the electronic device (100) can obtain an image similar to an image that has undergone local tone mapping and global tone mapping by performing a single tone mapping without performing local tone mapping and global tone mapping, respectively. For example, the electronic device (100) can obtain a different local gain table by synthesizing a local gain table for performing local tone mapping and a global gain table for performing global tone mapping. For example, when the electronic device (100) performs tone mapping using the different local gain table, the number of times image data is transferred from the memory to the line memory can be reduced because a single tone mapping is performed. For example, as the number of times data is transferred between the memory and the line memory is reduced, the time required to perform image processing on the image identified by the camera can be reduced. For example, when the electronic device (100) performs tone mapping using the different local gain table, the size of the line memory can be kept small. For example, the electronic device (100) may include a line memory of (relatively) small capacity as it performs tone mapping using the other local gain table. For example, since the electronic device (100) may include a line memory of (relatively) small size as it performs tone mapping using the local gain table, it may contribute to the miniaturization of the electronic device (100).

[0023] For example, the electronic device (100) may include hardware components used to perform or execute the above operations. The hardware components are described and illustrated with reference to FIG. 2.

[0024] Figure 2 is a simplified block diagram of an exemplary electronic device.

[0025] Referring to FIG. 2, the electronic device (100) may include at least one processor (207), memory (206), and camera (209).

[0026] At least one processor (207) may include a hardware component for processing data using instructions stored in memory (206). The hardware component for processing data may include a CPU (central processing unit) (e.g., including processing circuits). The hardware component for processing data may include a GPU (graphic processing unit) (e.g., including processing circuits). The hardware component for processing data may include a DPU (display processing unit) (e.g., including processing circuits). The hardware component for processing data may include an ISP (Image Signal Processor).

[0027] At least one processor (207) may include one or more cores. For example, at least one processor (207) may have the structure of a multi-core processor such as a dual core, a quad core, or a hexa core.

[0028] Memory (206) may include a hardware component for storing data and / or instructions that are input to and / or output from at least one processor (207). Memory (206) may include, for example, volatile memory such as random-access memory (RAM) and / or non-volatile memory such as read-only memory (ROM). Volatile memory may include, for example, at least one of dynamic RAM (DRAM), static RAM (SRAM), cache RAM, and pseudo SRAM (PSRAM). Non-volatile memory may include, for example, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, hard disk, compact disk, and embedded multimedia card (EMMC).

[0029] The camera (209) may include one or more light sensors (e.g., a CCD (charged coupled device) sensor, a CMOS (complementary metal oxide semiconductor) sensor) that generate an electrical signal indicating the color and / or brightness of light. For example, the camera (209) may be described as one or more image sensors. For example, the camera (209) may be available to acquire an image of the environment surrounding the electronic device (100).

[0030] At least one processor (207) can identify a source image for performing tone mapping through a camera (209). For example, the camera (209) can be used to acquire a source image. For example, the camera (209) can be used to acquire a source image representing a part of an environment in which the electronic device (100) is included. The electronic device (100) can acquire a first set of local gain tables corresponding to each of a plurality of regions of the source image using the identified source image (e.g., source image (415) of FIG. 4). For example, each local gain table included in the first set may include gains to be applied to the brightness values ​​of the pixels of the corresponding region of the source image. For example, at least one processor (207) can acquire a set of local histogram information corresponding to each of the plurality of regions, representing the distribution of brightness values ​​of the pixels of the plurality of regions modified by the local gain tables in the first set. For example, at least one processor (207) can obtain a global gain table (e.g., global gain table (588) of FIG. 5d) representing gains to be applied to pixels of an entire area of ​​a source image (e.g., source image (415) of FIG. 4) by using a set of local histogram information. For example, at least one processor (207) can obtain a second set of local gain tables corresponding to each of the plurality of areas by combining the global gain table and the local gain tables included in the first set. For example, at least one processor (207) can generate a result image (e.g., result image (435) of FIG. 4) which is the result of the tone mapping for the source image by applying the second set of local gain tables to each of the plurality of areas of the source image. For example, a memory (206) can be used to store data representing the source image.For example, the memory (206) can transmit data representing the source image to at least one processor (207). For example, the memory (206) can be used to store data representing the result image (435).

[0031] FIG. 3 is a flowchart illustrating the operation of an electronic device that generates a result image using a set of local gain tables. This method may be executed by the electronic device (100) illustrated in FIG. 2 or by at least one processor (207) of the electronic device (100).

[0032] Referring to FIG. 3, in operation 310, at least one processor (207) can identify a source image (e.g., source image (415) of FIG. 4) for performing tone mapping through a camera (209). For example, at least one processor (207) can acquire a source image (415) representing a part of an environment containing an electronic device (100) through the camera (209). However, it is not limited thereto. For example, at least one processor (207) can receive data representing the source image from an external electronic device (not shown).

[0033] In operation 320, at least one processor (207) may obtain a first set of local gain tables (e.g., first local gain table (522), first local gain table (524), first local gain table (526), ​​and first local gain table (528) of FIG. 5a and 5b) corresponding to each of a plurality of regions (e.g., region (502), region (504), region (506), and region (508)) of the source image (e.g., source image (415) of FIG. 4) using the identified source image. For example, each local gain table included in the first set (e.g., first local gain table (522)) may include gains to be applied to the brightness values ​​of pixels in the corresponding regions of the source image (e.g., source image (415) of FIG. 4). For example, the gains may be numbers to be multiplied to each of the brightness values. For example, a gain table can be described as a table in which a gain is assigned to a pixel having a specified brightness value. For example, the gain table can be described as a table in which a first gain is assigned to a first pixel having a first brightness value and a second gain is assigned to a second pixel having a second brightness value. For example, each of the local gain tables for the plurality of regions of a source image (e.g., source image (415) of FIG. 4) can be obtained using local histogram information. The operation of obtaining a local gain table using local histogram information will be described later with reference to FIG. 5a.

[0034] According to one embodiment, at least one processor (207) may obtain a first set of local mapping tables using a first set of local histogram information. For example, a mapping table may be described as a table in which a second brightness value is mapped to a first pixel having a first brightness value, and a fourth brightness value is mapped to a second pixel having a third brightness value. For example, a mapping table may be described as a table representing the brightness values ​​of pixels to be obtained using a gain table. For example, at least one processor (207) may change the brightness values ​​of pixels in an image using the mapping table.

[0035] In operation 330, at least one processor (207) may obtain a second set of local histogram information corresponding to each of the plurality of regions, representing the distribution of brightness values ​​of pixels of the plurality of regions. For example, the brightness values ​​of the pixels may be modified by local gain tables included in a first set of local gain tables (e.g., the first local gain tables (522, 524, 526, 528) of FIG. 5B). For example, the histogram may be described as a graph (or table) representing the number of pixels of the source image according to brightness values. For example, at least one processor (207) may obtain the second set of local histogram information using the first set of local gain tables and the first set of local histogram information for the plurality of regions of the source image. For example, at least one processor (207) can obtain a second set of local histogram information by multiplying the gain for a brightness value in a first set of local gain tables by the brightness value of a pixel having said brightness value. For example, a pixel having said brightness value can be identified through the first set of local histogram information for a plurality of regions of said source image. For example, at least one processor (207) can obtain a second set of local histogram information corresponding to each of the plurality of regions. For example, the local histogram information in the second set of local histogram information may represent the distribution of brightness values ​​of pixels changed by gains included in the first set of local gain tables. For example, the local histogram information in the second set may represent a second local histogram (532), a second local histogram (534), a second local histogram (536), or a second local histogram (538).

[0036] In operation 340, at least one processor (207) can obtain a global gain table (e.g., global gain table (588) of FIG. 5d) representing gains to be applied to pixels in an entire area of ​​a source image (e.g., source image (415) of FIG. 4) by using a second set of local histogram information. For example, at least one processor (207) can obtain the global gain table by accumulating the second set of local histogram information. For example, at least one processor (207) can estimate or obtain global histogram information for the entire source image by accumulating the second set of local histogram information. For example, the global histogram information may not represent a distribution of brightness values ​​for actual pixels of the source image. The global histogram information may be similar to or substantially identical to other global histogram information obtained by identifying the brightness values ​​of each pixel included in an entire area of ​​the source image (e.g., source image (415) of FIG. 4). For example, the other global histogram information may be referenced as legacy global histogram information. For example, since the global histogram information is obtained by accumulating a second set of local histogram information, the time required to obtain the global histogram information may be shorter than the time required to obtain the other global histogram information.

[0037] At least one processor (207) can obtain a global gain table (e.g., the global gain table (588) of FIG. 5d) using the global histogram information. For example, at least one processor (207) can obtain the global gain table by performing a histogram equalization operation on the global histogram information. For example, at least one processor (207) can obtain the global gain table by applying a cumulative distribution function to the global histogram information.

[0038] According to one embodiment, at least one processor (207) can obtain a global mapping table using global histogram information. For example, the global mapping table can be replaced with a global gain table. For example, the global gain table (588) can be replaced with a global mapping table.

[0039] In operation 350, at least one processor (207) can obtain a second set of local gain tables (e.g., second local gain table (592), second local gain table (594), second local gain table (596), and second local gain table (598)) corresponding to each of a plurality of regions (e.g., region (522), region (524), region (526), ​​and region (528)) of a source image (e.g., source image (415) of FIG. 4) by combining a global gain table (e.g., the global gain table (588) of FIG. 5d) and local gain tables included in the first set (e.g., the first local gain table (522), region (524), region (526), ​​and region (528) of FIG. 5b).

[0040] According to one embodiment, at least one processor (207) can obtain a second set of local mapping tables by combining the global gain table and the local gain tables included in the first set of local gain tables. For example, the second set of local gain tables may be replaced with the second set of local mapping tables. For example, the second set of local mapping tables may be replaced with the second set of local gain tables.

[0041] According to one embodiment, at least one processor (207) may operate in a first mode focused on speed (or velocity) for performing image processing. For example, at least one processor (207) may operate in a second mode focused on image quality. For example, the first mode may be referred to as a high-speed mode.

[0042] For example, the second mode may be referred to as a quality mode or a picture quality mode. For example, when at least one processor (207) operates in the first mode, it may perform at least some of operations 320 to 360. For example, when at least one processor (207) operates in the first mode among the first mode and the second mode, it may generate a result image (435) based on synthesizing or calculating a global gain table (e.g., the global gain table (588) of FIG. 5d) and local gain tables in a first set of local gain tables (e.g., the first local gain table (522), the first local gain table (524), the first local gain table (526), ​​and the first local gain table (528)). For example, when at least one processor (207) operates in the first mode, it can obtain a second set of local gain tables by synthesizing the global gain table and the local gain tables in the first set of local gain tables.

[0043] At least one processor (207) can obtain a result image (435) by performing interpolation (e.g., bilinear interpolation) using a second set of local gain tables when operating in the first mode. For example, when at least one processor (207) obtains a result image (435) using the second set, it can determine the brightness value of each pixel (e.g., pixel (656) in FIG. 6) in the result image (435) by performing interpolation based on each local gain table of the second set corresponding to each region (e.g., region (622) in FIG. 6) adjacent to each region (e.g., region (622) in FIG. 6) of the source image (415).

[0044] According to one embodiment, at least one processor (207) can generate a result image (435) based on combining a global gain table (588) and local gain tables in a first set of local gain tables while the electronic device (100) is in a low-power state. For example, at least one processor (207) can identify whether the electronic device (100) is in a low-power state. For example, at least one processor (207) can generate a result image (435) based on identifying that the electronic device (100) is in a low-power state. For example, at least one processor (207) can perform at least one of operations 310 to 360 of FIG. 3 while the electronic device (100) is in a low-power state.

[0045] For example, at least one processor (207) can generate an intermediate image, which is the result of local tone mapping based on local gain tables included in a first set, by applying gains included in local gain tables included in a first set of local gain tables to each of the pixels of a source image (415) while the electronic device (100) is not in a low-power state. For example, the intermediate image may be stored in a memory (206) different from the line memory of at least one processor (207). For example, at least one processor (207) may obtain another global gain table from the intermediate image. For example, at least one processor (207) may obtain another global gain table for the intermediate image using a cumulative distribution function by identifying the brightness values ​​of the pixels of the intermediate image. For example, at least one processor (207) may obtain other global histogram information based on identifying the brightness values ​​of each of the pixels in the entire area of ​​the intermediate image. For example, the cumulative distribution function may be obtained through the other global histogram information. For example, the other global histogram information may be obtained by counting the number of pixels having each brightness value among the pixels representing the intermediate image. For example, at least one processor (207) may generate another result image, which is the result of tone mapping for the source image (415), using the other global gain table. For example, at least one processor (207) may generate the other result image while the electronic device (100) is not in a low-power state.

[0046] For example, at least one processor (207) may modify local gain tables in a first set of local gain tables based on the interpolation of at least two local gain tables corresponding to adjacent regions while the electronic device (100) is in a different state different from the low power state. For example, at least one processor (207) may modify local gain tables by performing interpolation on local gain tables in the first set of local gain tables. For example, at least one processor (207) may generate an intermediate image, which is the result of local tone mapping based on the local gain tables, by applying gains included in the modified local gain tables in the first set to pixels in the source image (415). For example, the intermediate image may be stored in a memory (206) different from the line memory in at least one processor (207). For example, at least one processor (207) may obtain a different global gain table from the intermediate image using a cumulative distribution function. For example, at least one processor (207) can use the other global gain table to generate another result image, which is the result of tone mapping for the source image (415).

[0047] According to one embodiment, at least one processor (207) can obtain a source image (e.g., source image (415) of FIG. 4) from memory (206). For example, at least one processor (207) can generate a result image (e.g., result image (435) of FIG. 4) by performing tone mapping using the source image (415) transmitted from memory (206). For example, the transmission of the source image and the generation of the result image are described and illustrated in more detail with reference to FIG. 4.

[0048] FIG. 4 illustrates an exemplary operation of an electronic device that provides an image when performing tone mapping.

[0049] Referring to FIG. 4, at least one processor (207) may be used to generate a result image (435) using a source image (415). For example, a memory (206) may be used to store the image. For example, in operation 410, the memory (206) may provide the source image (415) to at least one processor (207). For example, the memory (206) may transmit or provide data representing the source image (415) to at least one processor (207). For example, at least one processor (207) may receive the source image (415) from the memory (206). For example, at least one processor (207) may receive data representing the source image (415) from the memory (206).

[0050] For example, at least one processor (207) may perform at least some of operations 310 to 350 of FIG. 3 based on receiving a source image (415) from memory (206). For example, at least one processor (207) may obtain a first set of local histogram information corresponding to each of a plurality of regions of the source image (415) based on receiving the source image (415) from memory (206). For example, at least one processor (207) may obtain a first set of local gain tables (e.g., the first local gain table (522), the first local gain table (524), the first local gain table (526), ​​and the first local gain table (528) of FIG. 5b) using the first set of local histogram information based on receiving the source image (415) from memory (206). For example, at least one processor (207) can obtain a second set of local histogram information by applying a first set of local gain tables to a first set of local histogram information based on receiving a source image (415) from memory (206). For example, at least one processor (207) can obtain a global gain table (e.g., the global gain table (588) of FIG. 5d) by using the second set of local histogram information based on receiving a source image (415) from memory (206). For example, at least one processor (207) may obtain a second set of local gain tables (e.g., the second local gain table (592), the second local gain table (594), the second local gain table (596), and the second local gain table (598) of FIG. 5d) corresponding to each of a plurality of regions of the source image (415) by combining or synthesizing the first set of the global gain table and the local gain tables based on receiving the source image (415) from memory (206).

[0051] For example, in operation 420, the memory (206) may provide or transmit the source image (415) to at least one processor (207). For example, the memory (206) may provide or transmit data representing the source image (415) to at least one processor (207). For example, at least one processor (207) may receive the source image (415) or data representing the source image (415) from the memory (206). For example, at least one processor (207) may receive the source image (415) or data representing the source image (415) from the memory (206). For example, at least one processor (207) may execute operation 360 of FIG. 3 based on the acquisition of the source image (415) of operation 420. For example, at least one processor (207) can generate a result image (435) as a result of tone mapping for the source image (415) by applying a second set of local gain tables obtained in operation 350 of FIG. 3 to a plurality of regions of the source image (415) based on obtaining the source image (415) from memory (206).

[0052] For example, in operation 430, at least one processor (207) may transmit or provide a result image (435) to memory (206). For example, since the line memory of at least one processor (207) has a small capacity, at least one processor (207) may be required to transmit data representing the result image (435) to memory (206).

[0053] For example, at least one processor (207) can obtain a first set of local histogram information for each of a plurality of regions of a source image (415). For example, at least one processor (207) can obtain a second set of local histogram information for each of a plurality of regions of a source image (415) by applying a first set of local gain tables to the first set of local histogram information. For example, at least one processor (207) can estimate global histogram information for the entire region of the source image (415) using the second set of local histogram information. For example, at least one processor (207) can obtain a global gain table (e.g., the global gain table (588) of FIG. 5d) using the global histogram information. For example, at least one processor (207) can obtain a second set of local gain tables by synthesizing the global gain table and the first set of local gain tables. For example, at least one processor (207) can obtain a third set of local gain tables by performing interpolation on a second set of local gain tables. For example, the acquisition of the first set of local gain tables to the third set of local gain tables is described and illustrated in more detail with reference to FIGS. 5a through 5d.

[0054] FIGS. 5a through 5d illustrate an exemplary operation of an electronic device for acquiring a second set of local gain tables for each of a plurality of regions of a source image.

[0055] Referring to FIG. 5a, the state (500) can be described as a state in which the source image (415) is divided into multiple regions. For example, the source image (415) may be divided into four regions. For example, the source image (415) may be divided into region (502), region (504), region (506), and region (508). However, it is not limited thereto. For example, the source image (415) may be divided into two or more regions. For example, the regions of the source image (415) may be referred to as a grid.

[0056] According to one embodiment, the shape of a plurality of regions of the source image (415) may not be polygonal. For example, the shape of a region of the source image (415) may be based on the shape of a visual object within the source image (415) (e.g., the shape of a person or the shape of a building).

[0057] State (510) can be described as a state in which a first local histogram (512) for region (502) and a first local histogram (514) for region (504) are represented. For example, at least one processor (207) can obtain local histogram information representing the first local histogram (512) using data representing region (502). For example, the local histogram (512) can represent the distribution of brightness values ​​of pixels within region (502). For example, at least one processor (207) can obtain local histogram information representing the first local histogram (514) using data representing region (504). For example, the local histogram (514) can represent the distribution of brightness values ​​of pixels within region (504). For example, at least one processor (207) can obtain a second set of local histogram information corresponding to each of the multiple regions of the source image, which represents the distribution of brightness values ​​of pixels in the multiple regions of the source image, using an identified source image (e.g., source image (415)) before obtaining a first set of local gain tables.

[0058] State (520) can be described as a state in which a first local gain table (522) for region (502) and a first local gain table (524) for region (504) are represented. For example, at least one processor (207) can obtain the first local gain table (522) using the first local histogram (512). For example, at least one processor (207) can obtain the first local gain table (522) by performing histogram equalization on the first local histogram (512). For example, at least one processor (207) can obtain the first local gain table (522) by performing normalization on the first local histogram (512). For example, at least one processor (207) can obtain a first local gain table (522) by applying a cumulative distribution function to a local histogram obtained by performing histogram equalization on a first local histogram (512). For example, at least one processor (207) can obtain a first local gain table (524) using a first local histogram (514). For example, at least one processor (207) can obtain a first local gain table (524) by performing histogram equalization on a first local histogram (514). For example, at least one processor (207) can obtain a first local gain table (524) by performing normalization on a first local histogram (514). For example, at least one processor (207) can obtain a first local gain table (524) by applying a cumulative distribution function to a local histogram obtained by performing histogram equalization on a first local histogram (514).For example, at least one processor (207) can obtain a first set of local gain tables by using a histogram equalization algorithm (e.g., AHE (Adaptive Histogram Equalization), CLAHE (Contrast Limited Adaptive Histogram Equalization)).

[0059] For example, at least one processor (207) can obtain a gain table from histogram information using a cumulative distribution function. For example, the following mathematical formula may be referenced for obtaining a gain table using a cumulative distribution function.

[0060]

[0061] The above represents the number of pixels with a brightness value of i in the image. The above represents the number of brightness values ​​a pixel can have. The above The following mathematical formula may be referenced for obtaining the cumulative distribution function using .

[0062]

[0063] The above represents a cumulative distribution function. For example, a mathematical formula for obtaining the above cumulative distribution function is described, but this is merely illustrative. The following mathematical formula may be referenced for obtaining a new brightness value of a pixel using the above cumulative distribution function.

[0064]

[0065] The above represents the cumulative distribution function. The above represents the number of brightness values ​​a pixel can have. The above represents a function for rounding decimal points. The above represents the total number of pixels. For example, at least one processor (207) can obtain a gain table using mathematical formulas 1 to 3.

[0066] State (530) can be described as a state in which a second local histogram (532) for region (502) and a second local histogram (534) for region (504) are represented. For example, at least one processor (207) can obtain the second local histogram (532) by performing, combining, or synthesizing the first local histogram (512) with the first local gain table (522). For example, at least one processor (207) can obtain the second local histogram (532) by applying the first local gain table (522) to the first local histogram (512). For example, at least one processor (207) can obtain the second local histogram (532) by multiplying each brightness value of each pixel represented by the first local histogram (512) by the gain for each brightness value represented by the first local gain table (522). For example, at least one processor (207) can obtain a second local histogram (534) by performing an operation on the first local histogram (514) with the first local gain table (524). For example, at least one processor (207) can obtain a second local histogram (534) by applying the first local gain table (524) to the first local histogram (514). For example, at least one processor (207) can obtain a second local histogram (534) by multiplying the gain for the brightness value represented by the first local gain table (524) by the pixels of the brightness value represented by the first local histogram (514).

[0067] Referring to FIG. 5b, the state (540) can be described as a state in which a local histogram (516) for region (506) and a first local histogram (518) for region (508) are represented. For example, at least one processor (207) can sequentially obtain histograms and local gain tables for each of a plurality of regions of a source image (415). For example, at least one processor (207) can obtain local histogram information representing the first local histogram (516) using data representing region (506). For example, the first local histogram (516) can represent the distribution of brightness values ​​of pixels within region (506). For example, at least one processor (207) can obtain local histogram information representing the first local histogram (518) using data representing region (508). For example, the local histogram (518) may represent the distribution of brightness values ​​of pixels within the region (508). For example, the first set of local histogram information may include local histogram information representing the first local histogram (512), local histogram information representing the first local histogram (514), local histogram information representing the first local histogram (516), and local histogram information representing the first local histogram (518). However, it is not limited thereto. For example, the first set of local histogram information may be based on divided regions of the source image (415).

[0068] State (550) can be described as a state in which a first local gain table (526) for region (506) and a first local gain table (528) for region (508) are represented. For example, at least one processor (207) can obtain the first local gain table (526) using the first local histogram (516). For example, at least one processor (207) can obtain the first local gain table (526) by performing histogram equalization on the first local histogram (516). For example, at least one processor (207) can obtain the first local gain table (526) by performing normalization on the first local histogram (516). For example, at least one processor (207) can obtain a first local gain table (526) by applying a cumulative distribution function to a local histogram obtained by performing histogram equalization on a first local histogram (516). For example, at least one processor (207) can obtain a first local gain table (528) using a first local histogram (518). For example, at least one processor (207) can obtain a first local gain table (528) by performing histogram equalization on a first local histogram (518). For example, at least one processor (207) can obtain a first local gain table (528) by performing normalization on a first local histogram (518). For example, at least one processor (207) can obtain a first local gain table (528) by applying a cumulative distribution function to a local histogram obtained by performing histogram equalization on a first local histogram (518). For example, a first set of local gain tables may include a first local gain table (522), a first local gain table (524), a first local gain table (526), ​​and a first local gain table (528).However, it is not limited to this. For example, the number of tables included in the first set of local gain tables may be based on the number of regions of the source image (415).

[0069] State (560) can be described as a state in which a second local histogram (536) for region (506) and a second local histogram (538) for region (508) are represented. For example, at least one processor (207) can obtain the second local histogram (536) by performing an operation on the first local histogram (516) with the first local gain table (526). For example, at least one processor (207) can obtain the second local histogram (536) by applying the first local gain table (526) to the first local histogram (516). For example, at least one processor (207) can obtain the second local histogram (536) by multiplying the gain for the brightness value represented by the first local gain table (526) by the pixels of the brightness value represented by the first local histogram (516). For example, at least one processor (207) can obtain a second local histogram (538) by performing an operation on the first local histogram (518) with the first local gain table (528). For example, at least one processor (207) can obtain a second local histogram (538) by applying the first local gain table (528) to the first local histogram (518). For example, at least one processor (207) can obtain a second local histogram (538) by multiplying the gain for the brightness value represented by the first local gain table (528) by the pixels of the brightness value represented by the first local histogram (518). For example, the second set of local histogram information may include local histogram information representing the second local histogram (532), local histogram information representing the second local histogram (534), local histogram information representing the second local histogram (536), and local histogram information representing the second local histogram (538). However, it is not limited thereto.For example, a second set of local histogram information may be based on segmented regions of the source image (415).

[0070] Referring to FIG. 5c, the state (570) can be described as a state in which the second local histogram (532), the second local histogram (534), the second local histogram (536), and the second local histogram (538) are represented. For example, the state (580) can be described as a state in which a global histogram (586) is obtained using the second local histogram (532), the second local histogram (534), the second local histogram (536), and the second local histogram (538). For example, at least one processor (207) can obtain data representing the histogram (582) using data representing the second local histogram (532) and data representing the second local histogram (534). For example, at least one processor (207) can obtain a histogram (582) by accumulating or adding a second local histogram (532) and a second local histogram (534). For example, the histogram (582) may be described as a histogram in which at least some of the second local histograms (532, 534, 536, 538) are accumulated. For example, at least one processor (207) can obtain data representing a histogram (584) using data representing a second local histogram (536) and data representing a histogram (582). For example, at least one processor (207) can obtain data representing a global histogram (586) using data representing a histogram (584) and data representing a second local histogram (538).

[0071] For example, at least one processor (207) can obtain data representing a global histogram (586) by accumulating or adding data representing a second local histogram (532), data representing a second local histogram (534), data representing a second local histogram (536), and data representing a second local histogram (538). For example, the global histogram (586) may not reflect the actual distribution of brightness values ​​of pixels of the source image (415). For example, the global histogram (586) may be a histogram that estimates the distribution of brightness values ​​of pixels of the source image (415). For example, at least one processor (207) can estimate the distribution of brightness values ​​of pixels of the source image (415) using a second set of local histogram information. For example, at least one processor (207) can estimate the distribution of brightness values ​​of pixels of a source image (415) by accumulating a second set of local histogram information. For example, at least one processor (207) can obtain a set of global histogram information representing a global histogram (586) for the entire area of ​​the source image (415) by using the second set of local histogram information representing a second local histogram. For example, at least one processor (207) can obtain a global gain table (e.g., the global gain table (588) of FIG. 5d) by using the set of global histogram information.

[0072] Referring to FIG. 5d, at least one processor (207) can obtain a global gain table (588) using global histogram information representing a global histogram (586). For example, at least one processor (207) can obtain a global gain table (588) when performing a histogram equalization operation on the global histogram (586). For example, the histogram equalization for obtaining the global gain table (588) may be different from the histogram equalization for obtaining a local histogram. For example, at least one processor (207) can obtain a global gain table (588) by applying a cumulative distribution function to the histogram obtained by performing histogram equalization on the global histogram (586).

[0073] For example, at least one processor (207) can obtain a second set of local gain tables by synthesizing or performing an operation on a first set of global gain tables (588) and local gain tables. For example, at least one processor (207) can obtain a second set of local gain tables by performing an operation (e.g., multiplication) on each of the local gain tables of the first set with the global gain table (588). For example, at least one processor (207) can obtain a second set of local gain tables by performing an operation (e.g., multiplication) on the gain corresponding to each pixel represented by each local gain table of the first set and the gain corresponding to each pixel represented by the global gain table (588). For example, the second set of local gain tables may include a second local gain table (592) for region (502), a second local gain table (594) for region (504), a second local gain table (596) for region (506), and a second local gain table (598) for region (508). However, it is not limited thereto. The number of local gain tables in the second set may be based on the number of divided regions of the source image (415).

[0074] Referring again to FIG. 3, in operation 360, at least one processor (207) can generate a result image (435), which is the result of tone mapping for the source image (415), by applying a second set of local gain tables (e.g., a second local gain table (592), a second local gain table (594), a second local gain table (596), and a second local gain table (598)) to a plurality of regions of the source image (415). For example, at least one processor (207) can generate the result image (435) by performing tone mapping using a second set of local gain tables. For example, at least one processor (207) can convert from the source image (415) to the result image (435) by performing tone mapping using a second set of local gain tables.

[0075] According to one embodiment, at least one processor (207) may obtain a result image (435) by applying each of a second set of local gain tables to each region of a source image (415) to generate a result image (435). However, the present disclosure is not limited thereto. For example, at least one processor (207) may determine at least some of the second set of local gain tables to perform interpolation (e.g., bilinear interpolation, bicubic interpolation, spline interpolation, etc.) based on the location of each pixel of the source image (415) (e.g., the region of the source image (415) where each pixel is located). For example, a bicubic interpolation technique may be described as a technique using a two-dimensional polynomial based on the brightness values ​​of a specific number (e.g., 16) of pixels surrounding the pixel to be calculated. For example, a spline interpolation technique can be described as a technique for obtaining a smooth function by using a low-order polynomial for intervals divided from the entire interval. For example, at least one processor (207) can obtain a tone-mapping result image (435) by determining the brightness value of each pixel of the source image (415) using the at least part of the above.

[0076] According to one embodiment, at least one processor (207) can generate a result image (435) by applying local gain tables in a second set of local gain tables to each of a plurality of regions of the source image (415) without applying only the global gain table (588) to the source image (415). For example, at least one processor (207) may refrain from, bypass, or block tone mapping using the global gain table (588) to reduce the number of times tone mapping is performed. For example, at least one processor (207) may perform a single tone mapping on the source image (415) without applying the global gain table (588) to the source image (415). According to one embodiment, when the electronic device (100) performs local tone mapping, it may be required to perform tone mapping twice to reduce the occurrence of grid artifacts in which boundaries between grids are represented. For example, the time required to perform the two tone mappings above may be longer than the time required to perform a single tone mapping according to an embodiment of the present disclosure. For example, when a single tone mapping of the present disclosure is performed, at least one processor (207) may not generate an intermediate image to obtain a result image (435) from a source image (415).

[0077] According to one embodiment, the dynamic range of the resulting image (435) may be wider than the dynamic range of the source image (415). For example, when at least one processor (207) performs tone mapping on the source image (415), histogram equalization is performed, and as a result, the dynamic range of the source image (415) is changed, the resulting image (435) may be generated.

[0078] According to one embodiment, at least one processor (207) can obtain a result image (435) by performing interpolation using a second set of local gain tables. For example, at least one processor (207) can determine the brightness value of each pixel of the result image (435) using a portion of the second set. The acquisition of the result image (435) based on the interpolation is described and illustrated in more detail with reference to FIG. 6.

[0079] FIG. 6 illustrates an exemplary operation of an electronic device that performs interpolation using a second set of local gain tables.

[0080] Referring to FIG. 6, in an image (620), at least one processor (207) can determine the brightness value of a pixel (656) of the image (620) by performing double linear interpolation. For example, an area (622) included in the image (620) may be divided into four parts based on lines (652) and lines (654). For example, a pixel (656) may be included in or located in one of the parts (a) included within the area (622). For example, at least one processor (207) can determine the brightness value of the pixel (656) using a second set of local gain tables (592, 594, 596, 598) corresponding to the areas (622, 624, 626, 628) associated with the part (658) where the pixel (656) is located. For example, when at least one processor (207) performs bilinear interpolation, the local gain tables used to determine the brightness value of the pixel (656) may include local gain tables (592, 594, 596, 598) corresponding to four regions (622, 624, 626, 628) close to the pixel (656).

[0081] For example, one or more regions used to perform bilinear interpolation may be described as regions satisfying a specific condition from a pixel (e.g., pixel (656)) within a region (e.g., region (622)). For example, the specific condition may include being the closest region among regions closer than half the width of the region containing the pixel (e.g., region (622)) and half the height of the region (e.g., region (622)). For example, one or more regions satisfying the specific condition may not overlap among regions within half the width of the region and half the height of the region from the pixel. However, if the number of one or more regions satisfying the specific condition is four or less (e.g., one or two), one or more regions satisfying the specific condition may be used redundantly to perform bilinear interpolation.

[0082] For example, when performing double linear interpolation on a portion located at the top left among the portions of the region (622) divided by line (652) and line (654) (e.g., a portion located at the 9 o'clock to 12 o'clock direction among the portions of the region (622)), the region satisfying the specific condition may be region (622). For example, when performing double linear interpolation on a portion located at the top left among the portions, a local gain table (592) corresponding to region (622) may be used redundantly.

[0083] For example, when performing double linear interpolation on a portion located at the lower left among the portions of area (622) divided by line (652) and line (654) (e.g., a portion located at the 6 o'clock to 9 o'clock direction among the portions of area (622)), the area satisfying the specific condition may include area (622) and area (626). For example, when performing double linear interpolation on a portion located at the lower left among the portions, a local gain table (592) corresponding to area (622) and / or a local gain table (596) corresponding to area (626) may be used redundantly.

[0084] For example, when at least one processor (207) performs bilinear interpolation, local gain tables (592, 594, 596, 598) used to determine the brightness value of a pixel (656) may include a local gain table (592) corresponding to a region (622) containing a part (658) containing the pixel (656), local gain tables (594, 596) corresponding to regions (624, 626) adjacent to the part (658) containing the pixel (656), and a local gain table (598) corresponding to a region (628) adjacent to each of the regions (624, 626) adjacent to the part (658) containing the pixel (656) and sharing one vertex (659) with the part (658).

[0085] For example, at least one processor (207) can determine the brightness value of a pixel (656) by performing bilinear interpolation on the brightness value represented by each of the second set of local gain tables associated with the pixel (656). For example, when acquiring the resulting image (435), at least one processor (207) can determine the brightness value of the pixel (656) using the second set of local gain tables associated with the pixel (656). For example, the pixel (656) can be described as any pixel within the image (620). For example, the second set of local gain tables associated with the pixel (656) can be determined based on the portion of the region (622) that includes the pixel (656). For example, the number of the second set of local gain tables used to determine the brightness value of the pixel (656) can be four.

[0086] At least one processor (207) may perform bilinear interpolation using a second set of local gain tables to determine the brightness value of each pixel included in regions (622, 624, 626, 628) of the image (620). For example, at least one processor (207) may obtain a region (632) of the tone-mapped image (630) from region (622) based on performing the bilinear interpolation. For example, the image (630) may be an example of a resulting image (435). For example, at least one processor (207) may obtain an image (630) including regions (632, 634, 636, 638) based on performing bilinear interpolation for each pixel of regions (622, 624, 626, 628).

[0087] According to one embodiment, the state (600) may be described as a state in which a second local gain table (594), a second local gain table (596), and a second local gain table (598) are selected to perform interpolation for the second local gain table (592). For example, at least one processor (207) may obtain a third set of local gain tables by performing interpolation for each of the multiple regions of the source image (415) using at least a portion of the local gain tables of each of the multiple regions of the source image (415) among the second set of local gain tables. For example, at least one processor (207) may perform interpolation for the designated region by using the second local gain table corresponding to each of the regions adjacent (or surrounding) to the designated region among the multiple regions of the source image (415). For example, at least one processor (207) may use a second local gain table for each of the regions (504), (506), and (508) surrounding the region (502) to perform interpolation for the region (502). For example, at least one processor (207) may perform interpolation (e.g., linear interpolation) for the region (502) using the second local gain table (592), the second local gain table (594), the second local gain table (596), and the second local gain table (598).

[0088] State (610) can be described as a state representing a third local gain table (612) that has been interpolated by the second local gain table (594), the second local gain table (596), and the second local gain table (598). For example, at least one processor (207) can obtain a third set of local gain tables by performing interpolation using a second set of local gain tables. For example, at least one processor (207) can obtain a third set of local gain tables by performing interpolation for one of the local gain tables in the second set of local gain tables using at least some of the remaining local gain tables in the second set, excluding the local gain table. For example, at least one processor (207) can obtain a third set of local gain tables by performing interpolation for each of the local gain tables in the second set using at least some of the local gain tables in the second set. For example, at least one processor (207) can obtain a third set of local gain tables by performing interpolation using a local gain table in a second set of local gain tables for one of the multiple regions of the source image (415), and other local gain tables in a second set of local gain tables for the surrounding regions of the region.

[0089] According to one embodiment, at least one processor (207) can obtain a third set of local mapping tables by performing the interpolation. The third set of local gain tables may be replaced with the third set of local mapping tables. For example, the third set of local mapping tables may be replaced with the third set of local gain tables.

[0090] According to one embodiment, at least one processor (207) can generate a result image (435) using a third set of local gain tables. For example, at least one processor (207) can generate a result image (435) as a result of tone mapping by applying each of the local gain tables of the third set to each of a plurality of regions of the source image (415).

[0091] The electronic device (100) can generate a resulting image (435) (e.g., image (630)) having the effect of local tone mapping and the effect of global tone mapping by performing interpolation (e.g., bilinear interpolation) on each pixel (e.g., pixel (656)) of the source image (415) using a second set of local gain tables. For example, the electronic device (100) can maintain or emphasize local and global characteristics of the source image (415) by performing the interpolation. For example, the electronic device (100) can mitigate grid artifacts by performing the interpolation. For example, the electronic device (100) can mitigate the disparity of the resulting image (435) by performing the interpolation.

[0092] FIG. 7 is a block diagram of an electronic device in a network environment according to various embodiments.

[0093] FIG. 7 is a block diagram of an electronic device (701) in a network environment (700) according to various embodiments. Referring to FIG. 7, in the network environment (700), the electronic device (701) may communicate with an electronic device (702) through a first network (798) (e.g., a short-range wireless communication network) or with at least one of an electronic device (704) or a server (708) through a second network (799) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (701) may communicate with the electronic device (704) through a server (708). According to one embodiment, the electronic device (701) may include a processor (720), memory (730), input module (750), sound output module (755), display module (760), audio module (770), sensor module (776), interface (777), connection terminal (778), haptic module (779), camera module (780), power management module (788), battery (789), communication module (790), subscriber identification module (796), or antenna module (797). In some embodiments, at least one of these components (e.g., connection terminal (778)) may be omitted from the electronic device (701), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (776), camera module (780), or antenna module (797)) may be integrated into a single component (e.g., display module (760)).

[0094] The processor (720) can control at least one other component (e.g., a hardware or software component) of the electronic device (701) connected to the processor (720) by executing software (e.g., a program (740)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (720) can store commands or data received from other components (e.g., a sensor module (776) or a communication module (790)) in volatile memory (732), process the commands or data stored in volatile memory (732), and store the resulting data in non-volatile memory (734). According to one embodiment, the processor (720) may include a main processor (721) (e.g., a central processing unit or an application processor) or an auxiliary processor (723) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (701) includes a main processor (721) and an auxiliary processor (723), the auxiliary processor (723) may be configured to use less power than the main processor (721) or to be specialized for a designated function. The auxiliary processor (723) may be implemented separately from the main processor (721) or as part thereof.

[0095] The auxiliary processor (723) may control at least some of the functions or states associated with at least one component of the electronic device (701) (e.g., display module (760), sensor module (776), or communication module (790)) on behalf of the main processor (721) while the main processor (721) is in an inactive (e.g., sleep) state, or together with the main processor (721) while the main processor (721) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (723) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (780) or communication module (790)). According to one embodiment, the auxiliary processor (723) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (701) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (708)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0096] The memory (730) can store various data used by at least one component of the electronic device (701) (e.g., processor (720) or sensor module (776)). The data may include, for example, software (e.g., program (740)) and input or output data for related commands. The memory (730) may include volatile memory (732) or non-volatile memory (734).

[0097] The program (740) may be stored as software in memory (730) and may include, for example, an operating system (742), middleware (744), or an application (746).

[0098] The input module (750) can receive commands or data to be used for a component of the electronic device (701) (e.g., processor (720)) from outside the electronic device (701) (e.g., user). The input module (750) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0099] The sound output module (755) can output a sound signal to the outside of the electronic device (701). The sound output module (755) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0100] The display module (760) can visually provide information to an external (e.g., user) of the electronic device (701). The display module (760) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (760) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0101] The audio module (770) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (770) can acquire sound through the input module (750) or output sound through the sound output module (755) or an external electronic device (e.g., electronic device (702)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (701).

[0102] The sensor module (776) can detect the operating state of the electronic device (701) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (776) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0103] The interface (777) may support one or more specified protocols that can be used for the electronic device (701) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (702)). According to one embodiment, the interface (777) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0104] The connection terminal (778) may include a connector through which the electronic device (701) can be physically connected to an external electronic device (e.g., electronic device (702)). According to one embodiment, the connection terminal (778) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0105] The haptic module (779) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (779) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0106] The camera module (780) can capture still images and video. According to one embodiment, the camera module (780) may include one or more lenses, image sensors, image signal processors, or flashes.

[0107] The power management module (788) can manage the power supplied to the electronic device (701). According to one embodiment, the power management module (788) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0108] The battery (789) can supply power to at least one component of the electronic device (701). According to one embodiment, the battery (789) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0109] The communication module (790) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (701) and an external electronic device (e.g., electronic device (702), electronic device (704), or server (708)), and the performance of communication through the established communication channel. The communication module (790) may include one or more communication processors that operate independently of the processor (720) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (790) may include a wireless communication module (792) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (794) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (704) through a first network (798) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (799) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (792) can identify or authenticate the electronic device (701) within a communication network such as the first network (798) or the second network (799) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (796).

[0110] The wireless communication module (792) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (792) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (792) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (792) can support various requirements specified in the electronic device (701), external electronic device (e.g., electronic device (704)), or network system (e.g., second network (799)). According to one embodiment, the wireless communication module (792) can support a Peak data rate (e.g., 20 Gbps or more) for eMBB realization, loss coverage (e.g., 164 dB or less) for mMTC realization, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for URLLC realization.

[0111] The antenna module (797) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (797) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (797) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (798) or a second network (799), may be selected from the plurality of antennas, for example, by a communication module (790). The signal or power may be transmitted or received between the communication module (790) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (797).

[0112] According to various embodiments, the antenna module (797) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0113] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0114] According to one embodiment, commands or data may be transmitted or received between the electronic device (701) and an external electronic device (704) through a server (708) connected to a second network (799). Each of the external electronic devices (702, or 704) may be the same or a different type of device as the electronic device (701). According to one embodiment, all or part of the operations performed on the electronic device (701) may be performed on one or more of the external electronic devices (702, 704, or 708). For example, if the electronic device (701) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (701) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (701). The electronic device (701) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (701) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (704) may include an Internet of Things (IoT) device. The server (708) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (704) or the server (708) may be included within the second network (799).The electronic device (701) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0115] The technical problems to be solved in this disclosure are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this disclosure pertains.

[0116] An electronic device as described above (e.g., electronic device (100)) may include a memory (e.g., memory (206)) for storing instructions. The electronic device may include a camera (e.g., camera (209)). The electronic device may include at least one processor (e.g., at least one processor (207)). The instructions may cause the electronic device to identify a source image for performing tone mapping through the camera when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to obtain a first set of local gain tables (e.g., first local gain table (522) to first local gain table (528)) corresponding to each of a plurality of regions of the source image using the identified source image (e.g., source image (415)) when executed individually or collectively by the at least one processor. Each local gain table included in the first set above may include gains to be applied to the brightness values ​​of pixels in the corresponding region of the source image. When the instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain a set of local histogram information corresponding to each of the plurality of regions. The local histogram information within the set may represent the distribution of brightness values ​​of pixels changed by the gains. When the instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain a global gain table (e.g., global gain table (588)) including gains to be applied to pixels in the entire region of the source image using the set of local histogram information.The above instructions may cause the electronic device to obtain a second set of local gain tables (e.g., second local gain table (592) to second local gain table (598)) corresponding to each of the plurality of regions by combining the global gain table and the local gain tables included in the first set when executed individually or collectively by the at least one processor. The above instructions may cause the electronic device to generate a result image (e.g., result image (435)) which is the result of the tone mapping for the source image by applying the local gain tables of the second set to the plurality of regions of the source image when executed individually or collectively by the at least one processor.

[0117] According to one embodiment, the result image can be obtained by performing bilinear interpolation on the plurality of regions of the source image using the second set of local gain tables.

[0118] According to one embodiment, the instructions may cause the electronic device to obtain a third set of local gain tables by performing interpolation using a local gain table in a second set of local gain tables for one (a) of the plurality of regions, and other local gain tables in a second set of local gain tables for the surrounding regions of the region, when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to generate another resulting image, which is the result of the tone mapping for the source image, by applying the local gain tables in the third set of local gain tables to each of the plurality of regions of the source image, when executed individually or collectively by the at least one processor.

[0119] According to one embodiment, the set of local histogram information may be a first set of local histogram information. When the instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain a second set of local histogram information corresponding to each of the plurality of regions of the source image, by using the identified source image, the distribution of brightness values ​​of pixels of the plurality of regions, before obtaining the first set of local gain tables. When the instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain the first set of local gain tables by using the second set of local histogram information.

[0120] According to one embodiment, the instructions may cause the electronic device to acquire global histogram information for the entire area of ​​the source image by accumulating the set of local histogram information when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to acquire the global gain table by using the global histogram information when executed individually or collectively by the at least one processor.

[0121] According to one embodiment, the set of local histogram information may be a first set of local histogram information. When the instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain the first set of local histogram information by applying the first set of local gain tables to a second set of local histogram information representing the distribution of brightness values ​​of pixels of the plurality of regions.

[0122] According to one embodiment, the instructions may cause the electronic device to generate the resulting image by applying local gain tables in a second set of local gain tables to the plurality of regions of the source image, without applying the global gain table to the source image, when executed individually or collectively by the at least one processor.

[0123] According to one embodiment, the first set of local gain tables can be obtained using a cumulative distribution function.

[0124] According to one embodiment, the instructions may cause the electronic device to generate the resulting image based on combining the global gain table and local gain tables in a first set of local gain tables while the electronic device is in a low-power state, when executed individually or collectively by the at least one processor.

[0125] According to one embodiment, the instructions may cause the electronic device to modify the local gain tables in the first set based on the interpolation of at least two local gain tables corresponding to adjacent regions while the electronic device is in a different state different from the low-power state when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to generate an intermediate image, which is the result of local tone mapping based on the local gain tables, by applying gains included in the modified local gain tables in the first set to pixels in the source image when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to obtain another global gain table from the intermediate image when executed individually or collectively by the at least one processor. When the above instructions are executed individually or collectively by the at least one processor, they may cause the electronic device to generate another result image, which is the result of tone mapping for the source image, using the other global gain table.

[0126] A method performed by an electronic device (e.g., electronic device (100)) having a camera (e.g., camera (209)) as described above may include an operation of identifying a source image (e.g., source image (415)) for performing tone mapping through the camera. The method may include an operation of obtaining a first set of local gain tables (e.g., first local gain table (522) to first local gain table (528)) corresponding to each of a plurality of regions of the source image using the identified source image. Each local gain table included in the first set may include gains to be applied to the brightness values ​​of pixels in the corresponding regions of the source image. The method may include an operation of obtaining a set of local histogram information corresponding to each of the plurality of regions. The local histogram information in the set may represent the distribution of brightness values ​​of pixels changed by the gains. The above method may include the operation of obtaining a global gain table (e.g., global gain table (588)) including gains to be applied to pixels of the entire area of ​​the source image using the set of local histogram information. The above method may include the operation of obtaining a second set of local gain tables (e.g., second local gain table (592) to second local gain table (598)) corresponding to each of the plurality of areas by combining the global gain table and the local gain tables included in the first set. The above method may include the operation of generating a result image (e.g., result image (435)) which is the result of the tone mapping for the source image by applying the local gain tables of the second set to each of the plurality of areas of the source image.

[0127] According to one embodiment, the result image can be obtained by performing bilinear interpolation on the plurality of regions of the source image using the second set of local gain tables.

[0128] According to one embodiment, the method may include the operation of obtaining a third set of local gain tables by performing interpolation using a local gain table in a second set of local gain tables for one (a) of the plurality of regions, and other local gain tables in a second set of local gain tables for the surrounding regions of the region. The method may include the operation of generating another result image, which is the result of the tone mapping for the source image, by applying the local gain tables in the third set of local gain tables to each of the plurality of regions of the source image.

[0129] According to one embodiment, the set of local histogram information may be a first set of local histogram information. The method may include, before obtaining the first set of local gain tables, an operation of using the identified source image to represent the distribution of brightness values ​​of pixels in the plurality of regions and obtaining a second set of local histogram information corresponding to each of the plurality of regions of the source image. The method may include an operation of obtaining the first set of local gain tables using the second set of local histogram information.

[0130] According to one embodiment, the method may include an operation of obtaining global histogram information for the entire area of ​​the source image by accumulating a set of local histogram information. The method may include an operation of obtaining a global gain table using the global histogram information.

[0131] According to one embodiment, the set of local histogram information may be a first set of local histogram information. The method may include the operation of obtaining the first set of local histogram information by applying the first set of local gain tables to a second set of local histogram information representing the distribution of brightness values ​​of pixels of the plurality of regions.

[0132] According to one embodiment, the method may include the operation of generating the result image by applying local gain tables within a second set of local gain tables to the plurality of regions of the source image without applying the global gain table to the source image.

[0133] According to one embodiment, the first set of local gain tables can be obtained using a cumulative distribution function.

[0134] According to one embodiment, the method may include an operation of generating the result image based on combining the global gain table and local gain tables in a first set of local gain tables while the electronic device is in a low-power state.

[0135] According to one embodiment, the method may include an operation of modifying the local gain tables in the first set based on the interpolation of at least two local gain tables corresponding to adjacent regions while the electronic device is in a different state different from the low power state. The method may include an operation of generating an intermediate image, which is the result of local tone mapping based on the local gain tables, by applying gains included in the modified local gain tables in the first set to pixels in the source image. The method may include an operation of obtaining another global gain table from the intermediate image. The method may include an operation of generating another result image, which is the result of tone mapping for the source image, using the other global gain table.

[0136] In a computer-readable storage medium in which one or more programs as described above are stored, the one or more programs may include instructions that cause the electronic device (e.g., electronic device (100)) having a camera (e.g., camera (209)) to identify a source image (e.g., source image (415)) for performing tone mapping through the camera when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to obtain a first set of local gain tables (e.g., first local gain table (522) to first local gain table (528)) corresponding to each of a plurality of regions of the source image using the identified source image when executed by the electronic device. Each local gain table included in the first set may include gains to be applied to the brightness values ​​of the pixels of the corresponding regions of the source image. The above one or more programs may include instructions that cause the electronic device to obtain a set of local histogram information corresponding to each of the plurality of regions when executed by the electronic device. The local histogram information within the set may represent the distribution of brightness values ​​of pixels changed by the gains. The above one or more programs may include instructions that cause the electronic device to obtain a global gain table (e.g., global gain table (588)) containing gains to be applied to pixels of the entire region of the source image using the set of local histogram information when executed by the electronic device.The above one or more programs may include instructions that cause the electronic device to obtain a second set of local gain tables (e.g., second local gain table (592) to second local gain table (598)) corresponding to each of the plurality of regions by combining the global gain table and the local gain tables included in the first set when executed by the electronic device. The above one or more programs may include instructions that cause the electronic device to generate a result image (e.g., result image (435)) which is the result of the tone mapping for the source image by applying the local gain tables of the second set to each of the plurality of regions of the source image when executed by the electronic device.

[0137] According to one embodiment, the result image can be obtained by performing bilinear interpolation on the plurality of regions of the source image using the second set of local gain tables.

[0138] According to one embodiment, the one or more programs may include instructions that cause the electronic device to obtain a third set of local gain tables by performing interpolation using a local gain table in a second set of local gain tables for one (a) of the plurality of regions, and other local gain tables in a second set of local gain tables for the surrounding regions of the region when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to generate another result image, which is the result of the tone mapping for the source image, by applying the local gain tables in the third set of local gain tables to each of the plurality of regions of the source image when executed by the electronic device.

[0139] According to one embodiment, the set of local histogram information may be a first set of local histogram information. The one or more programs may include instructions that cause the electronic device to obtain a second set of local histogram information corresponding to each of the plurality of regions of the source image, by using the identified source image, before obtaining the first set of local gain tables. The one or more programs may include instructions that cause the electronic device to obtain the first set of local gain tables by using the second set of local histogram information when executed by the electronic device.

[0140] According to one embodiment, the one or more programs may include instructions that cause the electronic device to acquire global histogram information for the entire area of ​​the source image by accumulating the set of local histogram information when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to acquire the global gain table by using the global histogram information when executed by the electronic device.

[0141] According to one embodiment, the set of local histogram information may be a first set of local histogram information. The one or more programs may include instructions that cause the electronic device to obtain the first set of local histogram information by applying the first set of local gain tables to a second set of local histogram information representing the distribution of brightness values ​​of pixels of the plurality of regions when executed by the electronic device.

[0142] According to one embodiment, the one or more programs may include instructions that cause the electronic device to generate the result image by applying local gain tables in a second set of local gain tables to the plurality of regions of the source image without applying the global gain table to the source image when executed by the electronic device.

[0143] According to one embodiment, the first set of local gain tables can be obtained using a cumulative distribution function.

[0144] According to one embodiment, the one or more programs may include instructions that cause the electronic device to generate the result image based on combining the global gain table and local gain tables in a first set of local gain tables while the electronic device is in a low-power state when executed by the electronic device.

[0145] According to one embodiment, the one or more programs may include instructions that cause the electronic device to modify the local gain tables in the first set based on the interpolation of at least two local gain tables corresponding to adjacent regions while the electronic device is in a different state different from the low power state when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to generate an intermediate image, which is the result of local tone mapping based on the local gain tables, by applying gains included in the modified local gain tables in the first set to pixels in the source image when executed by the electronic device. The one or more programs may include instructions that cause the electronic device to obtain another global gain table from the intermediate image when executed by the electronic device. The above one or more programs may include instructions that cause the electronic device to generate another result image, which is the result of tone mapping for the source image, by using the other global gain table when executed by the electronic device.

[0146] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs.

[0147] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0148] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or instruct the processing unit independently or collectively. Software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0149] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. In this case, the medium may continuously store a computer-executable program, or temporarily store it for execution or download. Additionally, the medium may be various recording or storage means in the form of a single or several combined hardware, and may not be limited to a medium directly connected to a computer system but may exist distributed over a network. Examples of media may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media configured to store program instructions, including ROM, RAM, and flash memory. Additionally, other examples of media may include recording or storage media managed by app stores that distribute applications or sites and servers that supply or distribute various other software.

[0150] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0151] Therefore, other implementations, other embodiments, and equivalents to the claims set forth below are also within the scope of the claims. According to one embodiment, the method according to the various embodiments disclosed herein may be provided as a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created in a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0152] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device, camera; Memory comprising one or more storage media for storing instructions; and It includes at least one processor comprising processing circuitry, and When the above instructions are executed individually or collectively by the at least one processor, Identify a source image for performing tone mapping through the above camera; Using the above-mentioned identified source image, a first set of local gain tables corresponding to each of a plurality of regions of the source image is obtained, and each local gain table included in the first set includes gains to be applied to the brightness values ​​of the pixels of the corresponding region of the source image. A set of local histogram information corresponding to each of the plurality of regions is obtained, and the local histogram information within the set represents the distribution of brightness values ​​of pixels changed by the gains; Using the set of local histogram information above, obtain a global gain table including gains to be applied to pixels of the entire area of ​​the source image; By combining the global gain table and the local gain tables included in the first set, a second set of local gain tables corresponding to each of the plurality of regions is obtained; and By applying the local gain tables of the second set above to the plurality of regions of the source image, a result image is generated, which is the result of the tone mapping for the source image. causing the above electronic device, Electronic device.

2. In claim 1, the result image is, A result obtained by performing bilinear interpolation on the plurality of regions of the source image using the second set of local gain tables. Electronic device.

3. In Claim 1, When the above instructions are executed individually or collectively by the at least one processor, By performing interpolation using a local gain table in a second set of local gain tables for one (a) of the plurality of regions, and other local gain tables in a second set of local gain tables for the surrounding regions of the region, a third set of local gain tables is obtained, and By applying the local gain tables within the third set of the above local gain tables to each of the plurality of regions of the source image, to generate another result image, which is the result of the tone mapping for the source image, causing the above electronic device, Electronic device.

4. In claim 1, the set of local histogram information is, It is the first set of local histogram information, and When the above instructions are executed individually or collectively by the at least one processor, Before obtaining the first set of the above local gain tables, using the identified source image, the distribution of brightness values ​​of pixels in the plurality of regions is represented, and a second set of local histogram information corresponding to each of the plurality of regions of the source image is obtained, and To obtain the first set of local gain tables by using the second set of the local histogram information above, causing the above electronic device, Electronic device.

5. In Claim 1, When the above instructions are executed individually or collectively by the at least one processor, By accumulating the set of local histogram information above, global histogram information for the entire area of ​​the source image is obtained, and To obtain the global gain table using the above global histogram information, causing the above electronic device, Electronic device.

6. In claim 1, the set of local histogram information is, It is the first set of local histogram information, and When the above instructions are executed individually or collectively by the at least one processor, By applying the first set of local gain tables to the second set of local histogram information representing the distribution of brightness values ​​of pixels of the plurality of regions, the first set of local histogram information is obtained. causing the above electronic device, Electronic device.

7. In Claim 1, When the above instructions are executed individually or collectively by the at least one processor, To generate the result image by applying the local gain tables within the second set of local gain tables to the plurality of regions of the source image without applying the global gain table to the source image. causing the above electronic device, Electronic device.

8. In claim 1, the first set of local gain tables is, Obtained using the cumulative distribution function, Electronic device.

9. In Claim 1, When the above instructions are executed individually or collectively by the at least one processor, While the electronic device is in a low-power state, to generate the result image based on combining the global gain table and the local gain tables in the first set of the local gain tables, causing the above electronic device, Electronic device.

10. In Claim 9, When the above instructions are executed individually or collectively by the at least one processor, While the electronic device is in a different state different from the low-power state, the local gain tables in the first set are modified based on the interpolation of at least two local gain tables corresponding to adjacent regions, and By applying the gains included in the modified local gain tables within the first set to the pixels in the source image, an intermediate image is generated, which is the result of local tone mapping based on the local gain tables, and Obtain another global gain table from the above intermediate image, and Using the above different global gain table, to generate another result image, which is the result of tone mapping for the source image, causing the above electronic device, Electronic device.

11. In a non-transient computer-readable storage medium storing one or more programs, said one or more programs are, When executed by an electronic device having a camera, Identify a source image for performing tone mapping through the above camera; Using the above-mentioned identified source image, a first set of local gain tables corresponding to each of a plurality of regions of the source image is obtained, and each local gain table included in the first set includes gains to be applied to the brightness values ​​of the pixels of the corresponding region of the source image. A set of local histogram information corresponding to each of the plurality of regions is obtained, and the local histogram information within the set represents the distribution of brightness values ​​of pixels changed by the gains; Using the set of local histogram information above, obtain a global gain table including gains to be applied to pixels of the entire area of ​​the source image; By combining the global gain table and the local gain tables included in the first set, a second set of local gain tables corresponding to each of the plurality of regions is obtained; and By applying the local gain tables of the second set above to the plurality of regions of the source image, a result image is generated, which is the result of the tone mapping for the source image. Including instructions that cause the above electronic device, Non-transient computer-readable storage media.

12. In claim 11, the result image is, A result obtained by performing bilinear interpolation on the plurality of regions of the source image using the second set of local gain tables. Non-transient computer-readable storage media.

13. In Claim 11, When the above one or more programs are executed by the electronic device, By performing interpolation using a local gain table in a second set of local gain tables for one (a) of the plurality of regions, and other local gain tables in a second set of local gain tables for the surrounding regions of the region, a third set of local gain tables is obtained, and By applying the local gain tables within the third set of the above local gain tables to each of the plurality of regions of the source image, to generate another result image, which is the result of the tone mapping for the source image, Including instructions that cause the above electronic device, Non-transient computer-readable storage media.

14. In claim 11, the set of local histogram information is, It is the first set of local histogram information, and When the above one or more programs are executed by the electronic device, Before obtaining the first set of the above local gain tables, using the identified source image, the distribution of brightness values ​​of pixels in the plurality of regions is represented, and a second set of local histogram information corresponding to each of the plurality of regions of the source image is obtained, and To obtain the first set of local gain tables by using the second set of the local histogram information above, Including instructions that cause the above electronic device, Non-transient computer-readable storage media.

15. In Claim 11, When the above one or more programs are executed by the electronic device, By accumulating the set of local histogram information above, global histogram information for the entire area of ​​the source image is obtained, and To obtain the global gain table using the above global histogram information, Including instructions that cause the above electronic device, Non-transient computer-readable storage media.

Images