Electronic device for generating HDR image and method for operating the same

Abstract

A method for generating a high dynamic range (HDR) image is provided, including: obtaining a plurality of images captured at different exposure levels; generating at least one transformation matrix indicating corresponding regions between the plurality of images; determining, based on the transformation matrix, the degree of movement of each corresponding region between the plurality of images; generating, based on the determined degree of movement, a blending map indicating a blending ratio of the plurality of images at each pixel; and generating an HDR image by blending the plurality of images based on the blending map.

Description

Technical Field

[0001] The present invention relates to an electronic device for generating a high dynamic range (HDR) image and an operating method thereof. More specifically, the present invention relates to an electronic device for generating an HDR image by synthesizing a plurality of images and an operating method thereof. Background Art

[0002] Natural scenes typically have a high dynamic range (HDR) that exceeds the capture range of ordinary digital cameras. Digital photos of natural scenes may not provide sufficient details in underexposed or overexposed areas.

[0003] Therefore, a method for generating an HDR image from images captured at different exposure levels is needed, and the HDR image provides sufficient details even in underexposed or overexposed areas of digital photos.

[0004] When a plurality of static still images are blended, an HDR image can be generated, but when there is movement of an object between the still images, due to the movement of the object, ghost artifacts may be generated in the HDR image.

[0005] In addition, when a plurality of static still images are blended, saturated areas may be generated when the pixel values of the still images are added to each other. Summary of the Invention

[0006] An electronic device for generating a high dynamic range (HDR) image and an operating method thereof are provided.

[0007] In addition, a computer-readable recording medium having a program for executing the method on a computer recorded thereon is provided. The technical problems to be solved are not limited to the above technical problems, and there may be other technical problems. Brief Description of the Drawings

[0008] Figure 1 is a block diagram for describing an example of generating a high dynamic range (HDR) image according to an embodiment.

[0009] Figure 2 is a block diagram for describing an internal configuration of an electronic device according to an embodiment.

[0010] Figure 3 is a block diagram for describing an internal configuration of an electronic device according to an embodiment.

[0011] Figure 4 is a flowchart of a method for generating an HDR image according to an embodiment.

[0012] Figure 5It is a diagram showing examples of images with different exposures according to an embodiment.

[0013] Figure 6 It is a block diagram for describing a method of matching feature points according to an embodiment.

[0014] Figure 7 It is a diagram showing examples of global feature points and local feature points according to an embodiment.

[0015] Figure 8 It is a block diagram for describing a method of generating a blended texture according to an embodiment.

[0016] Figure 9 It is a diagram showing an example of exposure difference according to an embodiment.

[0017] Figure 10 It is a diagram showing an example of movement difference according to an embodiment.

[0018] Figure 11 It is a diagram showing an example of blending images with different exposures according to the image pyramid method according to an embodiment.

[0019] Figure 12 It is a block diagram for describing a method of blending multiple images based on a blended texture according to an embodiment.

[0020] Figure 13 It is a diagram showing an example of a blended image according to an embodiment. Detailed implementation

[0021] According to an embodiment of the present invention, a method for generating a high dynamic range (HDR) image includes: obtaining a plurality of images captured with different exposure levels; generating at least one transformation matrix indicating corresponding regions between the plurality of images; determining, based on the transformation matrix, a movement degree of each corresponding region between the plurality of images; generating a blending map indicating a blending ratio of the plurality of images in each pixel based on the determined movement degree; and generating an HDR image by blending the plurality of images based on the blending map.

[0022] According to another embodiment of the present invention, an electronic device for generating a high dynamic range (HDR) image includes: a camera configured to obtain a plurality of images captured with different exposure levels; at least one processor configured to: generate at least one transformation matrix indicating corresponding regions between the plurality of images; determine, based on the transformation matrix, a movement degree of each corresponding region between the plurality of images; generate a blending map indicating a blending ratio of the plurality of images in each pixel based on the determined movement degree; and generate an HDR image by blending the plurality of images based on the blending map; and a memory storing the generated HDR image.

[0023] According to another embodiment of the present invention, a computer-readable recording medium has a program recorded thereon for implementing a method.

[0024] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the present invention. However, the present invention can be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description are omitted for clear description of the present disclosure, and like reference numerals designate like elements throughout the specification.

[0025] Throughout the specification, when one part is "connected" to another part, the part can not only be "directly connected" to the other part, but also be "electrically connected" to the other part through another part. In addition, unless otherwise specified, when one part "includes" a certain element, the part can further include another element without excluding other elements.

[0026] Hereinafter, the present disclosure will be described with reference to the accompanying drawings.

[0027] Figure 1 is a block diagram for describing an example of generating a high dynamic range (HDR) image according to an embodiment.

[0028] An HDR image according to an embodiment can be generated as a plurality of matching static images. The plurality of images according to an embodiment can be still images captured with different exposures.

[0029] Reference Figure 1 , when processing a plurality of images captured with different exposures in sequence via feature point matching 110, blend texture generation 120, and blending 130, an HDR image according to an embodiment can be generated.

[0030] According to an embodiment, when an object in an image moves while capturing a plurality of images, due to the afterimage of the object included in the image, ghost artifacts may be generated in each image. According to an embodiment, blend texture generation 120 can be performed so as to reduce ghost artifacts in the HDR image.

[0031] In addition, according to an embodiment, when blending still images during blending 130, halo artifacts may be generated due to the generation of saturated regions. According to an embodiment, blending 130 can be performed so as to reduce halo artifacts in the HDR image.

[0032] According to the feature point matching 110 according to an embodiment, corresponding feature points between multiple images can be matched. For example, global feature points corresponding to features (such as lines, corners, etc.) of an image included in the entire image can be extracted. For example, the global feature points can be oriented features from the Features from Accelerated Segment Test (FAST) and Rotated Binary Robust Independent Elementary Features (BRIEF) (ORB) feature points, but are not limited thereto, and can include various types of feature points indicating image features.

[0033] According to an embodiment, local feature points extracted based on the global feature points can be additionally extracted. According to an embodiment, the hybrid texture generation 120 and the blending 130 can be performed based on the matching results of the global feature points and the local feature points.

[0034] According to an embodiment, after the global feature points between images are matched with each other, block search can be performed using a radius set based on the global feature points of each image. In addition, according to the block search, local feature points can be additionally extracted and matched between images. The local feature points that are matched to outlier positions among the local feature points extracted according to an embodiment can be determined to correspond to noise caused by the movement of an object in the image, and can be excluded from the feature points used for subsequently determining the transformation matrix. According to an embodiment, the transformation matrix indicating corresponding pixels between multiple images can be determined based on the matching of the global feature points and the remaining local feature points after excluding the outliers. Therefore, according to the transformation matrix generated according to an embodiment, when the local feature points that are matched to the outlier positions are excluded, the consistency of the corresponding image regions can be maintained.

[0035] According to the hybrid texture generation 120 according to an embodiment, the ratio of blending multiple images can be determined for each pixel matched between images according to the transformation matrix.

[0036] The hybrid texture according to an embodiment can include weight values of multiple images applicable to each pixel. According to an embodiment, when the pixel values of images to which different weight values are applied are added according to the hybrid texture during the blending 130, an HDR image can be generated.

[0037] The hybrid texture according to an embodiment can be determined based on a deghosting map indicating a blending ratio determined based on the degree of movement of an object in the image and a well-exposed map indicating a blending ratio determined based on pixel intensity. For example, in each pixel, the hybrid texture can be determined according to a value obtained by simply multiplying the blending ratios determined according to the deghosting map and the well-exposed map.

[0038] The ghosting removal map according to an embodiment may indicate, for each pixel, a blending ratio determined based on the degree of movement of an object. According to an embodiment, a reference image and at least one non-reference image may be determined for a plurality of images, and the degree of movement of each region may be determined by comparing the reference image and the non-reference image. The degree of movement according to an embodiment may be determined for each tile including a plurality of pixels. The blending ratio of the ghosting removal map according to an embodiment may be determined such that the degree of the non-reference image to be blended when generating an HDR image decreases as the degree of movement increases.

[0039] The well-exposed map according to an embodiment may be determined based on the intensity value determined for each pixel. The intensity value according to an embodiment may be determined as a value of 0 or 1 based on the pixel value of each pixel. According to an embodiment, the intensity value may be determined for the pixels of a plurality of images, or the blending ratio may be determined depending on whether the intensity values of the matching pixels are close to 0.5. For example, the blending ratio of the pixel having the intensity value closest to 0.5 among the matching pixels of the image may be determined to be the highest.

[0040] According to the blending 130 according to an embodiment, when a plurality of images are blended based on the weight value of each pixel of each image determined according to the blending map, an HDR image may be generated.

[0041] According to an embodiment, a plurality of images may be gradually blended according to the weight value through the blending map. During the main blending according to an embodiment, the weight values W1 and 1-W1 of the non-reference image may be respectively applied to the reference image and the non-reference image to be blended. In addition, during the secondary blending, the weight values W3 and 1-W3 of another non-reference image may be respectively applied to the image of the main blending to be blended and another non-reference image.

[0042] According to an embodiment, in the current step, blending may be performed based on the feature regarding the detail of the non-reference image to be blended. For example, the number of pyramid levels indicating the number of the reference image and the non-reference image decomposed into different resolutions for blending may be determined, and the reference image and the non-reference image may be decomposed into different resolutions by the determined number of pyramid levels, thereby performing blending.

[0043] Mixing according to an embodiment can be performed because the weight values according to the mixing map are applied to each image in each layer generated when an image is decomposed into different resolutions. In the mixing according to the embodiment, features based on a non-reference image are combined into a reference image, and when determining the number of the reference image and the non-reference image decomposed into different resolutions, an HDR image in which details of the non-reference image are reflected as much as possible and halation artifacts are reduced can be generated, such that an optimal HDR image can be generated.

[0044] Operations such as tone mapping, noise filtering, and edge enhancement may be further performed on the HDR image, in which a plurality of images are mixed according to mixing 130 according to an embodiment. However, the present disclosure is not limited thereto, and various types of operations for optimizing the HDR image may be further performed on the HDR image completed by mixing 130.

[0045] Figure 2 is a block diagram for describing an internal configuration of an electronic device 1000 according to an embodiment.

[0046] Figure 3 is a block diagram for describing an internal configuration of an electronic device 1000 according to an embodiment.

[0047] The electronic device 1000 according to an embodiment may be implemented in various forms. For example, the electronic device 1000 described herein may be a digital camera, a smart phone, a laptop computer, a tablet PC, an e-book terminal, a digital broadcast terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, an MP3 player, a video phone, an e-book reader, a desktop PC, a workstation, a medical device, a camera, or a smart home appliance (e.g., a TV, a refrigerator, an air conditioner, a cleaner, or a set-top box), but is not limited thereto.

[0048] In addition, the electronic device 1000 described herein may be a wearable device of a user. Examples of the wearable device include at least one of an accessory type device (e.g., a watch, a ring, a wristband, an ankle band, a necklace, glasses, or contact lenses), a head-mounted device (HMD), a textile or clothing integrated device (e.g., an electronic clothing), an attachable device (e.g., a skin pad), or a bio-implantable device (e.g., an implantable circuit), but is not limited thereto. Hereinafter, for ease of description, an example in which the electronic device 1000 is a smart phone will be described.

[0049] Reference Figure 2 to, the electronic device 1000 may include a camera 1610, a processor 1300, and a memory 1700. However, Figure 2 the components shown in Figure 2more or fewer components than those shown.

[0050] For example, as Figure 3 shown, in addition to the camera 1610, the processor 1300, and the memory 1700, the electronic device 1000 according to some embodiments may further include a user inputter 1100, an outputter 1200, a sensing unit 1400, a communicator 1500, and an audio / video (A / V) inputter 1600.

[0051] The user inputter 1100 is a unit that inputs data for a user to control the electronic device 1000. For example, the user inputter 1100 may include a keyboard, a dome switch, a touchpad (capacitive touch type, piezoresistive type, infrared (IR) detection type, surface acoustic wave conduction type, integrated tension measurement type, piezoelectric effect type, etc.), a jog wheel, a jog switch, etc., but is not limited thereto.

[0052] According to an embodiment, the user inputter 1100 may receive a user input for generating an HDR image. For example, based on the user input received by the user inputter 1100, a plurality of still images may be captured, and an HDR image may be generated from the plurality of still images.

[0053] The outputter 1200 may output an audio signal, a video signal, or a vibration signal, and the outputter 1200 may include a display 1210, a sound outputter 1220, and a vibration motor 1230.

[0054] The display 1210 displays information processed by the electronic device 1000. According to an embodiment, the display 1210 may display an HDR image generated from a plurality of images.

[0055] When the display 1210 is configured as a touch screen by forming a layer structure with a touchpad, the display 1210 may be used as both an input device and an output device. The display 1210 may include at least one of a liquid crystal display, a thin film transistor liquid crystal display, an organic light emitting diode, a flexible display, a three-dimensional (3D) display, or an electrophoretic display. In addition, according to an embodiment of the electronic device 1000, the electronic device 1000 may include two or more displays 1210.

[0056] The sound outputter 1220 outputs audio data received from the communicator 1500 or stored in the memory 1700. The vibration motor 1230 may output a vibration signal. In addition, when a touch is input on the touch screen, the vibration motor 1230 may output a vibration signal.

[0057] According to an embodiment, the sound outputter 1220 and the vibration motor 1230 may be used to output information related to the HDR image generated according to the embodiment. For example, the sound outputter 1220 and the vibration motor 1230 may output information related to the HDR image displayed on the display 1210.

[0058] The processor 1300 generally controls all operations of the electronic device 1000. For example, the processor 1300 may generally execute a program stored in the memory 1700 to control the user inputter 1100, the outputter 1200, the sensing unit 1400, the communicator 1500, and the A / V inputter 1600.

[0059] The electronic device 1000 may include at least one processor 1300. For example, the electronic device 1000 may include various types of processors, such as a central processing unit (CPU), a graphics processing unit (GPU), and a neural processing unit (NPU).

[0060] The processor 1300 may be configured to process commands of a computer program by performing basic arithmetic, logical, and input / output operations. The commands may be provided to the processor 1300 from the memory 1700, or may be provided to the processor 1300 by being received via the communicator 1500. For example, the processor 1300 may be configured to execute commands according to program code stored in a recording device such as the memory (1700).

[0061] According to an embodiment, the processor 1300 may detect global feature points and local feature points from multiple images captured with different exposures, and match the global feature points and the local feature points between the multiple images. In addition, the processor 1300 may generate a transformation matrix representing corresponding regions between the multiple images based on the matching result.

[0062] According to an embodiment, the processor 1300 may determine the degree of movement of each image by comparing corresponding regions between the multiple images based on the transformation matrix, and generate a blending map based on the degree of movement. The blending map according to an embodiment may be generated by determining a weight value to be applied to the pixels of each image according to the degree of movement.

[0063] An HDR image may be generated by blending multiple images based on the blending map generated according to an embodiment. The processor 1300 according to an embodiment may gradually blend the multiple images based on the weight value determined according to the blending map. According to an embodiment, in the current step, when blending is performed based on features regarding the details of a non-reference image blended with a reference image, an HDR image in which the details of the non-reference image are reflected as much as possible and halo artifacts are reduced may be generated.

[0064] For example, mixing between a reference image and a first non-reference image may be first performed based on the weight value of the first non-reference image. In addition, mixing between the image obtained by mixing the reference image and the first non-reference image and a second non-reference image may be performed based on the weight value of the second non-reference image.

[0065] The sensing unit 1400 may detect the state of the electronic device 1000 or the state around the electronic device 1000, and send the detected information to the processor 1300.

[0066] The sensing unit 1400 may include at least one of a geomagnetic sensor 1410, an acceleration sensor 1420, a temperature / humidity sensor 1430, an infrared sensor 1440, a gyro sensor 1450, a position sensor 1460 (e.g., Global Positioning System (GPS)), an atmospheric pressure sensor 1470, a proximity sensor 1480, or a red, green, blue (RBG) sensor 1490 (illuminance sensor), but is not limited thereto.

[0067] According to an embodiment, an HDR image may be generated based on the information detected by the sensing unit 1400. For example, an HDR image may be generated based on the image captured by the RBG sensor 1490 of the sensing unit 1400.

[0068] The communicator 1500 may include one or more components that enable the electronic device 1000 to communicate with a server or an external device (not shown). For example, the communicator 1500 may include a short-range wireless communicator 1510, a mobile communicator 1520, and a broadcast receiver 1530.

[0069] The short-range wireless communicator 1510 may include a Bluetooth communicator, a low-power Bluetooth (BLE) communicator, a near-field communicator, a wireless local area network (WLAN) (Wi-Fi) communicator, a Zigbee communicator, an infrared data association (IrDA) communicator, a Wi-Fi direct (WFD) communicator, an ultra-wideband (UWB) communicator, or an Ant+ communicator, but is not limited thereto.

[0070] The mobile communicator 1520 may send or receive a wireless signal to / from at least one of a base station, an external terminal, or a server on a mobile communication network. Here, the wireless signal may include various types of data exchanged according to a voice call signal, an image call signal, or a text / multimedia message.

[0071] The broadcast receiver 1530 may receive a broadcast signal and / or broadcast-related information from an external source through a broadcast channel. The broadcast channel may include a satellite channel or a terrestrial channel. According to an embodiment, the electronic device 1000 may not include the broadcast receiver 1530.

[0072] The communicator 1500 according to an embodiment may receive information required to generate an HDR image. For example, the communicator 1500 may receive a plurality of images for generating an HDR image from an external source.

[0073] The A / V inputter 1600 is a unit into which an audio signal or a video signal is input, and may include a camera 1610 and a microphone 1620. The camera 1610 may obtain an image frame such as a still image or a moving image via an image sensor in an image call mode or a photographing mode. The image captured via the image sensor may be processed via the processor 1300 or a separate image processor (not shown).

[0074] The microphone 1620 receives an external sound signal and processes the external sound signal into electrical voice data.

[0075] According to an embodiment, an HDR image may be generated based on a plurality of images captured by the A / V inputter 1600.

[0076] The memory 1700 may store programs for being processed and controlled by the processor 1300, and may store data input to or output from the electronic device 1000.

[0077] The memory 1700 according to an embodiment may store data required to generate an HDR image. For example, the memory 1700 may store a plurality of images for generating an HDR image. In addition, the memory 1700 may store an HDR image generated according to an embodiment.

[0078] The memory 1700 may include at least one type of storage medium such as a flash memory type, a hard disk type, a multimedia card micro, a card type memory (e.g., a secure digital (SD) or an extreme digital (XD) memory), a random access memory (RAM), a static RAM (SRAM), a read only memory (ROM), an electrically erasable programmable ROM (EEPROM), a programmable ROM (PROM), a magnetic memory, a magnetic disk, or an optical disk.

[0079] The programs stored in the memory 1700 may be divided into a plurality of modules based on functions, and may be divided into, for example, a user interface (UI) module 1710, a touch screen module 1720, and a notification module 1730.

[0080] The UI module 1710 may provide a dedicated UI or a graphical user interface (GUI) that can interact with the electronic device 1000 for each application. The touch screen module 1720 may detect a user's touch gesture on the touch screen and send information about the touch gesture to the processor 1300. According to some embodiments, the touch screen module 1720 may recognize and analyze touch codes. The touch screen module 1720 may be configured as independent hardware including a controller.

[0081] Various sensors may be provided inside or near the touch screen to sense a touch or a near touch on the touch screen. Examples of sensors for sensing a touch on the touch screen include haptic sensors. A haptic sensor is a sensor that senses the degree of contact with a specific object to be greater than or equal to what a human can feel. The haptic sensor may sense various types of information, such as the roughness of the contact surface, the stiffness of the contacting object, and the temperature of the contact point.

[0082] A user's touch gesture may include a tap, a hold, a double - tap, a drag, a pan, a flick, a drag - and - drop, a slide, etc.

[0083] The notification module 1730 may generate a signal for notifying the occurrence of an event in the electronic device 1000.

[0084] Figure 4 is a flowchart of a method for generating an HDR image according to an embodiment.

[0085] Reference Figure 4 , in operation 410, the electronic device 1000 may obtain a plurality of images captured with different exposures. For example, the electronic device 1000 may obtain a plurality of images by using a camera included in the electronic device 1000 and capturing images with different exposure settings. According to an embodiment, the electronic device 1000 may obtain a plurality of images by receiving from the outside a plurality of images captured with different exposures.

[0086] In operation 420, the electronic device 1000 may generate a transformation matrix representing corresponding regions between the plurality of images. The transformation matrix according to an embodiment may be generated based on a matching result of global feature points and local feature points extracted from the plurality of images.

[0087] According to an embodiment, global feature points may be detected from the plurality of images, and corresponding feature points between the global feature points detected from each of the plurality of images may be matched with each other.

[0088] A global feature point according to an embodiment may be a point indicating a feature region of an image, and may be, for example, an ORB feature point. The global feature point is not limited thereto, and may be various types of feature points capable of indicating a feature region of the entire image. According to an embodiment, after global feature points are detected from each image, corresponding global feature points between the images may be matched.

[0089] According to an embodiment, the result of matching global feature points may be represented as a transformation matrix capable of indicating corresponding regions between two images. However, the result of matching global feature points is not limited thereto, and may be represented by various methods indicating corresponding regions between multiple images.

[0090] In operation 430, the electronic device 1000 may additionally detect local feature points based on the global feature points detected in operation 420. According to an embodiment, the electronic device 1000 may detect local feature points of multiple images by performing block search based on the result of matching global feature points. The detected local feature points may be matched with each other between multiple images.

[0091] According to an embodiment, one of the multiple images may be determined as a reference image. For example, an image having a middle exposure among the multiple images may be determined as the reference image. The reference image may be divided into multiple blocks, and the center point of the block may be determined as a local feature point.

[0092] The electronic device 1000 according to an embodiment may perform block search to search for blocks of a non-reference image corresponding to the blocks of the reference image, respectively. The block search according to the embodiment may be performed according to the sum of absolute difference (SAD) method for searching for a block by using the sum of differences between the block of the reference image and the block of the non-reference image.

[0093] According to an embodiment, based on the result of matching global feature points, a region of the non-reference image for any block whose correspondence to the reference image is to be determined may be determined. For example, based on the transformation matrix generated according to the result of matching global feature points, a region of the non-reference image corresponding to the block region of the reference image may be determined as the region of the non-reference image for which block search is to be performed.

[0094] According to an embodiment, a region of a non-reference image for which block search is to be performed may be determined based on similarity between two images, and the similarity may be determined according to a normalized cross correlation (NCC) method. For example, a region for which block search is to be performed may be determined from a region of the non-reference image based on the similarity between two images determined according to the NCC method, the region corresponding to a block region of the reference image and indicated by a transformation matrix generated based on a result of global feature point matching.

[0095] According to an embodiment, a center point of a block of the non-reference image (corresponding to a block of the reference image) may be detected as a local feature point based on a result of performing block search. The local feature point may be matched with a local feature point of a block of the reference image.

[0096] However, embodiments are not limited thereto, and the electronic device 1000 may detect local feature points via various methods for performing block search based on a result of matching global feature points, and match local feature points between multiple images.

[0097] According to an embodiment, local feature points with inconsistent matching results may be excluded and not used for generating a transformation matrix. For example, different from matching results of adjacent local feature points, a local feature point where an object has moved is matched with a relatively distant local feature point, and thus the matching results may be inconsistent. The electronic device 1000 according to an embodiment may generate a blended map based on global feature points and remaining local feature points after excluding inconsistent local feature points.

[0098] Therefore, according to an embodiment, consistency of matching image regions may be maintained in the transformation matrix because local feature points where an object has moved are excluded.

[0099] In operation 430, the electronic device 1000 may determine a degree of movement of each corresponding region between images based on the transformation matrix generated in operation 420. In addition, in operation 440, the electronic device 1000 may generate a blended map based on the degree of movement.

[0100] The blended map according to an embodiment may include a weight value between 0 and 1, and the weight value indicates a ratio of blending multiple images for each pixel. According to an embodiment, a weight value of the finally obtained blended map may be determined such that a sum of weight values of images for one pixel is 1. For example, according to the blended map, an HDR image based on multiple images may be generated by applying the weight value to pixels of multiple images.

[0101] The blended map according to an embodiment may be obtained based on a deghosting map determined based on the degree of movement and a well-exposed map determined based on pixel intensity.

[0102] When determining the weight value to be applied to each image based on the degree of movement determined for each pixel according to the image difference between the reference image and the non-reference image, a deghosting map according to an embodiment can be generated. For example, a deghosting map can be generated by determining a mixing ratio that indicates the degree to which pixels of the non-reference image compared to the reference image are reflected in the HDR image based on the degree of movement.

[0103] The image difference according to an embodiment can be generated by the exposure, noise, and movement of an object. According to an embodiment, the mixing ratio can be determined in a region where the image difference is generated by at least one of exposure or noise, such that details of the non-reference image can be significantly reflected. Meanwhile, in a region where the image difference is generated by the movement of an object, the non-reference image may include ghost artifacts, and thus the mixing ratio of the non-reference image can be determined to be low.

[0104] A well-exposed map according to an embodiment can be generated based on the mixing ratio determined according to the intensity of each pixel. For example, when the pixel intensity, which can have a value between 0 and 1, is close to 0.5, a mixing ratio with a larger value can be determined because this region may include more details.

[0105] When simply multiplying the weight value determined according to the deghosting map and the weight value determined according to the well-exposed map, a mixed map according to an embodiment can be generated. However, the embodiment is not limited thereto, and the mixed map can be generated via various methods based on the deghosting map and the well-exposed map.

[0106] In operation 450, the electronic device 1000 can generate an HDR image by mixing multiple images based on the mixed map determined in operation 440. The mixed map according to an embodiment can include weight values that can be applied to each pixel included in the multiple images.

[0107] The non-reference image among the multiple images mixed according to an embodiment can be an image in which pixels in a region including the movement of an object determined when generating the mixed map are replaced with pixels of the reference image.

[0108] According to an embodiment, multiple images can be mixed step by step. For example, multiple sub-mixings can be performed according to the number of non-reference images according to conditions set according to the characteristics of the non-reference image mixed with the reference image. Therefore, according to an embodiment, an HDR image in which the generation of halo artifacts is reduced and details included in the non-reference image are reflected as much as possible can be generated.

[0109] For example, the blending between the reference image and the first non-reference image may be first performed based on the weight value of the first non-reference image. In addition, the blending between the image obtained by blending the reference image and the first non-reference image and the second non-reference image may be performed based on the weight value of the second non-reference image.

[0110] Tone mapping, noise filtering, and edge enhancement may be further performed on the image in which multiple images are blended based on the blending map according to the embodiment. However, the embodiment is not limited thereto, and various operations for image optimization may be performed on the HDR image.

[0111] Figure 5 FIG. is a diagram showing an example of images with different exposures according to the embodiment.

[0112] Reference Figure 5 , the HDR image 504 according to the embodiment may be generated based on multiple images 501, 502, and 503 with different exposures. The multiple images 501, 502, and 503 according to the embodiment may include a short-exposure image 501, a medium-exposure image 502, and a long-exposure image 503 according to the exposure. The exposure differences between the images may be uniform or different, but are not limited thereto, and images with various exposure differences may be used for HDR image generation.

[0113] The multiple images 501, 502, and 503 with different exposures according to the embodiment may be images sequentially captured by one camera. Alternatively, the multiple images 501, 502, and 503 with different exposures may be images captured by using multiple different cameras.

[0114] Among the multiple images 501, 502, and 503 according to the embodiment, the image that is likely to include relatively more details may be set as the reference image, and the remaining images may be set as non-reference images. According to the embodiment, the medium-exposure image 502 includes most details, and thus may be set as the reference image.

[0115] According to the embodiment, the HDR image may be generated based on three or more images. And, the medium-exposure image among the three or more images may be set as the reference image to generate the HDR image.

[0116] Figure 6 FIG. is a block diagram for describing a method of matching feature points according to the embodiment.

[0117] Reference Figure 6, a transformation matrix between a non-reference image and a reference image can be obtained through feature point matching. The transformation matrix according to the embodiment may include corresponding regions between the images. For example, when the reference image is a medium-exposure image 502, a transformation matrix between the reference image and a short-exposure image 501 and a transformation matrix between the reference image and a long-exposure image 503 can be obtained.

[0118] The feature points according to the embodiment may include global matching points and local feature points, and the local feature points may be feature points that can be extracted from the image based on the global matching points.

[0119] During global feature point extraction and matching 610, the electronic device 1000 may extract global feature points by comparing the non-reference image and the reference image, and match the corresponding global feature points between the images. The global feature points according to the embodiment may include various types of feature points indicating image features.

[0120] During transformation matrix generation 620, the electronic device 1000 may generate a transformation matrix between the non-reference image and the reference image according to the matching result of the global feature points. The transformation matrix according to the embodiment may indicate the corresponding regions between the non-reference image and the reference image determined according to the matching result of the global feature points.

[0121] During local feature point extraction and matching 630, the electronic device 1000 may extract local feature points from the non-reference image and the reference image based on the transformation matrix generated according to the matching result of the global feature points, and match the local feature points. The local feature points of the reference image according to the embodiment may be determined as the center points of each block generated by dividing the reference image into multiple blocks.

[0122] The local feature points of the reference image according to the embodiment may be matched with the local feature points of the non-reference image through block search performed on the non-reference image. According to the embodiment, the electronic device 1000 may search for the block of the non-reference image corresponding to the block of the local feature point of the reference image through block search. In addition, the electronic device 1000 may extract the center point of the found block of the reference image as the local feature point matching the local feature point of the reference image. The block search according to the embodiment may be performed on the region of the non-reference image corresponding to the block of the reference image determined according to the transformation matrix.

[0123] During transformation matrix generation 640, the electronic device 1000 may generate a transformation matrix H between the reference image and the non-reference image based on the matching of the global feature points and the local feature points.

[0124] Among the local feature points according to an embodiment, local feature points whose matching results are inconsistent with those of adjacent local feature points can be excluded from the generation of the transformation matrix. For example, when the positions of the matched local feature points obtained due to the movement of an object are relatively far from the positions of the local feature points of the matched background, the matched local feature points can be excluded.

[0125] The transformation matrix according to an embodiment can be generated for each non-reference image. For example, a transformation matrix between a reference image and a short-exposure image 501, and a transformation matrix between the reference image and a long-exposure image 503 can be generated.

[0126] Figure 7 is a diagram showing examples of global feature points and local feature points according to an embodiment.

[0127] Figure 7 Reference numerals 710 and 720 thereof respectively show the global feature points 711 and 721 of the reference image and the non-reference image, and reference numerals 730 and 740 together show the global feature points and local feature points 731 and 741 of the reference image and the non-reference image respectively.

[0128] Referring to the global feature points 711 and 721 shown in reference numerals 710 and 720, the global feature points only cover a partial area of the image. Therefore, many errors may occur in the transformation matrix generated only by the global feature points. However, different from the global feature points, the local feature points 731 and 741 shown in reference numerals 730 and 740 can cover most of the area of the image. Therefore, the transformation matrix based on the local feature points and the global feature points may have less error than the transformation matrix generated only using the global feature points.

[0129] Figure 8 is a block diagram for describing a method of generating a hybrid map according to an embodiment.

[0130] Reference Figure 8 , a hybrid map can be generated based on a deghosting map and a well-exposed map.

[0131] The hybrid map according to an embodiment may include weight values W1, W2, and W3 between 0 and 1, which can be applied to each pixel of the image. According to an embodiment, weight values W1, W2, and W3 can be generated for each pixel for images 1, 2, and 3. Here, the weight values W1, W2, and W3 can be generated such that the sum of the weight values W1, W2, and W3 is 1.

[0132] In addition, according to an embodiment, the deghosting map and the well-exposed map may include weight values applicable to each pixel of the image, such as the blending map. For example, weight values WM1, WM2, and WM3 to be applied to Images 1, 2, and 3 for each pixel may be generated for the deghosting map. Similarly, weight values WE1, WE2, and WE3 to be applied to Images 1, 2, and 3 for each pixel may be generated for the well-exposed map.

[0133] Images according to an embodiment may be generated via histogram matching 810 and 820, motion analysis 830 and 860, and image difference analysis 840 and 850.

[0134] During histogram matching 810 and 820 according to an embodiment, the electronic device 1000 may match the histograms between a reference image (Image 2) and non-reference images (Images 1 and 3). According to the histogram matching according to the embodiment, when adjusting the color distributions of the images to match each other, the brightness and color sensations between the two images may be matched.

[0135] According to an embodiment, after histogram matching of two images, motion analysis 830 and 860 and image difference analysis 840 and 850 may be performed.

[0136] The degree of movement of an object between two images may be determined according to motion analysis 830 and 860 according to an embodiment. In addition, during image difference analysis 840 and 850, the blending ratio of the reference image (Image 2) and the non-reference images (Images 1 and 3) may be determined based on the degree of movement.

[0137] Therefore, according to an embodiment, the pixel values of the images are blended according to the well-exposed map, and according to the well-exposed map, the blending degree of the non-reference images may be reduced in partial regions where movement has occurred.

[0138] According to an embodiment, motion analysis 830 and 860 may be performed according to the tiles of the image. Therefore, a blending ratio different from that of a region without movement may be determined for a partial region with movement.

[0139] According to an embodiment, corresponding regions between two images (where the degree of movement is determined) may be identified according to the transformation matrix obtained through feature point matching 110.

[0140] The degree of movement according to an embodiment may be determined as detecting the movement of an object between two images. For example, the degree of movement may be determined according to the SAD technique for determining the movement of a corresponding block determined via block search. However, the embodiment is not limited thereto, and the degree of movement according to the embodiment may be determined according to various methods for movement detection between two images.

[0141] The mixing ratio determined based on the image difference analysis 840 and 850 according to the embodiment can be determined inversely proportional to the degree of movement. For example, in a region with a high degree of movement between two images, the mixing ratio of the non-reference image (image 1 or 3) can be determined to be low.

[0142] Therefore, according to the embodiment, the mixing ratio of the non-reference image (image 1 or 3) can be determined to have a low value in the region with a high degree of movement from the image regions where differences have occurred between the two images. On the other hand, the mixing ratio of the non-reference image (image 1 or 3) can be determined to have a high value in the region with a low degree of movement but a high exposure difference or noise difference from the image regions where differences have occurred between the two images.

[0143] According to the embodiment, in order to prevent the degree of movement of the texture region of the image from being determined as a high value due to the exposure difference, although there is no movement, the degree of movement can be determined after applying a smoothing filter to the region.

[0144] According to the embodiment, based on the case where the mixing ratio WM2 of the reference image is 1, according to the degree of movement, for each pixel, the mixing ratios WM1 and WM3 of the non-reference images determined via the image difference analysis 840 and 850 can be determined. According to the embodiment, a mixing map including the mixing ratios determined for each pixel of the non-reference image can be generated.

[0145] During the exposure analysis 870 according to the embodiment, the weight values WE1, WE2, and WE3 based on the intensity values of images 1 to 3 can be determined for each pixel. The intensity value of each pixel according to the embodiment can be determined as a value of 0 or 1. For example, the intensity of a pixel can be determined according to the luminance value of the pixel value. For example, when the luminance value is close to 0, the intensity of the pixel can be determined as a value close to 0. The embodiment is not limited thereto, and various values indicating the intensity of the pixel value can be determined according to various methods.

[0146] According to the embodiment, among the corresponding pixels between the images, pixels with intermediate intensity can be determined to include details. Therefore, the weight value according to the embodiment can be determined as a high value according to whether the intensity value of the pixel is close to 0.5.

[0147] In addition, according to the embodiment, the weight value can be determined according to the position of the intensity value of each pixel on the Gaussian curve. For example, the x-axis of the center of the Gaussian curve can be determined as the intensity value 0.5, and its y-axis can be determined as the weight value 1, and when the intensity values of the pixels are arranged on the Gaussian curve, the weight values to be applied to each pixel can be determined.

[0148] According to an embodiment, corresponding pixels between a plurality of images for determining weight values can be identified according to a transformation matrix obtained from feature point matching 110. According to an embodiment, when determining weight values to be applied to pixels based on intensity values of pixels of a plurality of images identified according to the transformation matrix, a good exposure map can be generated.

[0149] According to an embodiment, weight values W1, W2, and W3 of a good exposure map can be determined such that the sum of the weight values determined for one pixel is 1 (W1 + W2 + W3 = 1).

[0150] During multiplexing 880 according to an embodiment, a hybrid map can be generated based on a deghosting map and a well-exposed map.

[0151] According to an embodiment, multiplexing 880 can be performed on WM1, WM2, and WM3 and WE1, WE2, and WE3, and thus a hybrid map including W1, W2, and W3 for each pixel can be generated. For example, W1 can be obtained based on a value obtained by simple multiplication via WM1 and WE1. Similarly, W2 and W3 can be obtained based on a value obtained by simple multiplication via WM2 and WE2 and a value obtained by simple multiplication via WM3 and WE3. According to an embodiment, the values of W1, W2, and W3 can be adjusted such that the sum of the obtained W1, W2, and W3 is 1.

[0152] According to an embodiment, mixing 130 can be performed on a region of a non-reference image determined to have moved after replacing a region with pixel values of a reference image in use. For example, pixel values of a region of a non-reference image with a movement degree equal to or greater than a reference value can be replaced with pixel values of a reference image corresponding to the region.

[0153] Figure 9 is a diagram showing an example of exposure difference according to an embodiment.

[0154] Reference Figure 9 , exposure differences can be identified from images 910 and 920 with different exposures. For example, due to the exposure difference, some leaves and branches shown in region 911 may not be shown in region 921. In addition, due to the exposure difference, some leaves and branches shown in region 912 may not be shown in region 922.

[0155] According to an embodiment, weight values of pixel values in regions 911 and 912 can be determined to be higher values according to the good exposure map. In addition, since no movement occurs in regions 911 and 912, the weight values of the good exposure map may not be adjusted by the deghosting map.

[0156] Figure 10 is a diagram showing an example of movement difference according to an embodiment.

[0157] Reference Figure 10 , reference numerals 1020 and 1030 denote image maps of non-reference images among a plurality of images 1010 with different exposures. Different from the ghost removal map 1020, the ghost removal map 1030 is processed such that image differences caused by exposure differences are not recognized as movement. In the ghost removal map, regions close to 0 may appear darker.

[0158] According to the ghost removal maps 1020 and 1030 according to an embodiment, as Figure 10 shown, a low mixing ratio can be determined for the region 1021 indicated by the arrow where movement has occurred, because the degree of movement is determined to be high.

[0159] Figure 11 FIG. is a diagram showing an example of mixing images 1111 to 1113 with different exposures according to the image pyramid method according to an embodiment.

[0160] A plurality of images 1111 to 1113 according to an embodiment can be mixed according to the image pyramid method based on weight values determined according to a mixing map. Among the plurality of images 1111 to 1113 mixed according to an embodiment, a non-reference image can be an image in which pixel values of some regions of the non-reference image have been replaced with pixel values of a reference image according to the degree of movement.

[0161] According to the image pyramid method according to an embodiment, when the mixing map of each image is downsampled to different resolutions for each layer, a Gaussian pyramid can be generated. In addition, a Laplacian pyramid can be generated for each image.

[0162] The Laplacian pyramid according to an embodiment can be generated from the Gaussian pyramid. For example, the Laplacian pyramid image at level n can be determined as the difference image between the Gaussian pyramid image at level n and the Gaussian pyramid image at level n-1.

[0163] According to an embodiment, images generated according to the Laplacian pyramid can be mixed in each layer as weight values for applying the mixing map generated according to the Gaussian pyramid. After adjusting the resolutions of the images mixed in each layer, the images mixed in each layer can be sub-mixed again, and thus an HDR image can be generated.

[0164] According to the pyramid method of mixing a plurality of images according to a mixing map, according to an embodiment, noise such as contours or seams can be reduced, but halo artifacts may be generated or details of the non-reference images to be mixed may be lost.

[0165] For example, when the number of layers is low, halo artifacts may be generated in the HDR image, and when the number of layers is high, details included in the images to be mixed may not be sufficiently included in the HDR image and may be lost.

[0166] In Figure 11 it, reference numeral 1120 indicates an example of an HDR image mixed according to an 11-level pyramid, and reference numeral 1140 indicates an example of an HDR image mixed according to an 8-level pyramid. Reference numerals 1130 and 1150 respectively indicate an image before mixing according to the pyramid method, and an example of an image including details lost in reference numerals 1120 and 1140.

[0167] Referring to reference numeral 1120, when the number of layers is high, image details may be lost due to the generation of a saturated region in region 1121. In addition, details 1131 and 1132 in reference numeral 1130 may not be reflected in reference numeral 1120.

[0168] Referring to reference numeral 1140, when the number of layers is low, halo artifacts may be generated in regions 1141 and 1142.

[0169] Figure 12 is a block diagram for describing a method of mixing multiple images based on a mixed map according to an embodiment.

[0170] Referring to Figure 12 , the images 1' and 3' to which mixing is performed may be images in which pixel values of images 1 and 3 are replaced with pixel values of a reference image (image 2) according to the degree of movement. However, the embodiment is not limited thereto, and the images 1' and 3' may be images in which pixel values are not replaced according to the degree of movement.

[0171] According to an embodiment, multiple images are not mixed in a single secondary mixing, but may be gradually mixed according to a primary mixing 1201 and a secondary mixing 1203.

[0172] During the primary mixing 1201 according to an embodiment, a reference image (image 2) and an image 1' as one of the non-reference images may be mixed. W1 determined according to the mixed map may be applied to the image 1', and 1 - W1 may be applied to the reference image.

[0173] The image 1' of the primary mixing according to an embodiment may be a short-exposure image, including details of the saturated region of the reference image by including a dark region compared with the reference image. In addition, the image 3' of the secondary mixing according to an embodiment may be a long-exposure image, including details of the dark region of the reference image by including a bright region compared with the reference image.

[0174] According to an embodiment, in each step, blending may be performed according to a pyramid level determined based on characteristics regarding details of a non-reference image to be blended with a reference image. The pyramid level according to the embodiment may be determined to be a value such that, based on characteristics regarding details of the non-reference image, halation artifacts are reduced and details of the non-reference image are reflected in the HDR image as much as possible.

[0175] For example, according to the primary blend 1201, when blending a short exposure image, blending may be performed according to a pyramid of a level (e.g., a relatively high level) in which a saturated region and halation artifacts are reduced and details of the short exposure image can be sufficiently reflected. In addition, blending may be performed after adjusting at least one of W1 or 1-W1, which are weight values of a blending map, such that the saturated region and halation artifacts are reduced according to the pyramid of the level, and details of the short exposure image are sufficiently reflected.

[0176] In addition, according to the secondary blend 1203, when blending a long exposure image, blending may be performed according to a pyramid of a level (e.g., a relatively low level, alpha blending) in which details of the long exposure image are sufficiently reflected.

[0177] Therefore, according to the embodiment, blending is performed according to a pyramid level appropriately determined based on an image to be blended, and thus an HDR image in which noise is reduced and details of the non-reference image are reflected as much as possible can be generated.

[0178] During the primary blend 1201 according to the embodiment, images 1' and 2, which are blending maps of images, may be blended based on W1 and 1-W1 according to the above pyramid method. For example, the images of the primary blend may be generated such that Gaussian pyramids for W1 and 1-W1 are respectively applied to Laplacian pyramids of images 1' and 2. The embodiment is not limited thereto, and during the primary blend 1201, images 1' and 2 may be blended based on the blending map according to various methods.

[0179] During the histogram matching 1202 according to the embodiment, the histogram of image 3' may be adjusted such that the histogram of the image of the primary blend and image 3' match. During the histogram matching 1202 according to the embodiment, when adjusting the histogram of image 3', which is a short exposure image, the number of pyramid levels required during the secondary blend 1203 may be reduced as the intensity of image 3' increases.

[0180] During the secondary blend 1203 according to the embodiment, the image of the primary blend and image 3' may be blended. W3 determined according to the blending map may be applied to image 3', and 1-W3 may be applied to the image of the primary blend.

[0181] During the secondary blending 1203 according to an embodiment, based on the pyramid method described above, the main-blended image and the image 3' can be blended based on 1-W3 and W3, which are blend maps of the image. For example, the image of the secondary blend can be generated as the Laplacian pyramids of the Gaussian pyramids for 1-W3 and W3 being respectively applied to the main-blended image and the image 3'. The embodiment is not limited thereto, and during the secondary blending 1203, the main-blended image and the image 3' can be blended based on the blend map according to various methods.

[0182] According to an embodiment, an HDR image can be obtained from the image of the secondary blend. For example, an HDR image can be obtained by further performing operations for image optimization (such as tone mapping, noise filtering, and edge enhancement) on the image of the secondary blend. The embodiment is not limited thereto, and an HDR image can be finally obtained by further performing various types of operations for optimizing the image on the image of the secondary blend.

[0183] Figure 13 is a diagram showing an example of a blended image according to an embodiment.

[0184] Figure 13 Reference numeral 1301 in the figure represents an example of a blend map in an 8-level pyramid for a short-exposure image (image 3'), and reference numeral 1302 represents an example of a blend map in which the weight value is controlled in an 8-level pyramid for a short-exposure image (image 3'). According to an embodiment, as in reference numeral 1302, blending can be performed according to the blend map for controlling the weight value, so that the saturated region and the halo artifact are reduced, and the details of the short-exposure image are fully reflected. According to an embodiment, the weight value of the blend map can be controlled during the main blending 1201 at a high pyramid level.

[0185] Figure 13 Reference numerals 1303 and 1305 in the figure respectively represent examples of the main-blended image and the secondary-blended image, and reference numeral 1304 represents an example of a long-exposure image including details of a dark region for comparison with the blended image.

[0186] In the main-blended image of reference numeral 1303, the saturated region and the halo artifact can be reduced by blending with the short-exposure image. In addition, the secondary-blended image of reference numeral 1305 can also include details of the dark region included in the reference image by blending with the long-exposure image.

[0187] According to an embodiment, while reducing the ghost artifact and the halo artifact included in the HDR image, an HDR image in which the details included in each image are fully reflected can be generated.

[0188] The embodiments can also be implemented in the form of a recording medium, which includes instructions executable by a computer, such as program modules executable by a computer. The computer-readable recording medium can be any available medium accessible by a computer, and examples thereof include all volatile and non-volatile media, as well as separable and inseparable media. In addition, examples of the computer-readable recording medium can include computer storage media and communication media. Examples of computer storage media include all volatile and non-volatile media, as well as separable and inseparable media implemented by any method or technology for storing information such as computer-readable commands, data structures, program modules, and other data. Communication media generally include computer-readable instructions, data structures, or program modules, and include any information transmission medium.

[0189] In addition, in the specification, the term "unit" can be a hardware component such as a processor or a circuit and / or a software component executed by a hardware component such as a processor.

[0190] The above description of the present invention is provided for illustration purposes. Those of ordinary skill in the art will understand that various changes can be easily made in form and detail without departing from the basic features and scope of the present invention defined by the following claims. Therefore, the above embodiments are examples in all aspects and are not limited. For example, each component described as a single type can be implemented in a distributed manner, and similarly, components described as distributed can be implemented in a combined form.

[0191] The scope of the present invention is defined by the appended claims rather than the detailed description, and all changes or modifications within the scope of the appended claims and their equivalents will be construed as being included within the scope of the present invention.

Claims

1. A method for generating a high dynamic range (HDR) image, the method comprising: Obtaining a plurality of images captured with different exposure levels; Generating at least one transformation matrix indicating corresponding regions between the plurality of images; Based on the transformation matrix, determining the degree of movement of each corresponding region between the plurality of images; Based on the determined degree of movement, generating a blending map indicating the blending ratio of the plurality of images at each pixel; And Based on the blending map, generating the HDR image by blending the plurality of images, Wherein, the generation of the at least one transformation matrix includes: Detecting global feature points corresponding to features included in the plurality of images from the plurality of images, and matching the global feature points between the plurality of images; Based on the matching result of the global feature points, searching for corresponding blocks between the plurality of images; Detecting the center points of the found blocks as local feature points, and matching the corresponding local feature points between the plurality of images; and Based on the matching results of the global feature points and the local feature points, generating the at least one transformation matrix.

2. The method according to claim 1, wherein When generating the transformation matrix, excluding local feature points whose matching results are inconsistent with the matching results of adjacent local feature points from the detected local feature points.

3. The method according to claim 1, wherein The blending map is generated based on a ghost removal map determined based on the degree of movement and a well-exposed map determined based on the intensity of pixel values.

4. The method according to claim 1, wherein Setting a reference image and a plurality of non-reference images from the plurality of images, and The generation of the HDR image includes generating the HDR image by gradually blending the plurality of non-reference images with respect to the reference image based on the blending map.

5. The method according to claim 4, wherein, The blending is performed based on features related to the details of the non-reference image being blended with the reference image.

6. The method according to claim 5, wherein, In the current step, based on features regarding the details of the non-reference image being blended, the blending is performed by determining the number of pyramid levels and decomposing the reference image and the non-reference image into different resolutions through the determined number of pyramid levels, where the pyramid level indicates the number of the reference image and the non-reference image decomposed into different resolutions.

7. An electronic device for generating a high dynamic range (HDR) image, the electronic device comprising: A camera configured to obtain a plurality of images captured with different exposure levels; At least one processor configured to: Generate at least one transformation matrix indicating corresponding regions between the plurality of images; Based on the transformation matrix, determine the degree of movement of each corresponding region between the plurality of images; Based on the determined degree of movement, generate a blending map indicating the blending ratio of the plurality of images at each pixel; And Based on the blending map, generate the HDR image by blending the plurality of images; And A memory for storing the generated HDR image, Wherein, the at least one processor is further configured to: Detect global feature points corresponding to features included in the plurality of images from the plurality of images, and match the global feature points between the plurality of images; Search for corresponding blocks among the multiple images based on the matching results of the global feature points; Detect the center points of the found blocks as local feature points, and match the corresponding local feature points among the multiple images; and Generate the at least one transformation matrix based on the matching results of the global feature points and the local feature points.

8. The electronic device according to claim 7, wherein, When the transformation matrix is generated, exclude local feature points whose matching results are inconsistent with the matching results of adjacent local feature points from the detected local feature points.

9. The electronic device according to claim 7, wherein, The hybrid map is generated based on a deghosting map determined based on the degree of movement and a good exposure map determined based on the intensity of pixel values.

10. The electronic device according to claim 7, wherein, Set a reference image and multiple non-reference images from the multiple images, and The at least one processor is further configured to generate the HDR image by gradually blending the multiple non-reference images with respect to the reference image based on the hybrid map.

11. The electronic device according to claim 10, wherein, The blending is performed based on features related to the details of the non-reference images blended with the reference image.

12. The electronic device according to claim 11, wherein, In the current step, the blending is performed by determining the number of pyramid levels and decomposing the reference image and the non-reference images into different resolutions by the determined number of pyramid levels, where the pyramid levels indicate the number of the reference image and the non-reference images decomposed into different resolutions, based on features regarding the details of the non-reference images to be blended.

13. A computer-readable recording medium having recorded thereon a program for implementing the method according to any one of claims 1 to 6.

Images