Electronic device, image processing method thereof, and computer readable recording medium

Abstract

The present disclosure relates to an artificial intelligence (AI) system and its application using a machine learning algorithm including deep learning, etc. In particular, the electronic device of the present disclosure includes: a memory including at least one command; and a processor connected to the memory to control the electronic device, wherein the processor performs the following operations by executing at least one command: acquiring an image, acquiring a noise correction map for correcting the noise of the image based on the configuration information of the camera that captured the image or the brightness information of the image, and removing the noise of the image by the noise correction map. In particular, at least a part of the image processing method can use an artificial intelligence model acquired by learning according to at least one of a machine learning algorithm, a neural network algorithm, and a deep learning algorithm.

Description

Technical Field

[0001] The present disclosure relates to an electronic device, an image processing method thereof, and a computer-readable recording medium, and more particularly, to an electronic device that removes noise using brightness information of an image, an image processing method thereof, and a computer-readable recording medium.

[0002] The present disclosure also relates to an artificial intelligence (AI) system and its applications that use machine learning algorithms to simulate the recognition function, decision-making function, etc. of the human brain. Background Art

[0003] Recently, artificial intelligence systems that achieve human-level intelligence have been used in various fields. An artificial intelligence system is a system in which a machine performs learning and decision-making, and unlike the existing rule-based intelligent system, the machine becomes intelligent by itself. With the increased use of artificial intelligence systems, the recognition rate is improved, and the user's interests can be understood more accurately. Therefore, the existing rule-based intelligent system is gradually replaced by an artificial intelligence system based on deep learning.

[0004] Artificial intelligence technology includes machine learning (e.g., deep learning) and component technologies that use machine learning.

[0005] Machine learning is an algorithmic technology that classifies / learns the features of input data by itself, and meta-technology is a technology that uses machine learning algorithms such as deep learning to simulate the recognition, decision-making and other functions of the human brain, and includes technical fields such as language understanding, visual understanding, reasoning / prediction, knowledge representation, and motion control.

[0006] Various fields where artificial intelligence technology is applied are as follows. Language understanding is the technology of recognizing and applying / processing human language / characters, and includes natural language processing, machine translation, dialogue systems, question and answer, speech recognition / synthesis, etc. Visual understanding is the technology of recognizing and processing things like human vision, and includes object recognition, object tracking, image search, human recognition, scene understanding, spatial understanding, image improvement, etc. Inference / prediction is the technology of performing logical inference and prediction through decision information, and includes knowledge / probability-based inference, optimization prediction, preference-based planning, recommendation, etc. Knowledge representation is the technology of automating human experience information and processing it as knowledge data, and includes knowledge construction (data creation / classification), knowledge management (data utilization), etc. Motion control is the technology of controlling the autonomous driving of vehicles and the movement of robots, and includes motion control (navigation, collision, driving), manipulation control (behavior control), etc.

[0007] Meanwhile, recently, due to the popularity of digital cameras, the restrictions on the time and place of generating still images or moving images have been reduced. In addition, with the development of image technology, it is possible for ordinary people who are not professionals to easily obtain high-definition images.

[0008] However, there is a problem in that an image generated in a dark surrounding environment such as at night or in a closed space includes a large amount of noise, so that the image quality is degraded.

[0009] In order to solve this problem, conventionally, the noise of the image is measured and filtering is performed for each intensity of the noise to remove the noise. In this case, the noise is measured by analyzing the frequency of the image, and therefore, the local characteristics of the image are not considered. For example, there are many cases where it is unclear whether any part in the image is an edge or noise present in the image. Therefore, there has been a problem that the image quality is reduced due to the presence of a blurred area and the loss of the texture of the image due to filtering the area that is not noise in the compensated image.

[0010] In addition, in the related art of using an artificial intelligence model to remove noise, a separate model is used for each intensity of the measured noise. In this case, each of the multiple models prepared for each intensity of the noise should be trained, so there is a problem that a lot of input data and memory are used for training. Summary of the invention

[0011] Technical issues

[0012] The present disclosure provides an electronic device for removing noise using brightness information of an image, an image processing method thereof, and a computer-readable recording medium.

[0013] Technical Solution

[0014] According to an embodiment of the present disclosure, an electronic device includes: a memory including at least one command; and a processor connected to the memory and configured to control the electronic device, wherein the processor is configured, when executing at least one command, to: acquire an image, acquire a noise compensation map for compensating for noise in the image based on setting information of a camera that captures the image or brightness information of the image, and remove noise in the image through the noise compensation map.

[0015] In this case, the processor may be configured to acquire the noise compensation map based on brightness information of the image so that the noise removal intensity becomes high in a region with low brightness.

[0016] In this case, the brightness information of the image may be contrast information of each pixel of the image, and the processor may be configured to acquire a noise compensation map such that the noise removal intensity varies for each pixel.

[0017] Meanwhile, the processor may be configured to divide the image into a plurality of regions for each brightness, and acquire a noise compensation map such that the noise removal intensity varies for each divided region.

[0018] Meanwhile, the setting information of the camera may include at least one of lens information of the camera or ISO information of the camera.

[0019] In this case, the processor may be configured to acquire a noise compensation map based on lens information of the camera so that the noise removal strength varies according to the degree of vignetting compensation.

[0020] At the same time, the processor may be configured to obtain a noise compensation map based on ISO information, such that the higher the ISO value, the higher the noise removal strength.

[0021] At the same time, the processor can be configured to input the noise compensation map and the image into the learned artificial intelligence model to obtain a compensated image with noise removed.

[0022] According to another embodiment of the present disclosure, an image processing method of an electronic device includes: acquiring an image; acquiring a noise compensation map for compensating for noise in the image based on setting information of a camera that captures the image or brightness information of the image; and removing noise from the image through the noise compensation map.

[0023] In this case, in the process of acquiring the noise compensation map, the noise compensation map may be acquired based on brightness information of the image so that the noise removal intensity becomes high in a region with low brightness.

[0024] In this case, the brightness information of the image may be contrast information of each pixel of the image, and when acquiring the noise compensation map, the noise compensation map may be acquired so that the noise removal intensity varies for each pixel.

[0025] In this case, in the process of acquiring the noise compensation map, the image may be divided into a plurality of regions for each brightness and the noise compensation map may be acquired such that the noise removal intensity varies for each divided region.

[0026] Meanwhile, the setting information of the camera may include at least one of lens information of the camera or ISO information of the camera.

[0027] In this case, when acquiring the noise compensation map, the noise compensation map may be acquired based on the lens information of the camera so that the noise removal intensity varies according to the vignetting compensation degree.

[0028] Meanwhile, when acquiring the noise compensation map, the noise compensation map may be acquired based on ISO information, such that the higher the ISO value, the higher the noise removal strength.

[0029] At the same time, when removing noise, the noise compensation map and the image can be input into the learned artificial intelligence model to obtain a compensated image with the noise removed.

[0030] According to another embodiment of the present disclosure, a computer-readable medium recording medium includes a program for executing an image processing method, wherein the image processing method includes: acquiring an image; acquiring a noise compensation map for compensating for noise in the image based on setting information of a camera that captures the image or brightness information of the image; and removing noise in the image through the noise compensation map. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a block diagram for describing a schematic configuration of an electronic device according to an embodiment of the present disclosure;

[0032] Figure 2 Is used to describe Figure 1 A block diagram of a detailed configuration of an electronic device;

[0033] Figures 3 to 5 is a diagram for describing a noise compensation process according to different embodiments of the present disclosure;

[0034] Figure 6 is a block diagram showing a configuration of an electronic device for learning and using an artificial intelligence model according to an embodiment of the present disclosure;

[0035] Figure 7a and Figure 7b is a block diagram showing a specific configuration of a learner and an acquirer according to an embodiment of the present disclosure;

[0036] Figure 8 is a view for explaining a process of acquiring a noise compensation map according to an embodiment of the present disclosure;

[0037] Fig. 9 is a view for describing an embodiment of a noise compensation map acquired by reflecting setting information of a camera;

[0038] Fig.10 is a flowchart for describing an image compensation method according to an embodiment of the present disclosure; and

[0039] Fig.11 is a view for comparing an existing image and an image compensated according to the present disclosure with each other. DETAILED DESCRIPTION

[0040] After schematically describing the terms used in the specification, the present disclosure will be described in detail.

[0041] Considering the functions in the present disclosure, currently widely used general terms are selected as terms used in the embodiments of the present disclosure, but may be changed according to the intention of those skilled in the art or the emergence of judicial precedents, new technologies, etc. In addition, in certain cases, there may be terms arbitrarily selected by the applicant. In this case, the meanings of these terms will be mentioned in detail in the corresponding description parts of the present disclosure. Therefore, the terms used in the present disclosure should be defined based on the meaning of the terms and the content throughout the present disclosure rather than the simple names of the terms.

[0042] Because the present disclosure can be variously modified and has several embodiments, specific embodiments of the present disclosure will be shown in the drawings and will be described in detail in the specific embodiments. However, it should be understood that the present disclosure is not limited to specific embodiments, but includes all modifications, equivalents and substitutions without departing from the scope and spirit of the present disclosure. When it is determined that a detailed description of known technologies related to the present disclosure may obscure the subject matter of the present disclosure, the detailed description will be omitted.

[0043] The terms "first", "second", etc. may be used to describe various components, but these components should not be construed as being limited by these terms. These terms are only used to distinguish one component from another.

[0044] Unless the context clearly indicates otherwise, the singular form is intended to include the plural form. It should be understood that the terms "comprising" or "formed by" used in the specification indicate the presence of the features, numbers, steps, operations, components, parts or combinations thereof mentioned in the specification, but do not exclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0045] In the embodiments of the present disclosure, a "module" or "unit" may perform at least one function or operation, and may be implemented by hardware or software, or may be implemented by a combination of hardware and software. In addition, multiple "modules" or multiple "units" may be integrated into at least one module, and may be implemented by at least one processor other than the "modules" or "units" that need to be implemented by specific hardware.

[0046] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present disclosure belongs can easily practice the present disclosure. However, the present disclosure can be modified in various different forms and is not limited to the embodiments described herein. In addition, in the accompanying drawings, parts not related to the description will be omitted to clearly describe the present disclosure, and similar reference numerals will be used to describe similar parts throughout the specification.

[0047] Hereinafter, the present disclosure will be described in more detail with reference to the accompanying drawings.

[0048] Figure 1 is a block diagram for describing a schematic configuration of an electronic device according to an embodiment of the present disclosure.

[0049] Reference Figure 1 According to an embodiment of the present disclosure, the electronic device 100 includes a memory 110 and a processor 120 .

[0050] The electronic device 100 may be a device capable of processing an image. For example, the electronic device 100 may be a desktop personal computer (PC), a laptop computer, a television (TV), a smart phone, a tablet PC, a server, a camera, a portable camera, etc. Alternatively, the electronic device 100 may be a system itself in which a cloud computing environment is configured. In addition, the electronic device 100 may be a device capable of artificial intelligence learning.

[0051] The artificial intelligence model learned in the present disclosure can be configured in consideration of the application field of the recognition model, the computer performance of the device, and the like. For example, an artificial intelligence model can be learned to obtain information about an object included in a moving image by inputting moving image data. The learned artificial intelligence model can be, for example, a model based on a neural network. The recognition model can be designed to simulate the structure of the human brain on a computer, and can include multiple network nodes that simulate neurons of a human neural network and have weights. Multiple network nodes can form a connection relationship between them to simulate the synaptic activity of neurons that send and receive signals through synapses. In this case, the artificial intelligence model can be a deep neural network (DNN), but this is only an example, and another artificial intelligence model can be used.

[0052] In addition, as described above, the electronic device 100 can use an artificial intelligence ontology to search for information related to an object. In this case, the artificial intelligence ontology is a dedicated program for providing services based on artificial intelligence (AI) (e.g., speech recognition services, secretarial services, translation services, search services, etc.), and can be executed by an existing general-purpose processor (e.g., a central processing unit (CPU)) or a separate AI-specific processor (e.g., a graphics processing unit (GPU)), etc.

[0053] The memory 110 may be implemented by memories of various formats, such as a hard disk drive (HDD), a solid-state drive (SSD), a dynamic random access memory (DRAM), a static random access memory (SRAM), a ferroelectric random access memory (FRAM), a flash memory, etc.

[0054] Specifically, the memory 110 may store various programs and data required for the operation of the electronic device 100. For example, at least one command required for the operation of the electronic device 100 may be stored in the memory 110.

[0055] In addition, the artificial intelligence model may be stored in the memory 110. Here, the artificial intelligence model may be a learned model. In addition, the artificial intelligence model may remove noise from an input image. Specifically, the artificial intelligence model may receive an image and a noise compensation map acquired by the processor 120, and remove noise from the acquired image. Here, the image input to the artificial intelligence model may be a noisy image acquired based on a high-definition original image, and the parameters of the artificial intelligence model may be trained so that the image from which the noise is removed is consistent with the high-definition original image.

[0056] The processor 120 may generally control the operation of the electronic device 100 by executing at least one command stored in the memory 110 .

[0057] According to an embodiment, the processor 120 may be implemented by a digital signal processor (DSP), a microprocessor, or a time controller (TCON). However, the processor 120 is not limited thereto, but may include one or more of a central processing unit (CPU), a microcontroller unit (MCU), a microprocessing unit (MPU), a controller, an application processor (AP), a communication processor (CP), and an ARM processor, or may be defined by these terms. In addition, the processor 140 may be implemented by a system on chip (SoC) or a large-scale integration (LSI) embedded with a processing algorithm, or may be implemented in the form of a field programmable gate array (FPGA).

[0058] The processor 120 may acquire an image and may remove noise of the acquired image. In this case, the image may be acquired by a camera (not shown) included in the electronic device 100, or may be received from an external device.

[0059] Specifically, the processor 120 may acquire a noise compensation map for compensating for the noise of the image, and may use the acquired noise compensation map to remove the noise of the image. In this case, the noise compensation map may be acquired based on at least one of the setting information of the camera that captured the acquired image or the brightness information of the acquired image.

[0060] Specifically, the processor 120 can acquire a noise compensation map based on the brightness information of the acquired image so that the noise removal intensity becomes high in the area with low brightness. Even if the conditions other than the brightness of the image are the same, the lower the brightness of the image, the more noise is generated. Therefore, a noise compensation map is generated so that the lower the brightness of the image, the higher the noise removal intensity. Here, the noise removal intensity may refer to the intensity of applying the noise removal effect.

[0061] Specifically, the processor 120 may acquire brightness information of the image. In this case, the brightness information of the image may refer to the contrast information of the image. The processor 120 may acquire a grayscale image including contrast information of the acquired image, and acquire a noise compensation image using the grayscale image. In this case, the contrast information of the image is different for each image, and therefore, the processor 120 may generate a corresponding noise compensation image for each acquired image.

[0062] As an implementation, the processor 120 may acquire contrast information of each pixel of the image, and acquire a noise compensation map so that the noise removal strength varies for each pixel.

[0063] As another embodiment, the processor 120 may divide the image into a plurality of regions, and acquire a noise compensation map so that the noise removal strength varies for each divided region. Specifically, the processor 120 may divide the image into a plurality of regions for each brightness. For example, the processor 120 may detect an edge within the image, and divide the image into a plurality of regions based on the detected edge. Here, an edge may refer to a boundary where the difference in contrast in the image is a predetermined value or greater.

[0064] At the same time, the processor 120 may use the setting information of the camera to obtain the noise compensation map. Here, the camera is a device that captures the acquired image, and the setting information of the camera may include various items related to the brightness information of the acquired image. For example, the setting information of the camera may include at least one of the lens information of the camera or the ISO information of the camera.

[0065] Specifically, the processor 120 may acquire a noise compensation map based on the lens information of the camera so that the noise removal intensity varies according to the degree of vignetting compensation. Here, vignetting may refer to a phenomenon in which the corners or the outside of the captured image become dark or covered with black due to a reduction in the amount of light in the peripheral portion of the lens. Vignetting is a problem of the lens itself, and the range and degree of occurrence of vignetting may vary for each lens used in the camera. At the same time, unless the lens of the camera is replaced, the noise compensation map based on the lens information may be the same. Reference will be made to Figure 8 and Fig. 9 Embodiments of compensating for vignetting are described in detail.

[0066] In addition, the processor 120 may acquire a noise compensation map based on ISO information of the camera. Here, ISO refers to the sensitivity of the camera to light, and because the camera recognizes weak light well as the ISO value of the camera becomes larger, the camera is mainly used to capture images in dark places. When the ISO value is large, the image capturing time is shortened to obtain a clear image without shaking, but there is a problem that the image becomes rough due to increased noise.

[0067] This ISO value may be specified by the user, but may also be set automatically based on sensed ambient brightness.

[0068] That is, when the ambient brightness is low when taking an image, the noise in the taken image increases, and therefore a noise compensation map with a high noise removal intensity is required, and the processor 120 can obtain the noise compensation map based on ISO information reflecting the brightness information, so that the higher the ISO value, the higher the noise removal intensity.

[0069] It has been described above that each of the image contrast information, the camera lens information, and the camera ISO information is used to obtain a noise compensation map, but in actual implementation, such as Figure 3 and Figure 8 As shown, a noise compensation map reflecting both the map acquired based on the camera's setting information and the grayscale map of the image can be acquired. In addition, in addition to the camera's setting information and the image's contrast information, various data such as the value sensed by the illuminance sensor can also be reflected.

[0070] At the same time, the processor 120 may output an image from which noise is removed using the artificial intelligence model stored in the memory 110. Specifically, the processor 120 may input the acquired image and the acquired noise compensation map into the artificial intelligence model to acquire a compensated image from which noise has been removed.

[0071] At the same time, the artificial intelligence model can be a pre-learned model. Specifically, the acquired image can be a noisy image generated from a high-definition original image. In this case, the processor 120 can generate a noise compensation map for the noise image, input the noise image and the noise compensation map into the artificial intelligence model, and output the compensated image. In addition, the processor 120 can compare the output compensated image and the high-definition original image with each other to train the parameters of the artificial intelligence model. This training operation can be performed during or before the manufacturing process of the electronic device 100, and the manufactured electronic device 100 may include an artificial intelligence model using the trained parameters. The following will refer to Figure 4 and Figure 5 Various implementations including artificial intelligence models are described in detail.

[0072] As described above, according to the present disclosure, the noise of an image can be removed without directly measuring the noise of the image, and the noise can be removed more effectively without losing texture by reflecting both the global characteristics and local characteristics of the image.

[0073] Figure 2 Is used to describe Figure 1 A block diagram of a detailed configuration of an electronic device.

[0074] Reference Figure 2 , the electronic device 100 may include a memory 110, a processor 120, a communicator 130, a display 140, a button 150, a video processor 160, and a camera 170. Here, the memory 110 and the processor 120 are Figure 1 The memory 110 and the processor 120 shown in FIG. 1 are the same, and thus repeated descriptions thereof will be omitted.

[0075] First, the processor 120 may include a random access memory (RAM) 121, a read only memory (ROM) 122, a central processing unit (CPU) 123, a graphics processing unit (GPU) 124, and a bus 125. The RAM 121, the ROM 122, the CPU 123, the GPU 124, etc. may be connected to each other through the bus 125.

[0076] The CPU 123 accesses the memory 110 to perform booting using an operating system (O / S) stored in the memory 110. In addition, the CPU 123 uses various programs, contents, data, etc. stored in the memory 110 to perform various operations.

[0077] An instruction set for booting the system and the like are stored in the ROM 122. When a power-on command is input to supply power to the CPU 123, the CPU 123 copies an operating system (O / S) stored in the memory 110 to the RAM 121 according to the instructions stored in the ROM 122, and executes the O / S to boot the system. When the booting is completed, the CPU 123 copies various programs stored in the memory 110 to the RAM 121, and executes the programs copied to the RAM 121 to perform various operations.

[0078] When the booting of the electronic device 100 is completed, the GPU 124 displays a user interface (UI) on the display 140. In detail, the GPU 124 can use a calculator (not shown) and a renderer (not shown) to render a screen including various objects such as icons, images, texts, etc. The calculator calculates the attribute values ​​of each object according to the layout of the screen, such as the coordinate values, form, size, color, etc. of each object to be displayed. The renderer renders screens including various layouts of objects based on the attribute values ​​calculated in the calculator. The screen (or user interface window) rendered in the renderer is provided to the display 140 and is displayed on the main display area and the sub-display area, respectively.

[0079] The communicator 130 is a component that performs communication with various types of external devices according to various types of communication methods. Specifically, the communicator 130 can receive an image to be processed from an external device. In addition, the communicator 130 can receive setting information of a camera that captures an image. Specifically, the communicator 130 can receive at least one of lens information or ISO information of the camera. At the same time, the communicator 130 can receive a value sensed by an external illumination sensor. In addition, the communicator 130 can send an image from which noise has been removed to an external device.

[0080] Specifically, the communicator 130 may receive an image from an external device via a wired manner (e.g., an antenna, a cable, or a port) or may receive an image via a wireless manner (e.g., wireless fidelity (Wi-Fi) or Bluetooth). Meanwhile, in actual implementation, the electronic device 100 may receive an image selected by a user from a plurality of images stored in the memory 110 included in the electronic device 100, and process the received image.

[0081] In the case where the electronic device 100 is capable of wireless communication, the communicator 130 may include a Wi-Fi chip, a Bluetooth chip, a wireless communication chip, and a near field communication (NFC) chip. Specifically, the Wi-Fi chip and the Bluetooth chip perform communication in a Wi-Fi manner and a Bluetooth manner, respectively. In the case of using a Wi-Fi chip or a Bluetooth chip, various connection information such as a service set identifier (SSID), a session key, etc. are first sent and received, the communication is connected using the connection information, and various information can be sent and received next. A wireless communication chip refers to a chip that performs communication according to various communication protocols such as the Institute of Electrical and Electronics Engineers (IEEE), Zigbee, the third generation (3G), the third generation partnership project (3GPP), and long term evolution (LTE). An NFC chip refers to a chip that operates in an NFC manner using a band of 13.56MHz in various radio frequency identification (RFID) bands such as 135kHz, 13.56MHz, 433MHz, 860 to 960MHz, 2.45GHz, etc.

[0082] The display 140 may display an image from which noise has been removed by the processor 120. In addition, the image before noise removal may be displayed on the display 140 according to the design of software such as an application. The display 140 may be implemented by various types of displays, such as a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display panel (PDP), etc. The display 140 may include a driving circuit, a backlight unit, etc. that may be implemented in a form such as a-si thin film transistor (TFT), low temperature polycrystalline silicon (LTPS), TFT, an organic TFT (OTFT), etc. In addition, the display 140 may be implemented by a flexible display.

[0083] Meanwhile, according to various embodiments, the display 140 may not be included in the electronic device 100 .

[0084] The button 150 may be various types of buttons formed in any area (eg, front surface portion, side surface portion, rear surface portion, etc.) of the body surface of the electronic device 100, such as a mechanical button, a touch pad, a wheel, etc.

[0085] The video processor 160 is a component for processing video data included in the content received through the communicator 130 or the content stored in the memory 110. In the video processor 160, various types of image processing may be performed on the video data, such as decoding, scaling, noise filtering, frame rate conversion, resolution conversion, etc. In this case, the video data may be video data from which noise has been removed for each frame by the processor 120.

[0086] The camera 170 is a component for taking still images or moving images according to the control of the user. Specifically, the camera 170 may include a lens, an image sensor, etc. In addition, the camera 170 may be implemented by a plurality of cameras, such as a front camera and a rear camera. In addition, according to different embodiments, the camera 170 may not be included in the electronic device 100.

[0087] Despite Figure 2 Although not shown in the figure, the electronic device 100 may further include a universal serial bus (USB) port for connecting a USB connector, an earphone, a mouse, various external input ports for connecting to various external terminals such as a local area network (LAN), a digital multimedia broadcasting (DMB) chip for receiving and processing DMB signals, an audio processor, a microphone, an audio output, various sensors, etc., according to an embodiment.

[0088] Figures 3 to 5 is a view for describing the noise compensation process according to different embodiments of the present disclosure. For ease of explanation, Figures 3 to 5 The solid-line boxes shown in FIG. 1 refer to operations performed by at least one processor included in the electronic device, and Figures 3 to 5 The dashed box shown in refers to input data or output data.

[0089] Reference Figure 3 , the electronic device may first obtain brightness information 302 of the input image 301 from the input image 301. Specifically, the electronic device may obtain contrast information of the input image 301. In this case, the brightness information 302 may be contrast information for each pixel of the input image 301, or may be contrast information for each area divided for each brightness.

[0090] In addition, the electronic device may acquire a noise compensation map 304 (310) using at least one of the acquired brightness information 302 or the camera setting information 303. In this case, the camera setting information 303 is information about the camera that photographed the input image 301, and may include at least one of the lens information or ISO information of the camera.

[0091] The electronic device may acquire the noise compensation map 304 based on the brightness information 302. Specifically, the electronic device may acquire the noise compensation map 304 based on the brightness information 302 so that the lower the brightness of the image, the higher the noise removal intensity.

[0092] The electronic device may remove noise of the input image 301 using the input image 301 and the noise compensation map 304 ( 320 ), and obtain an output image 305 for which noise has been compensated.

[0093] In this case, the electronic device can use the input image 301, the noise compensation map 304 and the artificial intelligence model to obtain the output image 305. Specifically, this will refer to Figure 4 and Figure 5 Detailed description.

[0094] Reference Figure 4 , the electronic device can obtain the output image 403 according to the input image 401 and the noise compensation map 402. Here, Figure 3 The noise compensation map 402 is shown to be acquired using brightness information of the input image 401, and thus repeated description will be omitted.

[0095] First, the electronic device may concatenate the input image 401 and the noise compensation map 402 to each other (410). Specifically, the electronic device may mosaic a plurality of images into one image by connecting the plurality of images to each other.

[0096] In addition, the electronic device may input the connected image to the artificial intelligence model 420 to remove noise. Specifically, the electronic device may extract features of the image while passing the connected image through a convolution (conv) layer. In this case, the connected image may include multiple frames, and the electronic device may extract features of each of the multiple frames.

[0097] In addition, the electronic device may selectively pass the connected image through a rectified linear unit (Relu) layer. The Relu function is used to solve the problem that it is impossible to know what effect the input value has on the result value or the effect of the input value on the result value becomes very slight when the connected image is passed through multiple layers, and the Relu function is a function that performs processing to represent values ​​less than 0 as 0 and represents values ​​greater than 0 in the form of a linear function. Therefore, the effect of the input value on the result value can be better observed, so that accurate learning can be performed.

[0098] At the same time, the electronic device can use the noise removal module 421 to remove the noise of the connected image to remove the noise of the image. Specifically, the noise removal module 421 includes multiple convolutional layers, a batch normalization (BN) layer for normalizing the extracted image features, a Relu layer, etc., and can remove the noise of the connected image by passing the connected image through multiple layers.

[0099] At the same time, the electronic device can selectively reduce the connected image and input the reduced image to the noise removal module 421. As a result, the noise reduction effect can be maximized. Specifically, it has been experimentally confirmed that the electronic device can compress or remove unnecessary information and can more appropriately perform noise removal by reducing the image.

[0100] Meanwhile, a plurality of parameters included in the artificial intelligence model 420 may be pre-trained parameters. Specifically, in addition to the input image 40 being a noise image, the electronic device may also input a high-definition original image of the input image 401 to the artificial intelligence model 420, and may train the parameters of the artificial intelligence model 420 so that the output image 403 from which the noise has been removed is consistent with the high-definition original image.

[0101] At the same time, Figure 4 It is shown in FIG that the input image and the noise compensation map are cascaded with each other, and the cascaded image is input to the artificial intelligence model, but the cascade process can be performed within the artificial intelligence model.

[0102] In addition, Figure 4 , it is shown that the acquired noise compensation map is input into the artificial intelligence model, but only the input image may be input into the artificial intelligence model, and the artificial intelligence model may generate the noise compensation map.

[0103] The above has described an embodiment in which the noise compensation map is input to the artificial intelligence model together with the input image, but the noise compensation map can also be input during the noise removal process. This will be referred to below Figure 5 Give a description.

[0104] Reference Figure 5 , the electronic device may input the input image 501 and the noise compensation map 502 generated based on the input image 501 into the artificial intelligence model 520. In addition, the electronic device may obtain the output image 503 after the noise is compensated.

[0105] In this case, the artificial intelligence model 520 may include a plurality of noise removal modules 521-1, 521-2, and 521-3. Figure 4 The noise removal modules shown are the same, so repeated descriptions will be omitted. Figure 5 Not shown, but in Figure 5 In the embodiment shown, it is also possible to Figure 4 An operation of concatenating an input image 501 and a noise compensation map 502 to each other and an operation of reducing and enlarging an image before and after a noise removal module are shown to be performed.

[0106] In addition, the electronic device can input a noise compensation map between the multiple noise removal modules 521-1, 521-2 and 521-3. As a result, the noise reduction effect is maximized. Specifically, when the image in which the input image 501 and the noise compensation map 502 are connected to each other passes through the convolution layer, the information of the input image and the noise compensation map is combined with each other to obtain new information, and the noise compensation map 502 is input again in the middle, so that the electronic device can perform noise compensation using clearer information.

[0107] Figure 6 is a block diagram showing the configuration of an electronic device for learning and using an artificial intelligence model according to an embodiment of the present disclosure.

[0108] Reference Figure 6 , the processor 600 may include at least one of a learner 610 or an acquirer 620 . Figure 6 The processor 600 may correspond to Figure 1 and Figure 2 The processor 120 of the electronic device 100 or the processor of the data learning server (not shown).

[0109] The learner 610 may generate or learn a model for removing image noise. The learner 610 may generate an artificial intelligence model for removing noise from an image using the collected learning data. The learner 610 may use the collected learning data to generate a learning model with noise removal criteria. Such a learner 610 may correspond to a training set of an artificial intelligence model.

[0110] As an example, the learner 610 may generate, learn, or update a model for removing noise using an input image as a noise image and a noise compensation map as input data. Specifically, the learner 610 may generate, learn, or update a model for removing noise from an image based on brightness information of the image. In addition, the learner 610 may learn or update a model so that the input image as a noise image and a high-definition original image corresponding to the input image are consistent with each other.

[0111] The acquirer 620 may acquire various information using predetermined data as input data of the learned model.

[0112] For example, the acquirer 620 can use the input image and the noise compensation map as input data to acquire (or identify, estimate, infer) the brightness information of the image or the feature information of the image. In addition, the acquirer 620 can use the brightness information and feature information of the acquired image to acquire information related to the noise removal intensity, the pixel value of the image from which the noise is removed, etc.

[0113] At least part of the learner 610 and at least part of the acquirer 620 may be implemented by a software module, or manufactured in the form of at least one hardware chip and installed in an electronic device. For example, at least one of the learner 610 or the acquirer 620 may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured into some existing general-purpose processors (e.g., CPU or application processor) or graphics-specific processors (e.g., graphics processing unit (GPU)), and may be installed on the above-mentioned various electronic devices. In this case, the dedicated hardware chip for artificial intelligence is a dedicated processor dedicated to probability calculation, and due to the higher parallel processing performance than the existing general-purpose processor, computing tasks in the field of artificial intelligence such as machine learning can be quickly processed. In the case where the learner 610 and the acquirer 620 are implemented by a software module (or a program module including instructions), the software module may be stored in a non-temporary computer-readable medium. In this case, the software module may be provided by an operating system (OS) or may be provided by a predetermined application. Alternatively, some software modules may be provided by an operating system (OS), while other software modules may be provided by a predetermined application.

[0114] In this case, the learner 610 and the acquirer 620 may be installed in one electronic device, or may be installed in separate electronic devices, respectively. For example, one of the learner 610 or the acquirer 620 may be included in the electronic device 100, and the other of the learner 610 or the acquirer 620 may be included in an external server. In addition, the learner 610 and the acquirer 620 may communicate with each other in a wired or wireless manner to provide the acquirer 620 with model information configured by the learner 610, and provide the data input to the learner 610 as additional learning data to the learner 610.

[0115] Figure 7a is a block diagram of a learner 610 and an acquirer 620 according to various implementations.

[0116] Reference Figure 7a (a), according to some embodiments, the learner 610 may include a learning data acquirer 610-1 and a model learner 610-4. In addition, the learner 610 may also optionally include at least one of a learning data preprocessor 610-2, a learning data selector 610-3, or a model evaluator 610-5.

[0117] The learning data acquirer 610-1 may acquire the learning data required by the model. As an embodiment of the present disclosure, the learning data acquirer 610-1 may use the input image, the noise compensation map, and the noise compensation map to acquire a compensated image from which the noise of the input image is removed, an original image, etc. as learning data. The learning data may be data collected or tested by the learner 610 or the manufacturer of the learner 610.

[0118] The model learner 610-4 can learn how much noise to remove and how to compensate for the difference between the compensated image and the original image based on the input image and the noise compensation map using the learning data. As an example, the model learner 610-4 can learn the artificial intelligence model by supervised learning using at least some of the learning data as a decision criterion. Optionally, the model learner 610-4 can learn the artificial intelligence model by, for example, unsupervised learning, which finds the decision criteria for situation decisions by performing self-learning using the learning data without separate supervision. In addition, the model learner 610-4 can learn the artificial intelligence model by reinforcement learning, which uses feedback on whether the decision result is correct according to the learned situation. In addition, for example, the model learner 610-4 can learn the artificial intelligence model using a learning algorithm including error back propagation or gradient descent.

[0119] When learning the artificial intelligence model, the model learner 610-4 can store the learned artificial intelligence model. In this case, the model learner 610-4 can store the learned artificial intelligence model in the memory 110 of the electronic device 100. Alternatively, the model learner 610-4 can also store the learned artificial intelligence model in the memory of a server (e.g., an artificial intelligence server) connected to the electronic device 100 via a wired network or a wireless network.

[0120] The learner 610 may further include a learning data preprocessor 610 - 2 and a learning data selector 610 - 3 to improve the recognition result of the artificial intelligence model or save resources or time required to generate the artificial intelligence model.

[0121] The learning data preprocessor 610-2 can preprocess the acquired data so that the acquired data can be used for learning to remove noise from the image. The learning data preprocessor 610-2 can process the acquired data into a predetermined format so that the model learner 610-4 can use the acquired data for learning to remove noise from the image.

[0122] The learning data selector 610-3 can select data required for learning from the data acquired by the learning data acquirer 610-1 or the data preprocessed by the learning data preprocessor 610-2. The selected learning data can be provided to the model learner 610-4. The learning data selector 610-3 can select the learning data required for learning from the acquired or preprocessed data according to a predetermined selection criterion. In addition, the learning data selector 610-3 can select the learning data according to a predetermined selection criterion through the learning performed by the model learner 610-4.

[0123] The learner 610 may also include a model evaluator 610 - 5 to improve the recognition results of the artificial intelligence model.

[0124] The model evaluator 610-5 may input the evaluation data to the artificial intelligence model, and may allow the model learner 610-4 to learn the artificial intelligence model again if the recognition result output from the evaluation data does not meet the predetermined criteria. In this case, the evaluation data may be predetermined data for evaluating the artificial intelligence model.

[0125] For example, the model evaluator 610-5 may evaluate that the learned artificial intelligence model does not meet predetermined criteria when the number or proportion of evaluation data from which the recognition results of the learned artificial intelligence model used for the evaluation data are inaccurate exceeds a predetermined threshold.

[0126] At the same time, in the case where there are multiple learned artificial intelligence models, the model evaluator 610-5 can evaluate whether each learned artificial intelligence model meets the predetermined criteria, and determine the learned artificial intelligence model that meets the predetermined criteria as the final artificial intelligence model. In this case, in the case where the number of learned artificial intelligence models that meet the predetermined criteria is plural, the model evaluator 610-5 can determine any one predetermined learned artificial intelligence model or a predetermined number of learned artificial intelligence models as the final data recognition model in descending order of evaluation scores.

[0127] Reference Figure 7a (b), the acquirer 620 according to some embodiments may include an input data acquirer 620-1 and a supplier 620-4.

[0128] In addition, the acquirer 620 may also optionally include at least one of an input data preprocessor 620 - 2 , an input data selector 620 - 3 , or a model updater 620 - 5 .

[0129] The input data acquirer 620-1 may acquire brightness information in the input image, and acquire a noise compensation map for compensating noise of the input image based on the brightness information of the input image and the setting information of the camera. The supplier 620-4 may apply the input data acquired by the input data acquirer 620-1 as an input value to the learned artificial intelligence model to acquire a compensated image from which noise is removed. The supplier 620-4 applies the data selected by the input data preprocessor 620-2 or the input data selector 620-3, which will be described later, as an input value to the artificial intelligence model to acquire a compensated image from which noise is removed.

[0130] As an example, the supplier 620-4 may apply the input image and the noise compensation map acquired by the input data acquirer 620-1 to the learned artificial intelligence model to acquire (or estimate) a compensated image from which noise is removed.

[0131] The acquirer 620 may further include an input data preprocessor 620 - 2 and an input data selector 620 - 3 to improve the recognition result of the artificial intelligence model or save resources or time used to provide the recognition result.

[0132] The input data preprocessor 620-2 may preprocess the acquired data so that the acquired data may be used to be input to the first model and the second model. The input data preprocessor 620-2 may process the acquired data into a predetermined format so that the supplier 620-4 may use the acquired data to remove noise from the input image.

[0133] The input data selector 620-3 can select data required for situation decision from the data acquired by the input data acquirer 620-1 or the data preprocessed by the input data preprocessor 620-2. The selected data can be provided to the supplier 620-4. The input data selector 620-3 can select some or all of the acquired data or preprocessed data for situation decision according to a predetermined selection criterion. In addition, the input data selector 620-3 can select data according to a predetermined selection criterion through the learning of the model learner 610-4.

[0134] The model updater 620-5 may perform control to update the data recognition model based on the evaluation of the recognition result provided by the supplier 620-4. For example, the model updater 620-5 may provide the compensated image provided by the supplier 620-4 to the model learner 610-4 to request the model learner 610-4 to additionally learn or update the artificial intelligence model.

[0135] Figure 7b is a view showing an example in which an electronic device A and an external server S cooperate with each other to learn and decide data according to an embodiment.

[0136] Reference Figure 7b , an external server S can obtain information related to an input image and a noise compensation map corresponding to the input image, and learn a standard for removing noise from the input image, and electronic device A can use a model generated by server S based on the learning result to obtain a compensated image with the noise of the input image removed.

[0137] In this case, the model learner 610-4 of server S may execute Figure 6 Functions of the learner 610 shown. The model learner 610-4 of the server S can learn the decision criteria (or recognition criteria) for the artificial intelligence model.

[0138] In addition, the supplier 620-4 of the electronic device A may apply the data selected by the input data selector 620-3 to the artificial intelligence model generated by the server S to obtain a compensated image from which the noise of the input image is removed. Alternatively, the supplier 620-4 of the electronic device A may receive the artificial intelligence model generated by the server S from the server S, and obtain a compensated image from which the noise of the input image is removed using the received artificial intelligence model.

[0139] Figure 8 is a view for explaining a process of acquiring a noise compensation map according to an embodiment of the present disclosure.

[0140] Reference Figure 8 , the electronic device can obtain a grayscale image 802 including brightness information of the input image (test image) 801 from the input image 801. In this case, the brightness information may include contrast information of the image.

[0141] For example, the electronic device may acquire a grayscale image of the image based on Equation 1 below.

[0142]

[0143] Here, Im(gray) refers to a value of contrast of a pixel of an image and may have a value from 0 to 255.

[0144] In addition, the electronic device may acquire the vignetting map 803 using lens information in the setting information of the camera that photographs the input image 801. For example, vignetting occurs at the corners of the image, and thus the electronic device may generate the vignetting map 803 based on the positions of pixels included in the image.

[0145] Specifically, the electronic device may acquire the distance between the target pixel and the center pixel as in equation (2).

[0146] Distance = (x-cx) 2 +(y-cy)2 (2)

[0147] Here, (x, y) is the location of the target pixel, and (cx, cy) is the location of the center pixel.

[0148] In addition, the electronic device may acquire the vignetting map 803 using the ratio of the distance acquired from equation (2) to the maximum distance as in equation (3).

[0149] Vignetting = distance / maximum distance (3)

[0150] Here, the maximum distance is the distance between the center pixel and the pixel at the farthest distance, and may refer to the distance between the center pixel and the pixel at the corner of the image. That is, the electronic device may acquire the vignetting image 803 so that the greater the distance between the target pixel and the center pixel, the higher the noise removal intensity.

[0151] In addition, the electronic device may overlap the acquired grayscale image 802 and vignetting image 803 with each other to acquire a noise compensation image 804. In this case, the electronic device may also acquire the noise compensation image 804 using ISO information in the setting information of the camera.

[0152] In addition, the electronic device may overlap the acquired grayscale image 802 and vignetting image 803 with each other to acquire a noise compensation image 804. In this case, the electronic device may also acquire the noise compensation image 804 using ISO information in the setting information of the camera.

[0153] As an example, the electronic device may obtain the noise compensation map 804 using equation (4).

[0154]

[0155] Here, max(iso) refers to the maximum ISO value of the camera that captures the input image 801, and iso refers to the ISO value when capturing the input image 801. That is, the electronic device may acquire a noise compensation map so that the higher the ISO value when capturing the input image 801, the higher the noise removal strength.

[0156] Meanwhile, it has been described above that the grayscale image 802 and the vignetting image 803 overlap each other at the same ratio, but the present disclosure is not limited thereto.

[0157] Fig. 9 is a view for describing an embodiment of a noise compensation map acquired by reflecting setting information of a camera.

[0158] First, if Fig. 9As shown in (a) of FIG. 9 , the input image 901 input for noise removal may include a vignetting phenomenon. Specifically, the outer portion or corner of the input image 901 may become dark or covered with black. This may occur because the outer portion of the lens of the camera is covered by the hood, filter, etc. when a hood, filter, etc. is installed in front of the lens. Alternatively, due to problems with the lens itself, the image of the light passing through the lens is shorter than the diagonal length of the captured picture, which may cause vignetting in the corner portion of the picture. That is, different vignetting may be generated for each lens of the camera that captures the input image 901 or for each installed hood or filter.

[0159] Therefore, the electronic device can use the lens information of the camera that photographs the input image 901 or use the hood or filter information to obtain the image 901. Fig. 9 The vignetting diagram 902 shown in (b) of FIG. Fig. 9 (b) of the present invention can generate a vignetting image 902 in the form of compensating for the vignetting of the input image 901. Specifically, the four corner portions of the input image 901 are dark and have a lot of noise, and thus the electronic device can generate the vignetting image 902 so that the noise removal strength of the four corners of the input image 901 becomes high.

[0160] Fig.10 is a flowchart for describing an image compensation method according to an embodiment of the present invention.

[0161] First, the electronic device can acquire an image (S1010). Specifically, when the electronic device is provided with a camera, the electronic device can acquire an image using the camera. At the same time, the electronic device can also receive an image taken by an external camera.

[0162] Then, the electronic device may obtain a noise compensation map (S1020). Specifically, the electronic device may use the brightness information of the image and the setting information of the camera to obtain the noise compensation map. For example, the electronic device may use at least one of the contrast information of the image, the lens information of the camera, or the ISO information of the camera to obtain the noise compensation map.

[0163] Then, the electronic device may remove the noise of the image (S1030). Specifically, the electronic device may use a noise compensation map to remove the noise of the input image. In this case, the electronic device may input the input image and the noise compensation map into the artificial intelligence model, and output a compensated image with the noise removed. Here, the parameters included in the noise compensation map may be pre-learned parameters.

[0164] During the training process, the electronic device can use the input image as a noise image and the high-definition original image corresponding to the input image to train the parameters of the artificial intelligence model. Specifically, the electronic device can use the noise compensation map of the input image to train the parameters of the artificial intelligence model so that the compensated image with the noise of the input image removed is consistent with the high-definition original image.

[0165] After the training is sufficiently performed, during the testing process, the electronic device may use the trained parameters to remove noise from the input image.

[0166] Fig.11 is a view for comparing an existing image and an image compensated according to the present disclosure with each other. Specifically, Fig.11 (a) is the input image without compensation. Fig.11 (b) is a compensated image obtained by the related art, and Fig.11 (c) is a compensated image obtained according to an embodiment of the present disclosure.

[0167] Fig.11 (b) is a compensated image obtained when only the input image as a noise image is input into the artificial intelligence model to remove noise. Fig.11 The artificial intelligence model in (b) is learned to remove Gaussian noise, and the noise of the input image is removed as a whole, but the image is not processed locally. Therefore, in particularly dark areas within the image, the noise may not be adequately removed and the texture of the image may be lost.

[0168] on the other hand, Fig.11 (c) is a compensated image obtained when a noise compensation map corresponding to an input image is generated and the noise compensation map and the input image are input together into the artificial intelligence model according to the present disclosure, and can be obtained from Fig.11 (c) Confirm the noise removal effect ratio Fig.11 The noise removal effect of (b) is better and the texture of the image is preserved.

[0169] At the same time, the above-mentioned different embodiments can be implemented in a computer or computer-readable recording medium using software, hardware or a combination of software and hardware. According to the hardware implementation, the embodiments described in this disclosure can be implemented using at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a processor, a controller, a microcontroller, a microprocessor or an electronic unit for performing other functions. In some cases, the embodiments described in this specification can be implemented by the processor 120 itself. According to the software implementation, the embodiments such as the processes and functions described in the specification can be implemented by separate software modules. Each software module can perform one or more functions and operations described in the specification.

[0170] Meanwhile, the image processing methods according to the above different embodiments of the present disclosure may be stored in a non-transitory readable medium, which may be installed and used in various devices.

[0171] The non-transitory computer-readable medium is not a medium that temporarily stores data, such as a register, a cache, a memory, etc., but refers to a medium that stores data semi-permanently and can be read by a device. In detail, the program for executing the above-mentioned various methods can be stored and set in a non-transitory readable medium such as a compact disc (CD), a digital versatile disc (DVD), a hard disk, a Blu-ray disc, a universal serial bus (USB), a memory card, a read-only memory (ROM), etc.

[0172] According to an embodiment, the method according to various embodiments disclosed in the present disclosure may include and be provided in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)) or through an application store (e.g., PlayStore). TM ) Online distribution. In the case of online distribution, at least parts of the computer program product may be at least temporarily stored in a storage medium such as a memory of a manufacturer's server, a server of an application store or a relay server, or may be created ad hoc.

[0173] Although the embodiments of the present disclosure have been shown and described above, the present disclosure is not limited to the specific embodiments described above, but can be variously modified by those skilled in the art without departing from the gist of the present disclosure as claimed in the claims. These modifications should be understood to fall within the scope and spirit of the present disclosure.

Claims

1. Electronic equipment, including: a memory including at least one command; as well as a processor, connected to the memory, configured to control the electronic device, Wherein, the processor is configured to: Get an image and a vignetting map based on the camera's lens information, acquiring a noise compensation map for compensating for noise of the image by overlapping the grayscale map of the image and the vignetting map with each other, and The noise of the image is removed by the noise compensation map.

2. The electronic device according to claim 1, wherein: The processor is configured to acquire the noise compensation map based on brightness information of the image so that the noise removal intensity becomes high in a region with low brightness.

3. The electronic device according to claim 2, wherein: The brightness information of the image is the contrast information of each pixel of the image, and The processor is configured to acquire the noise compensation map such that the noise removal strength varies for each pixel.

4. The electronic device according to claim 2, wherein: The processor is configured to divide the image into a plurality of regions for each brightness and acquire the noise compensation map such that the noise removal intensity varies for each divided region.

5. The electronic device according to claim 1, wherein: The processor is configured to acquire setting information of the camera, the setting information of the camera comprising at least one of the lens information of the camera and ISO information of the camera.

6. The electronic device according to claim 5, wherein: The processor is configured to acquire the noise compensation map based on the lens information of the camera so that a noise removal intensity varies according to a vignetting compensation degree.

7. The electronic device according to claim 5, wherein: The processor is configured to obtain the noise compensation map based on the ISO information such that the higher the ISO value, the higher the noise removal strength.

8. The electronic device according to claim 1, wherein: The processor is configured to input the noise compensation map and the image into the learned artificial intelligence model to obtain a compensated image with the noise removed.

9. An image processing method for an electronic device, comprising: Get an image and a vignetting map based on the camera's lens information; acquiring a noise compensation map for compensating for noise of the image by overlapping the grayscale map of the image and the vignetting map with each other; as well as The noise of the image is removed by the noise compensation map.

10. The image processing method according to claim 9, wherein: Acquiring the noise compensation map includes acquiring the noise compensation map based on brightness information of the image so that the noise removal intensity becomes higher in a region with low brightness.

11. The image processing method according to claim 10, wherein: The brightness information of the image is the contrast information of each pixel of the image, and Wherein, acquiring the noise compensation map includes: acquiring the noise compensation map so that the noise removal intensity varies for each pixel.

12. The image processing method according to claim 11, wherein: Acquiring the noise compensation map includes dividing the image into a plurality of regions for each brightness and acquiring the noise compensation map such that the noise removal intensity varies for each divided region. 13 . The image processing method according to claim 9 , further comprising acquiring setting information of the camera, the setting information of the camera comprising at least one of the lens information of the camera and ISO information of the camera.

14. The image processing method according to claim 13, wherein: Acquiring the noise compensation map includes acquiring the noise compensation map based on the lens information of the camera so that the noise removal intensity varies according to the vignetting compensation degree.

15. The image processing method according to claim 13, wherein: Acquiring the noise compensation map includes: acquiring the noise compensation map based on the ISO information, such that the higher the ISO value, the higher the noise removal strength.

Images