Electronic device and method for enhancing partial area of image

Abstract

An electronic device according to one embodiment: estimates, from an image, a first region including a designated object and a second region distinguished from the first region; changes a first luminance of pixels included in the first region to a second luminance in order to move a luminance distribution range of the pixels; changes a first color of at least one of the pixels to a second color such that a luminance deviation of the pixels increases; and changes a color of at least one of the pixels to a third color on the basis of a weighted sum of the first color and the second color, thereby providing a corrected image for the image.

Description

Electronic device and method for enhancing a portion of an image

[0001] The present disclosure relates to an electronic device and method for enhancing a portion of an image.

[0002] An electronic device can acquire an image captured by a camera or downloaded from an external electronic device. The electronic device can correct the acquired image. For example, the electronic device can correct a certain area within the image by changing the brightness and / or color of the area to make the area clearer.

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art in connection with the present disclosure.

[0004] An electronic device is provided. The electronic device may include at least one processor. The electronic device may include a memory including one or more storage media storing instructions. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to estimate, from an image stored in the memory, a first region including a specified object and a second region distinct from the first region using a first artificial intelligence model. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to change a first luminance of pixels included in the first region to a second luminance so as to shift a luminance distribution range of the pixels included in the first region based on a reference luminance. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to change a first color of at least one pixel among the pixels to a second color so as to increase a luminance deviation of the pixels. The instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to provide a corrected image for the image by changing the color of at least one pixel of the pixels to a third color based on a weighted sum of the first color and the second color.

[0005] A method performed by an electronic device including a memory is provided. The method may include an operation of estimating, from an image stored in the memory, a first region including a specified object and a second region distinct from the first region using a first artificial intelligence model. The method may include an operation of changing a first luminance of pixels included in the first region to a second luminance to shift a luminance distribution range of the pixels included in the first region based on a reference luminance. The method may include an operation of changing a first color of at least one of the pixels to a second color so that a luminance deviation of the pixels increases. The method may include an operation of providing a corrected image for the image by changing the color of at least one of the pixels to a third color based on a weighted sum of the first color and the second color.

[0006] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0007] Figure 2 illustrates components of an electronic device according to one embodiment.

[0008] Figure 3 shows an example of an image.

[0009] FIG. 4 is a flow chart showing an operation of an electronic device correcting an image according to one embodiment.

[0010] Figure 5 shows an example of a histogram obtained from an image.

[0011] Figure 6a shows an example of a histogram for a first region of the original image.

[0012] Figure 6b shows an example of a histogram for a first region of a corrected image.

[0013] Figure 7 illustrates an example of image correction based on a weighted sum.

[0014] FIG. 8 is a flowchart illustrating an operation of an electronic device according to one embodiment of the present invention to determine whether to correct an image based on detection of a specified object.

[0015] FIG. 9 is a flowchart illustrating an operation of correcting a first area of ​​an electronic device according to one embodiment.

[0016] FIGS. 10A, 10B, and 10C illustrate a process of an electronic device according to one embodiment of the present invention for reinforcing a first region.

[0017] FIGS. 11A, 11B, 11C, and 11D illustrate a process of an electronic device according to one embodiment of the present invention for reinforcing a first region.

[0018] FIG. 12 illustrates an example of an electronic device displaying preferred images according to one embodiment.

[0019] FIG. 13 is a flowchart illustrating an example of an operation of an electronic device according to one embodiment of the present invention to change the brightness of first pixels.

[0020] FIG. 14 is a flowchart illustrating an example of an operation of an electronic device according to one embodiment of the present invention to change the color of first pixels.

[0021] Figure 15a illustrates a display displaying a first visual object.

[0022] Figure 15b illustrates a display displaying a second visual object.

[0023] Figure 15c illustrates a display showing an original image and a corrected image.

[0024] Figure 16 illustrates a display that displays a fourth visual object.

[0025] FIG. 1 is a block diagram of an electronic device within a network environment, according to one embodiment.

[0026] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0027] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0028] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, in the electronic device (101) itself where artificial intelligence is performed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0029] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0030] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0031] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0032] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0033] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0034] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0035] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0036] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) to an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0037] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0038] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0039] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0040] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0041] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0042] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0043] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0044] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0045] In one embodiment, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0046] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0047] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0048] Figure 2 illustrates components of an electronic device according to one embodiment.

[0049] Referring to FIG. 2, an electronic device (e.g., electronic device (101) of FIG. 1) may include a memory (210) (e.g., memory (130) of FIG. 1), a processor (220) (e.g., processor (120) of FIG. 1), a display (230) (e.g., display module (160) of FIG. 1), and / or a camera (240) (e.g., camera module (180) of FIG. 1).

[0050] According to one embodiment, the memory (210) may store data such as instructions for the operation of the electronic device (101), basic programs, applications, and setting information. The memory (210) may be configured as a volatile memory, a non-volatile memory, or a combination of volatile and non-volatile memories. The memory (210) may store at least one image. For example, the memory (210) may store an image captured using the camera (240) or an image acquired from an external electronic device. The memory (210) may provide at least one stored image based on a request from at least one processor (220). The memory (210) may include various memories. For example, the memory (210) may include a buffer memory for temporarily storing at least a portion of an image acquired through the camera (240) for the next image processing task, or a memory (e.g., a server, a cloud) connected to an image signal processor (ISP). Within the present disclosure, the image to be corrected may include a copy image previewed through the display (230) in addition to the image stored in the memory (210).

[0051] Any function or operation described herein may be processed by at least one processor (220). According to one embodiment, at least one processor (220) may include a processing circuit. At least one processor (220) may include, but is not limited to, an application processor (AP, e.g., a central processing unit (CPU)) and / or a communication processor (CP, e.g., a modem). At least one processor (220) may include, but is not limited to, a graphics processing unit (e.g., a GPU), a neural processing unit (NPU) (e.g., an artificial intelligence (AI) chip), a wireless-fidelity (Wi-Fi) chip, or a Bluetooth chip. ® , GPS (global positioning system) chip, near field communication (NFC) chip, connectivity chips, sensor controller, touch controller, finger-print sensor controller, display drive integrated circuit (DDI), audio CODEC chip, universal serial bus (USB) controller, camera controller, image processing IC, microprocessor unit (MPU), system on chip (SoC), integrated circuity (IC), or similar circuits.

[0052] According to one embodiment, at least one processor (220) may control the operation of the electronic device (101). For example, the at least one processor (220) may cause the electronic device (101) to correct at least one image stored in the memory (210) by executing instructions stored in the memory (210). Within the present disclosure, correction of an image may be referred to as enhancing a contrast effect for an object (e.g., a cloud) included within a certain area of ​​the image (e.g., a sky area), but is not limited thereto.

[0053] According to one embodiment, at least one processor (220) may obtain an image from the memory (210). The at least one processor (220) may detect a designated object (330) included in the image (e.g., a tree (341), grass (342), a road (343), a building (344), clouds, the sky, etc. of FIG. 3) using an AI model or algorithm. Based on the detection of the designated object (330), the at least one processor (220) may estimate a first region of the image (e.g., the first region (310) of FIG. 3) including the designated object (330) and a second region of the image (e.g., the second region (320) of FIG. 3) that is distinct from the first region (310). The at least one processor (220) may correct the image to enhance the first region (310) including the designated object (330). According to one embodiment, at least one processor (220) can distinguish and identify at least one area (e.g., sky area, non-sky area) using an AI model. For example, the designated object (330) may be, but is not limited to, a cloud. For example, correction of the image may be performed by changing the luminance of the first area (310) in order to shift the luminance distribution range of the first area (310). For example, correction of the image may be performed by increasing the luminance deviation of the first area (310). An operation of providing a corrected image by strengthening the first area (310) by the at least one processor (220) is described below with reference to FIG. 4.

[0054] According to one embodiment, at least one processor (220) may include an object detection unit (221) for detecting a designated object (330) included in an image, a region information extraction unit (222) for extracting a first region (310) and a second region (320), and / or an image enhancement unit for providing a corrected image by enhancing the first region (310). According to one embodiment, the object detection unit (221), the region information extraction unit (222), and / or the image enhancement unit (223) may be a storage space that stores instructions / codes or instructions / codes that are at least temporarily resided in at least one processor (220) as a stored instruction set or code, or may be a part of circuitry constituting at least one processor (220). According to one embodiment, at least one of the object detection unit (221), the area information extraction unit (222), and / or the image enhancement unit (223) may be logically separated within at least one processor (220) or configured as a separate image processing processor physically separated from at least one processor (220).

[0055] According to one embodiment, the display (230) may be configured to display visual information. For example, the display (230) may be configured to display images and / or visual objects based on the control of at least one processor (220). The at least one processor (220) may control a display driver integrated circuit (DDI) so that the images and / or visual objects are displayed on the display (230). For example, the image data may include at least one pixel-specific information (e.g., color, brightness). By driving at least one pixel based on a voltage value or a current value, visual information corresponding to the image data may be displayed on the display (230).

[0056] According to one embodiment, the display (230) can display an original image before correction and / or a corrected image after correction. For example, the display (230) can display a corrected image in which the saturation, brightness, and / or color of the image have been changed according to the processing of the image enhancement unit (223). According to one embodiment, the display (230) can display at least one visual object (e.g., a user interface) for receiving user input. The original image, the corrected image, and the at least one visual object displayed on the display (230) are described below with reference to FIG. 15C.

[0057] In one embodiment, the camera (240) can generate an image by photographing a subject. For example, the camera (240) may include components such as a lens for collecting light emitted from the subject and an image sensor for converting the light collected through the lens into an electrical signal. The image captured by the camera (240) may be stored in the memory (210).

[0058] According to one embodiment, at least one processor (220) can generate a corrected image by enhancing or improving at least a portion of an original image using an artificial intelligence model. For example, image processing using an artificial intelligence model can be performed based on machine learning and deep learning algorithms. The artificial intelligence model can improve the quality of the image by learning how computers understand and analyze images and identifying features and patterns in the image. Image processing techniques utilizing the results of the learning can provide a corrected image by improving details within the image, removing noise, and optimizing color and brightness. The electronic device (101) can provide the corrected image to the user through image processing. For example, when a user captures a landscape using a camera (240), the sky area within the landscape may have a relatively narrow range of brightness distribution. If clouds are included within the sky area, the boundaries of the clouds may not be clearly distinguished, resulting in an image that feels different from the actual landscape. In order to enhance the sky area, if the RGB value of at least one pixel included in the sky area is simply corrected to a specified RGB value, it may be difficult to produce a high-quality image even if the original image is corrected.

[0059] An electronic device (101) according to one embodiment can provide a corrected image by distinguishing a first region (310) and a second region (320) of an image including a specified object (330) based on detection of a specified object (330) included in the image and enhancing the first region (310). The electronic device (101) can enhance a user experience by providing a corrected image similar to an actual landscape by enhancing an original image using image processing technology. Hereinafter, an electronic device (101) for providing a corrected image using image processing technology will be described.

[0060] Figure 3 shows an example of an image.

[0061] The image (300) illustrated in FIG. 3 may be an example of at least one image stored in a memory (e.g., memory (210) of FIG. 2). The image (300) may be an image captured by a camera (e.g., camera (240) of FIG. 2) or an image acquired from an external electronic device.

[0062] Referring to FIG. 3, the image (300) may be an image of a landscape. For example, various objects (341, 342, 343, 344, 330) may be included in the image (300). The various objects (341, 342, 343, 344, 330) may include, but are not limited to, objects such as trees (341), grass (342), roads (343), buildings (344), and clouds (330).

[0063] According to one embodiment, the image (300) may include multiple regions. For example, based on the type, characteristics, or settings of at least one object, the multiple regions may be distinguished into multiple regions, such as a first region, a second region, and / or a third region. In the examples described below, the image (300) is described as including a first region (310) and a second region (320), but is not limited thereto. For example, the image (300) may include three or more regions.

[0064] According to one embodiment, the image (300) may include a first region (310) and a second region (320). The first region (310) may be referred to as a portion of the image (300) that includes a designated object (330). For example, the designated object (330) may be a cloud. For example, the first region (310) may be a sky region that includes a cloud. However, the present invention is not limited thereto. For example, the designated object (330) may be a tree, and the first region (310) may be a grass region that includes a tree. The second region (320) may be referred to as another portion of the image (300) that is distinct from the first region (310). For example, the second region (320) may be a ground region that is distinct from the sky region. The ground area may be an area that includes objects such as trees (341), grass (342), roads (343), and buildings (344). Within the present disclosure, as an example of the first area (310), the second area (320), and the designated object (330), the first area (310) may be referred to as a sky area, the second area (320) may be referred to as a non-sky area, and the designated object (330) may be referred to as a cloud. Within the present disclosure, the first area (310) may be referred to as a sky area, the second area (320) may be referred to as a non-sky area, and the designated object (330) may be referred to as a cloud, but these are merely examples for convenience of description and are not limiting.

[0065] According to one embodiment, when the image (300) is an image captured during the daytime, the brightness of pixels included in the sky area of ​​the image (300) may appear high due to sunlight. Since clouds are located within the sky area, the boundaries of the clouds may not appear clearly. If the brightness of the pixels included in the sky area appears high and the boundaries of the clouds are not clearly visible, the quality of the image (300) capturing the landscape may deteriorate. For example, since the brightness of the pixels included in the sky area appears high, it may be difficult to express a clear sky, and the contrast between the clouds and the sky area may not be clearly expressed. This deterioration may cause a difference between an actual landscape and a landscape displayed by the image (300).

[0066] An electronic device (e.g., electronic device (101) of FIG. 2) according to one embodiment can correct an image (300) by extracting a sky area and a non-sky area within an image (300) and adjusting the brightness and color of first pixels included within the sky area. In the corrected image, the brightness and color of the sky area are enhanced, thereby representing a clear sky area.

[0067] FIG. 4 is a flowchart illustrating an operation of an electronic device correcting an image according to one embodiment. FIG. 5 illustrates an example of a histogram obtained from an image. FIG. 6a illustrates an example of a histogram for a first region of an original image. FIG. 6b illustrates an example of a histogram for a first region of a corrected image. FIG. 7 illustrates an example of correcting an image based on a weighted sum.

[0068] The operations described in FIG. 4 may be operations performed by an electronic device (e.g., an electronic device (101) of FIG. 2) when instructions stored in a memory (e.g., a memory (210) of FIG. 2) are individually or collectively executed by at least one processor (e.g., at least one processor (220) of FIG. 2).

[0069] Referring to FIG. 4, at operation 401, the instructions, when individually or collectively executed by at least one processor (220), may cause the electronic device (101) to estimate a first region (e.g., the first region (310) of FIG. 3) containing a designated object (e.g., the designated object (330) of FIG. 3) from an image (e.g., the image (300) of FIG. 3)) and a second region (320) distinct from a second region (e.g., the second region (320) of FIG. 3).

[0070] According to one embodiment, at least one processor (220) may acquire an image stored in the memory (210). The at least one processor (220) may estimate a first region (310) and a second region (320) within the image based on a user input requesting generation of a corrected image for the acquired original image. For example, the at least one processor (220) may use a first artificial intelligence model to distinguish between the first region (310) and the second region (320). For example, the first artificial intelligence model may be, but is not limited to, an artificial intelligence model trained to distinguish between the first region (310) and the second region (320) based on detection of a designated object (330) included within the first region (310). For example, the first artificial intelligence model may be an artificial intelligence model trained to distinguish between the first area (310) and the second area (320) based on the difference between the color of the first area (310) and the color of the second area (320).

[0071] According to one embodiment, the estimation of the first region (310) and the second region (320) may be referred to as classifying all pixels included in the image into a class corresponding to the first region (310) and a class corresponding to the second region (320). For example, at least one processor (220) may assign, to each of the total pixels included in the image, a first label corresponding to the first region (310) or a second label corresponding to the second region (320). The first label may be assigned to the first pixels included in the first region (310), and the second label may be assigned to the second pixels included in the second region (320).

[0072] In operation 403, the instructions, when individually or collectively executed by at least one processor (220), may cause the electronic device (101) to change the first luminance of the first pixels to a second luminance so as to shift the luminance distribution range of the first pixels included in the first region (310) based on a reference luminance (e.g., the reference luminance (500) of FIG. 5).

[0073] According to one embodiment, the first region (310) may be referred to as a sky region, and the second region (320) may be referred to as a non-sky region that is distinct from the sky region. In the case of an image captured during the day when the sun is within the sky region, the sky region may appear relatively bright, and in the case of an image captured at night when the sun has set, the sky region may appear relatively dark. When the sky region appears relatively bright, the non-sky region may appear relatively bright, and when the sky region appears relatively dark, the non-sky region may appear relatively bright. The impression and atmosphere of the image may be distorted due to the contrast effect between the luminance of the first pixels included in the sky region and the luminance of the second pixels included in the non-sky region. At least one processor (220) may adjust the luminance of the first pixels by shifting the luminance distribution range of the first pixels based on the reference luminance (500). For example, at least one processor (220) may change the first luminance of the pixels to a second luminance. Changing the luminance of the pixels may have substantially the same effect as changing the exposure value of a camera (e.g., camera (240) of FIG. 1).

[0074] According to one embodiment, at least one processor (220) may obtain a histogram of the luminance of first pixels estimated to be in the first region (310). The histogram is a graphical representation of the luminance distribution of an image, and may be expressed as the number of pixels according to luminance values. Referring to FIG. 5, the horizontal axis of the histogram represents luminance values, and the vertical axis of the histogram represents the number of pixels.

[0075] According to one embodiment, a reference luminance (500) may be determined in advance. The reference luminance (500) may be determined based on the luminance distribution of first pixels included in a sky region within an image when a sky exhibiting ideal brightness is photographed. The sky exhibiting ideal brightness may be referred to as a sky whose brightness is neither too bright nor too dark compared to the brightness of at least one object included in the image, and has a similar brightness. For example, the first graph (501) of FIG. 5 may be a graph representing the distribution of a histogram of first pixels for a sky region that is neither too bright nor too dark compared to a non-sky region. The reference luminance (500) may be a default value. For example, the reference luminance (500) may be, as a representative value of the first graph, one of the mean, median, mode, or expected value for the luminance distribution represented by the first graph (501), but is not limited thereto. According to one embodiment, the reference luminance (500) may be determined based on at least one preferred image. Descriptions of the at least one preferred image are described below with reference to FIG. 12. According to one embodiment, the reference luminance (500) may be used to determine whether the luminance of the first pixels is bright or dark. For example, by comparing the total bin point (the sum of the bin start point (e.g., cumulative 0.03%) and the bin end point (e.g., cumulative 99.7%) of the first pixels with the reference luminance (500), it can be determined whether the value is brighter or darker than the reference luminance (500).For example, if the total blank point is greater than the reference luminance (500), the luminance of the first pixels may be judged to be bright, and if the total blank point is less than the reference luminance (500), the luminance of the first pixels may be judged to be dark.

[0076] The second graph (502) of FIG. 5 may be a graph having a brighter luminance distribution range than the first graph (501). For example, in the case of an image of the sky having a brightness brighter than the ideal brightness, the luminance distribution range of the first pixels included in the sky area may have the same luminance distribution range as the second graph (502). For example, the representative value of the second graph (502) may have a luminance value higher than the reference luminance (500). According to one embodiment, at least one processor (220) may shift the luminance distribution range of the first pixels based on the reference luminance (500). For example, at least one processor (220) may lower the luminance of the first pixels so that the representative value of the second graph (502) becomes closer to the reference luminance (500). As the luminance of the first pixels decreases, the second graph (502) can move closer to the first graph (501) having the reference luminance (500). As the second graph (502) moves closer to the first graph (501), sky areas having too high luminance can be changed to have an ideal luminance or a luminance similar to the ideal luminance.

[0077] The third graph (503) of FIG. 5 may be a graph having a darker luminance distribution range than the first graph (501). For example, in the case of an image of a sky having a brightness darker than the ideal brightness, the luminance distribution range of the first pixels included in the sky area may have the same luminance distribution range as the third graph (503). For example, the representative value of the third graph (503) may have a luminance value lower than the reference luminance (500). According to one embodiment, at least one processor (220) may shift the luminance distribution range of the first pixels based on the reference luminance (500). For example, at least one processor (220) may increase the luminance of the first pixels so that the representative value of the third graph (503) becomes closer to the reference luminance (500). As the luminance of the first pixels increases, the third graph (503) can move closer to the first graph (501) having the reference luminance (500). As the third graph (503) moves closer to the first graph (501), sky areas having too low luminance can be changed to have ideal luminance or luminance similar to the ideal luminance.

[0078] According to one embodiment, at least one processor (220) may set a maximum correction value (e.g., a first maximum correction value) of the luminance to be changed from the first luminance when shifting the luminance distribution range of the first pixels. In order to reduce overcorrection due to excessive shift, the shift of the luminance distribution range may be limited by the first maximum correction value. For example, an example of pseudo code for changing the luminance of the first pixels may be referred to in [Table 1] below.

[0079]

[0080] Referring to the above [Table 1], a change in the exposure value may cause a change in the luminance value. MAX_CHANGED_EXPOSURE may be referenced as the maximum correction value of the luminance to be changed (e.g., the first maximum correction value). Referring to the above [Table 1], the first maximum correction value is exemplified as 40, but is not limited thereto. For example, when the total bin point is greater than or less than the reference luminance (500), the luminance distribution of the first pixels may shift within the limit of 40. The above pseudo code is merely exemplary and is not limited thereto.

[0081] Referring again to FIG. 4, at operation 405, the instructions, when individually or collectively executed by at least one processor (220), may cause the electronic device (101) to change a first color of at least one pixel of the first pixels to a second color such that a luminance deviation of the first pixels increases.

[0082] The graph (601) illustrated in Fig. 6a represents a histogram for a first region (310) of the original image before correction. Referring to Fig. 6a, the histogram of the first pixels included in the sky region of the original image may be concentrated within a certain range. Since the colors represented within the sky region are limited, the luminance distribution range of the first pixels included in the sky region may be narrow. For example, objects that can be found within the sky may include objects such as airplanes, satellites, and birds, but since it is difficult for fixed objects to appear other than objects such as the sun, clouds, and the moon, it may be difficult for the sky region to have colors other than colors corresponding to clouds and colors corresponding to the sky. For example, for a range of 256 luminance values ​​from 0 to 255, the luminance distribution range of the first pixels may be concentrated within a relatively narrow first range (610). Since the luminance distribution range of the first pixels is concentrated within the first range (610), the contrast between clouds and the sky is not effectively displayed within the original image, making it difficult for the sky area to appear clearly.

[0083] According to one embodiment, at least one processor (220) may change a first color of at least one pixel among the first pixels to a second color so as to increase a luminance deviation of the first pixels. The first color may be referenced as a color of a pixel within an original image, and the second color may be referenced as a color of a pixel within a corrected image. For example, the at least one processor (220) may expand a distribution of the first pixels by histogram stretching of the first pixels. For example, the at least one processor (220) may change the color of the first pixels by scaling RGB channel values.

[0084] The graph (602) illustrated in FIG. 6B represents a histogram for a first region (310) of a corrected image. Referring to FIG. 6B, the luminance distribution range of the first pixels, which was concentrated within a relatively narrow first range (610) in the original image, may be stretched to a relatively wide second range (620). For example, at least one processor (220) may increase the luminance deviation of the first pixels through histogram stretching or RGB channel value scaling of the first pixels. As the luminance deviation increases, the distribution range of the first pixels may increase, so that the contrast between clouds and the sky in the corrected image may be clearly expressed, thereby making the sky region appear clear.

[0085] Referring again to FIG. 4, at operation 407, the instructions, when individually or collectively executed by at least one processor (220), may cause the electronic device (101) to change the color of at least one pixel of the first pixels to a third color based on a weighted sum of the first color and the second color.

[0086] According to one embodiment, at least one processor (220) can change the color of at least a portion of the first pixels based on a weighted sum of the first color and the second color. For example, the luminance and color of at least a portion of the first pixels included in the sky area can be changed according to the performance of operations 403 and 405. The first luminance of the first pixels can be changed to a second luminance. The first color of at least one pixel among the first pixels can be changed to a second color. Since the pixels changed by the performance of operations 403 and 405 can be the first pixels, even if operations 403 and 405 are performed, the luminance and color of the second pixels included in the non-sky area can be maintained. Since the luminance and color of the second pixels are maintained, the luminance and color of the second pixels in the original image can be substantially the same as the luminance and color of the second pixels in the corrected image. The brightness and color of the first pixels positioned on the boundary between the first region (310) and the second region (320) may differ significantly from the brightness and color of the second pixels. If the brightness and color of the first pixels positioned on the boundary change abruptly, the difference may cause the image to appear unnatural and awkward.

[0087] According to one embodiment, at least one pixel among the first pixels may be changed to a third color based on a weighted sum of the first color and the second color. FIG. 7 illustrates an example in which a weighted sum is applied to at least one pixel among the first pixels (710) that is adjacent to the second pixels (720). In FIG. 7, the at least one pixel that is changed to the third color based on the weighted sum is illustrated as a pixel located on the border between the first region (310) and the second region (320), but is not limited thereto. For example, at least one processor (220) may also change the color of each of the first pixels (710) based on the weighted sum.

[0088] Referring to FIG. 7, among the first pixels (710), the color of the third pixel (730) may be changed to a third color based on a weighted sum of the first color and the second color. The weight of the weighted sum may be determined based on the number of first pixels (710) included in the first region (310) and the number of second pixels (720) included in the second region (320) within a specified range (740) from at least one pixel that is changed to the third color. For example, assuming that the specified range (740) is a virtual circle having a radius including three pixels centered on the third pixel (730), 29 pixels may be included within the virtual circle. Among the 29 pixels included in the above-mentioned range (740), the number of first pixels (711) included in the sky area corresponding to the first area (310) may be 19, and the number of second pixels (721) included in the non-sky area corresponding to the second area (320) may be 10. Since the second pixels (721) have the first color, which is the color in the original image, the first weight corresponding to the first color may be calculated as 0.345 (=10 / 29) according to the number of the second pixels (721). Since the first pixels (711) have changed from the first color to the second color according to the performance of operation 405, the second weight corresponding to the second color may be calculated as 0.655 (=19 / 29) according to the number of the first pixels (711). In the example illustrated in FIG. 7, at least one processor (220) can determine a third color of a third pixel (730) based on the following [Mathematical Formula 1].

[0089]

[0090] According to one embodiment, when applying a weighted sum according to the above weights, as the number of first pixels (711) included in the specified range (740) increases, the third color may be closer to the pixel value (e.g., the second color) of the corrected image, and as the number of second pixels (721) included in the specified range (740) increases, the third color may be closer to the pixel value (e.g., the first color) of the original image. When applying the weighted sum of the first color and the second color to the entire first pixels (710), the color of the third pixel (730) may be determined as a third color that is closer to the second color as it is spaced away from the boundary between the first region (310) and the second region (320) toward the interior of the first region (310). For example, if only the first pixels (711) are included within a specified range (740) from the third pixel (730), the third color may be substantially the same as the second color. By being positioned on the boundary, the third pixels (730) that are in contact with the second pixels (720) may be changed to a third color close to the first color. When a weighted sum is applied to all of the first pixels (710), the color change of the first pixels (710) is performed gradually, thereby generating a natural final image.

[0091] An electronic device (101) according to one embodiment can provide a corrected image by enhancing a first region (310) within an original image. The first region (310) within the corrected image can have a luminance distribution range close to the luminance distribution range of the ideal first region (310) by moving the luminance distribution range closer to the ambient luminance. The first region (310) within the corrected image can display a clear first region (310) by having a histogram with increased luminance deviation. The first region (310) within the corrected image can reduce unnaturalness due to the correction and provide a natural impression by naturally connecting at the boundary where it touches the second region (320). The electronic device (101) according to one embodiment can provide an enhanced user experience by providing a corrected image including an enhanced first region (310).

[0092] FIG. 8 is a flowchart illustrating an operation of an electronic device according to one embodiment of the present invention to determine whether to correct an image based on detection of a specified object.

[0093] The operations described in FIG. 8 may be operations performed by an electronic device (e.g., an electronic device (101) of FIG. 2) when instructions stored in a memory (e.g., a memory (210) of FIG. 2) are individually or collectively executed by at least one processor (e.g., at least one processor (220) of FIG. 2).

[0094] Referring to FIG. 8, at operation 801, the instructions, when individually or collectively executed by at least one processor (220), may cause the electronic device (101) to detect a specified object (e.g., the specified object (330) of FIG. 3) from an image.

[0095] According to one embodiment, at least one processor (220) may be configured to detect a designated object (330) within an image using a second artificial intelligence model. For example, the designated object (330) may be referred to as an object for estimating a first region (e.g., the first region (310) of FIG. 3). For example, if the first region (310) is a sky region, the designated object (330) may be, but is not limited to, a cloud.

[0096] According to one embodiment, at least one processor (220) may utilize a second artificial intelligence model to detect a designated object (330). For example, the second artificial intelligence model may be an artificial intelligence model trained to detect the designated object (330). The second artificial intelligence model may be substantially the same as the first artificial intelligence model for distinguishing between the first region (310) and the second region (e.g., the second region (320) of FIG. 3), or may be implemented as a part of the first artificial intelligence model, but is not limited thereto.

[0097] At operation 803, the instructions, when individually or collectively executed by at least one processor (220), cause the electronic device (101) to identify whether an object has been detected.

[0098] According to one embodiment, at least one processor (220) may be configured to identify whether a designated object (330) is detected within an image and perform enhancement of the original image based on the identification. For example, a enhanced image of the first region (310) may be provided if the designated object (330) is present within the first region (310). If the designated object (330) is present, the at least one processor (220) may perform the operations described in FIG. 4, and if the designated object (330) is not present, the operations illustrated in FIG. 4 may not be performed. For example, if the designated object (330) is not present within the first region (310), the at least one processor (220) may not perform enhancement operations for the first region (310) because contrast by the designated object (330) is unnecessary. For example, if a designated object (330) exists within the first region (310), at least one processor (220) may perform enhancement operations on the first region (310), since the first region (310) may be expressed unclearly.

[0099] According to one embodiment, whether a designated object (330) is detected may be determined based on whether a specified ratio of the designated object (330) is present in the image, in addition to the presence of the designated object (330). Even if the designated object (330) is present in the image, if the ratio of the area of ​​the first region (310) occupied by the designated object (330) is less than the specified ratio, it may be substantially the same as if the designated object (330) does not exist. For example, at least one processor (220) may be configured to detect the designated object (330) based on identifying, using the second artificial intelligence model, that the ratio of the area of ​​the designated object (330) to the area of ​​the first region (310) is greater than or equal to the specified ratio. For example, at least one processor (220) may determine that the designated object (330) is present based on identifying that the ratio of the area occupied by clouds to the area of ​​the sky region in the image is greater than or equal to the specified ratio (e.g., about 5%). In operation 803, if the specified object (330) is detected, operation 805 may be performed. In operation 803, if the specified object (330) is not detected, operation 807 may be performed.

[0100] At operation 805, the instructions, when individually or collectively executed by at least one processor (220), may cause the electronic device (101) to provide a corrected image.

[0101] According to one embodiment, at least one processor (220) may provide a corrected image in which the first region (310) is enhanced based on detection of a designated object (330). For example, if the designated object (330) is present in the image, or if the ratio of the area of ​​the designated object (330) to the area of ​​the first region (310) is greater than or equal to a designated ratio, the at least one processor (220) may provide a corrected image by performing the operations described in FIG. 4. As the corrected image is provided, an image in which the first region (310) is enhanced may be provided.

[0102] At operation 807, the instructions, when individually or collectively executed by at least one processor (220), may cause the electronic device (101) to provide an original image.

[0103] According to one embodiment, at least one processor (220) may provide an original image based on the failure to detect a specified object (330). Providing an original image may be referred to as at least one processor (220) not correcting the original image. For example, if the specified object (330) does not exist in the image, or the ratio of the area of ​​the specified object (330) to the area of ​​the first region (310) is less than a specified ratio, the at least one processor (220) may not perform the operations described in FIG. 4.

[0104] According to one embodiment, the enhancement of the first region (310) may be configured to correct the original image based on the detection of the specified object (330), since it is intended to effectively contrast the first region (310) with the specified object (330). For example, if clouds exist within the sky region or a certain amount of clouds exist, the electronic device (101) may provide a clear image by enhancing the contrast between the sky and the clouds. The descriptions of the sky and clouds are merely exemplary, and the present disclosure is not limited thereto. For example, if the image includes flowers in a lawn, buildings placed on a street, or food comprising multiple ingredients, a corrected image may be provided.

[0105] FIG. 9 is a flowchart illustrating an operation for correcting a first region of an electronic device according to one embodiment. FIGS. 10a, 10b, and 10c illustrate a process for strengthening a first region of an electronic device according to one embodiment.

[0106] The operations described in FIG. 9 may be operations performed by an electronic device (e.g., an electronic device (101) of FIG. 2) when instructions stored in a memory (e.g., a memory (210) of FIG. 2) are individually or collectively executed by at least one processor (e.g., at least one processor (220) of FIG. 2).

[0107] The operations described in FIG. 9 may be operations performed between operations 401 and 403 of FIG. 4. If an error is identified in the estimation of a first region (e.g., the first region (310) of FIG. 3) and a second region (e.g., the second region (320) of FIG. 3), at least one processor (220) may correct the first region (310) and the second region (320) by removing the error. For example, an error may be caused by assigning a label corresponding to a non-sky region to a pixel included in a sky region, or by assigning a label corresponding to a sky region to a pixel included in a non-sky region. At least one processor (220) may correct the estimated first region (310) and the second region (320) by comparing a segmentation mask and an edge map to remove the error.

[0108] Referring to FIG. 9, at operation 901, the instructions, when individually or collectively executed by at least one processor (220), may cause the electronic device (101) to obtain a segmentation mask according to the first region (310) and the second region (320).

[0109] According to one embodiment, at least one processor (220) may obtain a segmentation mask (e.g., segmentation mask (1001) of FIG. 10A) according to a first region (310) and a second region (320) using a first artificial intelligence model. According to one embodiment, at least one processor (220) may estimate the first region (310) and the second region (320) and obtain a segmentation mask according to the first region (310) and the second region (320). At least one processor (220) may obtain the segmentation mask by assigning each pixel included in the image to a label corresponding to the first region (310) or the second region (320).

[0110] FIG. 10A illustrates a segmentation mask (1001) generated for an image illustrated in FIG. 3 (e.g., image (300) of FIG. 3). Referring to FIG. 10A, at least one processor (220) may obtain a segmentation mask (1001) based on estimates of a first region (310) and a second region (320). At least one processor (220) may assign, to each of all pixels in the image, a first label corresponding to the first region (310) or a second label corresponding to the second region (320). As illustrated in FIG. 10A, within the segmentation mask (1001), pixels (1010) assigned with the first label may be displayed in white, and pixels (1020) assigned with the second label may be displayed in black.

[0111] Referring again to FIG. 9, at operation 903, the instructions, when individually or collectively executed by at least one processor (220), may cause the electronic device (101) to identify errors for the estimated first region (310) and second region (320).

[0112] According to one embodiment, at least one processor (220) may be configured to identify an error in the estimated first region (310) and second region (320). For example, an error may occur in the estimated first region (310) and second region (320) using the first artificial intelligence model. As illustrated in FIG. 10A, a hole region (1030) may occur in the segmentation mask (1001) due to improper recognition of a certain region within the image. For example, a hole region (1030) may occur due to a portion that is distinguished as a non-sky region within a sky region. At least one processor (220) may be configured to identify an error when the hole region (1030) is identified. If an error is identified in operation 903, operation 905 may be performed. In operation 903, if no error is identified, there is no need to correct the estimated first area (310) and second area (320), so the operation can be terminated.

[0113] At operation 905, the instructions, when individually or collectively executed by at least one processor (220), may cause the electronic device (101) to obtain an edge map (e.g., edge map (1002) of FIG. 10B) based on objects within an image.

[0114] According to one embodiment, at least one processor (220) may be configured to obtain an edge map representing edge information within an image to eliminate errors. FIG. 10B illustrates an edge map (1002) for the image illustrated in FIG. 3. Referring to FIG. 10B , the edge map (1002) may be generated by extracting edge information of objects (341, 342, 343, 344, 330). For example, the at least one processor (220) may generate the edge map (1002) by identifying a portion of the image where brightness or color changes abruptly. For example, the at least one processor (220) may extract edge information by differentiating pixel values ​​within the image, or by processing pixel values ​​within the image according to specific criteria, but is not limited thereto. For example, the at least one processor (220) may also extract edge information using deep learning.

[0115] At operation 907, the instructions, when individually or collectively executed by at least one processor (220), may cause the electronic device (101) to remove errors based on the segmentation mask and the edge map.

[0116] According to one embodiment, at least one processor (220) can detect an area corresponding to an identified hole area (1030) within the edge map (1002) and the original image. If edge information is not detected for the hole area (1030), the at least one processor (220) can identify an error for the hole area (1030) and remove the error by removing the hole area (1030). FIG. 10C illustrates a segmentation mask (1003) from which the hole area (1030) of the segmentation mask (1001) illustrated in FIG. 10A has been removed. Referring to FIG. 10C, a corrected segmentation mask (1003) can be generated by removing the hole area (1030). For example, the first region (310) and the second region (320) may be corrected by changing the pixels (1010) assigned with the first label and / or the pixels (1020) assigned with the second label. At least one processor (220) may perform operation 903 again to identify whether an error exists in the corrected first region (310) and the second region (320). If no error is identified in the corrected segmentation mask (1003), the operation may be terminated. If an error is identified in the corrected segmentation mask (1003), operation 905 may be performed again.

[0117] According to one embodiment, the electronic device (101) can accurately distinguish the first region (310) and the second region (320) by estimating the first region (310) and the second region (320) and then correcting the estimated first region (310) and the second region (320) using a segmentation mask and an edge map. If there is an error in distinguishing the first region (310) and the second region (320), even if the image is corrected by enhancing it, the correction for the region where the error exists may be incorrectly performed. At least one processor (220) can accurately enhance the first region (310) by correcting the estimated first region (310) and the second region (320) before correcting the original image.

[0118] FIGS. 11A, 11B, 11C, and 11D illustrate a process of an electronic device according to one embodiment of the present invention for reinforcing a first region.

[0119] According to one embodiment, when the boundary between the first region (310) and the second region (320) is formed in a complex manner, correction may be required for the first region (310) and the second region (320).

[0120] Fig. 11a illustrates an example of an image (1101) stored in a memory (210). Referring to Fig. 11a, the image (1101) may include a first region (310) and a second region (320). For example, the first region (310) may be referred to as a sky region, and the second region (320) may be referred to as a non-sky region. If multiple objects exist on the boundary between the first region (310) and the second region (320), an error may occur in distinguishing between the first region (310) and the second region (320).

[0121] Fig. 11B illustrates a segmentation mask (1102) for the image illustrated in Fig. 11A. Pixels (1110) assigned with a first label may be displayed in white, and pixels (1120) assigned with a second label may be displayed in black. When multiple objects extend into the first area (310), the first area (310) may be incorrectly estimated as the second area (320) due to the multiple objects. For example, when multiple streetlights extending toward the sky (e.g., multiple streetlights (1130) of Fig. 11A) are arranged along a road (1140), the multiple streetlights (1130) visible from a distance may be displayed overlapping the sky, resulting in an error in estimating the first area (310) as the second area (320). According to one embodiment, at least one processor (220) may be configured to identify an error within the segmentation mask (1102).

[0122] FIG. 11C illustrates an edge map (1103) for the image illustrated in FIG. 11A. Referring to FIG. 11C, at least one processor (220) may obtain an edge map by extracting edge information. For example, at least one processor (220) may generate an edge map (1103) by extracting edge information of objects, such as cars, streetlights, roads, and street trees, included in the image. The edge map (1103) may be generated based on edge information of objects included in the image. According to one embodiment, at least one processor (220) may be configured to remove errors based on the segmentation mask (1102) and the edge map (1103). For example, at least one processor (220) can detect an error using a segmentation mask (1102) and an edge map (1103), and remove the error by modifying the segmentation mask (1102) by referring to pixel values ​​and edge information of the original image corresponding to the area where the error occurred.

[0123] FIG. 11D illustrates a segmentation mask (1104) in which errors in the segmentation mask (1102) illustrated in FIG. 11B are removed. Referring to FIG. 11D, a corrected segmentation mask (1104) can be generated by removing errors. For example, the errors can be removed by distinguishing pixels that were incorrectly identified as the second region (320) by the plurality of streetlights (1130) as the first region (310). According to one embodiment, at least one processor (220) can accurately extract the first region (310) by correcting the first region (310) and the second region (320). As the first region (310) is accurately extracted, a corrected image in which the first region (310) is enhanced can be provided by changing the brightness and / or color of the first pixels included in the first region (310).

[0124] FIG. 12 illustrates an example of an electronic device displaying preferred images according to one embodiment.

[0125] Referring to FIG. 12, an electronic device (101) according to one embodiment can distinguish at least one preferred image (1210) for at least one image stored in a memory (e.g., memory (210) of FIG. 2). For example, the electronic device (101) can receive a user input for selecting a preferred image (1210) from among the images stored in the memory (210). The electronic device (101) can store information about at least one preferred image (1210) selected by the user as the preferred image (1210). For example, the user can set a favorite for the preferred image from among the images stored in the memory (210), but is not limited thereto. For example, the electronic device (101) can be configured to display a visual object (1220) representing at least one preferred image (1210) selected as the preferred image (1210).

[0126] According to one embodiment, at least one processor (e.g., at least one processor (220) of FIG. 2) may be configured to generate a corrected image based on at least one preferred image (1210). For example, at least one processor (220) may be configured to set a reference luminance (e.g., the reference luminance (500) of FIG. 5) based on a luminance distribution range of pixels included in an area corresponding to a first area (e.g., the first area (310) of FIG. 3) within the at least one preferred image (1210). For example, at least one processor (220) may estimate information related to an average luminance distribution range for at least one histogram obtained from the at least one preferred image (1210) and set the reference luminance (500) based on the information. For example, a first image (1201), a second image (1202), a third image (1203), and a fourth image (1204) illustrated in FIG. 12 may be stored in the memory (210). If, among the images, the first image (1201), the second image (1202), and the third image (1203) are selected as at least one preferred image (1210) preferred by the user, at least one processor (220) may obtain histograms for each of the first image (1201), the second image (1202), and the third image (1203). The at least one processor (220) may be configured to estimate information related to an average luminance distribution range of the histograms, and set a representative value of the estimated average luminance distribution range as a reference luminance (500). As the reference luminance (500) is set based on at least one preferred image (1210), when moving the luminance distribution range of the first pixels, a corrected image having a luminance distribution range similar to at least one preferred image (1210) can be provided.

[0127] According to one embodiment, at least one processor (220) may be configured to determine the second color based on a luminance deviation of pixels included in an area corresponding to the first region (310) within at least one preferred image (1210). For example, the at least one processor (220) may determine the second color such that a deviation of a histogram for the first region (310) of the corrected image corresponds to a luminance deviation of pixels included in an area corresponding to the first region (310) within the at least one preferred image (1210).

[0128] For example, at least one processor (220) can estimate information related to an average luminance deviation for at least one histogram obtained from at least one preferred image (1210), and set a luminance deviation to be changed based on the information. For example, at least one processor (220) can obtain histograms for each of a first image (1201), a second image (1202), and a third image (1203). At least one processor (220) can estimate information related to an average luminance deviation of the histograms, and determine a second color such that the luminance deviation of the corrected image corresponds to the estimated average luminance deviation. Since the luminance deviation is set based on at least one preferred image (1210), when changing the color of the first pixels, a corrected image having a luminance deviation similar to that of at least one preferred image (1210) can be provided.

[0129] According to one embodiment, at least one processor (220) may set a luminance to be changed from the first luminance based on a luminance distribution range of pixels included in an area corresponding to the first area (310) within at least one preferred image (1210). For example, at least one processor (220) may set a maximum correction value (e.g., a first maximum correction value) of the luminance to be changed from the first luminance. Movement of the luminance distribution range may be limited by the first maximum correction value. Operations of the electronic device (101) according to the first maximum correction value are described below with reference to FIG. 13.

[0130] According to one embodiment, at least one processor (220) may set a maximum correction value (e.g., a second maximum correction value) of a color to be changed from a first color based on a luminance deviation of pixels included in an area corresponding to a first area (310) within at least one preferred image (1210). The stretching of the luminance deviation may be limited by the second maximum correction value. Operations of the electronic device (101) according to the second maximum correction value are described below with reference to FIG. 14.

[0131] According to one embodiment, at least one preferred image (1210) may be an image directly selected by the user, but is not limited thereto. According to one embodiment, at least one processor (220) may distinguish at least one preferred image (1210) using all or part of the images stored in the memory (210). For example, at least one processor (220) may distinguish at least one image including a designated object (330) among the images stored in the memory (210) as at least one preferred image (1210). According to one embodiment, at least one processor (220) may assign different weights to each of the images. For example, at least one processor (220) may assign a relatively high weight to an image corrected by the user and / or an image including a designated object (330), and assign a relatively low weight to the remaining images, thereby obtaining information related to a luminance distribution range and / or luminance deviation.

[0132] FIG. 13 is a flowchart illustrating an example of an operation of an electronic device according to one embodiment of the present invention to change the brightness of first pixels.

[0133] The operations described in FIG. 13 may be operations performed by an electronic device (e.g., an electronic device (101) of FIG. 2) when instructions stored in a memory (e.g., a memory (210) of FIG. 2) are individually or collectively executed by at least one processor (e.g., at least one processor (220) of FIG. 2).

[0134] The operations described in FIG. 13 may be referred to as operations for operation 403 of FIG. 4. According to one embodiment, at least one processor (220) may reduce overcorrection of an original image by changing the luminance by considering a preset first maximum correction value when changing the first luminance of first pixels included in a first region (e.g., the first region (310) of FIG. 3). The first maximum correction value may be set based on at least one preferred image.

[0135] Referring to FIG. 13, at operation 1301, the instructions, when individually or collectively executed by at least one processor (220), may cause the electronic device (101) to compare a difference between the first luminance and the second luminance with a first maximum correction value.

[0136] According to one embodiment, the first maximum correction value may be predetermined. According to one embodiment, the first maximum correction value may be a value preset by the user, but is not limited thereto. According to one embodiment, the first maximum correction value may be set based on at least one preferred image. For example, at least one processor (220) may be configured to identify a luminance distribution range of pixels included in an area corresponding to the first area (310) within at least one preferred image, and set the first maximum correction value based on the luminance distribution range. According to one embodiment, at least one processor (220) may compare a difference between a first luminance of the first pixels within the original image and a second luminance according to a luminance distribution range of the first pixels moved based on a reference luminance (e.g., the reference luminance (500) of FIG. 5) with the first maximum correction value.

[0137] At operation 1303, the instructions, when individually or collectively executed by at least one processor (220), may cause the electronic device (101) to identify whether a difference between the first luminance and the second luminance is less than or equal to a first maximum correction value.

[0138] According to one embodiment, at least one processor (220) may be configured to identify whether the difference is less than or equal to a first maximum correction value or whether the difference exceeds the first maximum correction value. For example, at least one processor (220) may identify a second luminance of the first pixels for shifting a luminance distribution range of the first pixels based on a reference luminance (500). The at least one processor (220) may determine whether the difference is less than or equal to the first maximum correction value by comparing a difference between the second luminance to be changed and the first luminance in the original image with a first maximum correction value. For example, if a range of movement of the first pixels as the luminance of the first pixels changes from the first luminance to the second luminance is within the first maximum correction value, the at least one processor (220) may be configured to identify the difference as less than or equal to the first maximum correction value. If the difference is less than or equal to the first maximum correction value, operation 1305 may be performed. If the above difference exceeds the first maximum correction value, operation 1307 may be performed.

[0139] At operation 1305, the instructions, when individually or collectively executed by at least one processor (220), may cause the electronic device (101) to change the first luminance of the first pixels to a second luminance.

[0140] According to one embodiment, at least one processor (220) may be configured to change the first luminance to the second luminance based on identifying the difference that is less than or equal to a first maximum correction value. If the difference between the second luminance and the first luminance is less than or equal to the first maximum correction value, even if the first luminance of the first pixels is changed to the second luminance, since the difference does not exceed the first maximum correction value, the at least one processor (220) may change the first luminance to the second luminance. As the first luminance is changed to the second luminance, the luminance distribution range of the first pixels may be shifted closer to the reference luminance (500). For example, by shifting the histogram for the luminance of the first pixels closer to the reference luminance (500), the first region (310) may have an ideal luminance or a luminance similar to the ideal luminance.

[0141] At operation 1307, the instructions, when individually or collectively executed by at least one processor (220), may cause the electronic device (101) to change the first luminance of the first pixels based on the first maximum correction value.

[0142] According to one embodiment, at least one processor (220) may be configured to change the first luminance based on the first maximum correction value based on identifying the difference exceeding the first maximum correction value. When the reference luminance (500) and the luminance distribution range of the first pixels are far apart, the difference may exceed the first maximum correction value as the first luminance is changed to the second luminance. If the luminance distribution range of the first pixels is shifted to a range exceeding the first maximum correction value, the original image may be overcorrected, resulting in an unnaturally corrected image. According to one embodiment, at least one processor (220) may shift a histogram for the luminance of the first pixels within the first maximum correction range so as to become closer to the reference luminance (500). As the luminance distribution range of the first pixels is shifted in a limited manner by the first maximum correction value, the first region may have a luminance similar to an ideal luminance. An electronic device (101) according to one embodiment can reduce overcorrection of an original image.

[0143] FIG. 14 is a flowchart illustrating an example of an operation of an electronic device according to one embodiment of the present invention to change the color of first pixels.

[0144] The operations described in FIG. 14 may be operations performed by an electronic device (e.g., an electronic device (101) of FIG. 2) when instructions stored in a memory (e.g., a memory (210) of FIG. 2) are individually or collectively executed by at least one processor (e.g., at least one processor (220) of FIG. 2).

[0145] The operations described in FIG. 14 may be referred to as operations for operation 405 of FIG. 4. According to one embodiment, at least one processor (220) may reduce overcorrection of an original image by changing the color by considering a preset second maximum correction value when changing the first color of the first pixels included in a first region (e.g., the first region (310) of FIG. 3). The second maximum correction value may be set based on at least one preferred image.

[0146] Referring to FIG. 14, at operation 1401, the instructions, when individually or collectively executed by at least one processor (220), may cause the electronic device (101) to compare the difference between the first color and the second color with a second maximum compensation value.

[0147] According to one embodiment, the second maximum correction value may be predetermined. According to one embodiment, the second maximum correction value may be a value preset by the user, but is not limited thereto. According to one embodiment, the first maximum correction value may be set based on at least one preferred image. For example, at least one processor (220) may be configured to identify a luminance deviation of pixels included in an area corresponding to the first area (310) within at least one preferred image, and set the second maximum correction value based on the luminance deviation. According to one embodiment, at least one processor (220) may compare a difference between a first color of the first pixels within the original image and a second color of the first pixels according to histogram stretching with the second maximum correction value.

[0148] At operation 1403, the instructions, when individually or collectively executed by at least one processor (220), may cause the electronic device (101) to identify whether a difference between a first color and a second color is less than or equal to a second maximum compensation value.

[0149] According to one embodiment, at least one processor (220) may be configured to identify whether the difference is less than or equal to a second maximum correction value or whether the difference exceeds the second maximum correction value. For example, at least one processor (220) may identify a second color of the first pixels for increasing a luminance deviation of the first pixels. The at least one processor (220) may determine whether the difference is less than or equal to the second maximum correction value by comparing the difference between the second color to be changed and the first color in the original image with the second maximum correction value. For example, if a histogram stretching range or a range in which RGB channel values ​​of the first pixels are scaled as the color of the first pixels changes from the first color to the second color is within the second maximum correction value, the at least one processor (220) may be configured to identify the difference as less than or equal to the second maximum correction value. If the difference is less than or equal to the second maximum correction value, operation 1405 may be performed. If the above difference exceeds the second maximum correction value, operation 1407 may be performed.

[0150] At operation 1405, the instructions, when individually or collectively executed by at least one processor (220), may cause the electronic device (101) to change the first color of the first pixels to a second color.

[0151] According to one embodiment, at least one processor (220) may be configured to change the first color to the second color based on identifying the difference that is less than or equal to a second maximum compensation value. If the difference between the second color and the first color is less than or equal to the second maximum compensation value, even if the first color of the first pixels is changed to the second color, since the difference does not exceed the second maximum compensation value, the at least one processor (220) may change the first color to the second color. As the first color is changed to the second color, the luminance deviation of the first pixels may increase. For example, as the range in which the histogram of the luminance of the first pixels is distributed increases from a relatively narrow first range (e.g., the first range (610) of FIG. 6A) to a relatively wide second range (e.g., the second range (620) of FIG. 6B), the first region (310) may effectively display contrast with a designated object (e.g., the designated object (330) of FIG. 3). For example, the contrast between clouds and the sky can be effectively expressed by increasing the luminance deviation of the first pixels included within the sky area.

[0152] At operation 1407, the instructions, when individually or collectively executed by at least one processor (220), may cause the electronic device (101) to change the first color of the first pixels based on the second maximum correction value.

[0153] According to one embodiment, at least one processor (220) may be configured to change the first color based on the second maximum compensation value based on identifying the difference exceeding the second maximum compensation value. If the difference between the second color to be changed and the first color before the change exceeds the second maximum compensation value, the original image may be overcorrected as the first color is changed to the second color, resulting in an unnatural appearance of the corrected image. According to one embodiment, at least one processor (220) may change the color of the first pixels such that a histogram for the luminance of the first pixels increases within the second maximum compensation range. As the color of the first pixels is limitedly changed by the second maximum compensation value, the first area (310) may effectively display contrast with respect to a designated object (330). The electronic device (101) according to one embodiment may reduce overcorrection of the original image.

[0154] Figure 15a illustrates a display displaying a first visual object. Figure 15b illustrates a display displaying a second visual object. Figure 15c illustrates a display displaying an original image and a corrected image.

[0155] Referring to FIG. 15A, an electronic device (101) according to one embodiment may include a display (230). The display (230) may be configured to display an image and at least one visual object.

[0156] An electronic device (101) according to one embodiment may be configured to display a first visual object (1510) on a display (230) for receiving a first user input (1541). The first user input (1541) may be a first user input (1541) for requesting generation of a corrected image on the display (230). When a user wishes to obtain a corrected image for an original image, the first user input (1541) may be provided through the first visual object (1510). For example, the first user input (1541) may be provided according to a user's touch input to the first visual object (1510). At least one processor (e.g., at least one processor (220) of FIG. 2) may generate a corrected image based on receiving the first user input (1541) for the first visual object (1510). For example, at least one processor (220) may be configured to generate a corrected image by performing the operations described in FIG. 4 based on receiving a first user input (1541).

[0157] Referring to FIG. 15B, an electronic device (101) according to one embodiment may be configured to display a second visual object (1520) on a display (230). The second visual object (1520) may be a visual object for indicating that an original image is being corrected. For example, the second visual object (1520) may include, but is not limited to, an icon (1521) indicating that the image is being corrected and / or text (1522) such as “remastering.” According to one embodiment, at least one processor (220) may control the display (230) to display the second visual object (1520) while the image is being corrected. A user may recognize that the image is being corrected through the second visual object (1520).

[0158] Referring to FIG. 15C, an electronic device (101) according to one embodiment may be configured to display each of an original image (1501) and a corrected image (1502) on a display (230). According to one embodiment, at least one processor (220) may be configured to display each of the original image (1501) and the corrected image (1502) on the display (230) based on generating the corrected image (1502). For example, a portion of the image may be displayed as the original image (1501), and a remaining portion of the image may be displayed as the corrected image (1502). For example, at least one processor (220) may be configured to display a third visual object (1530) on the display (230) that may adjust a boundary between the original image (1501) and the corrected image (1502). The user can adjust the ratio of the area of ​​the original image (1501) to the area of ​​the corrected image (1502) by adjusting the boundary through the third visual object (1530). For example, when a drag input (1542) is provided for the third visual object (1530) in a first direction (D1) toward the corrected image (1502), the third visual object (1530) can move in the first direction (D1). In this case, the area of ​​the corrected image (1502) can be decreased, and the area of ​​the original image (1501) can be increased. When a drag input (1542) is provided for the third visual object (1530) in a second direction (D2) toward the original image (1501), the third visual object (1530) can move in the second direction (D2). In this case, the area of ​​the corrected image (1502) may increase, and the area of ​​the original image (1501) may decrease. However, the operation of the display (230) displaying each of the original image (1501) and the corrected image (1502) is not limited thereto.Although not shown, at least one processor (220) may display the entire original image (1501) and the entire corrected image (1502) on the display (230). By displaying each of the original image (1501) and the corrected image (1502) on the display (230), the user can intuitively recognize the enhancement effect of the first area (310) within the corrected image (1502).

[0159] Figure 16 illustrates a display that displays a fourth visual object.

[0160] Referring to FIG. 16, an electronic device (101) according to one embodiment may be configured to display a fourth visual object (1610) on a display (230) to receive a second user input (1620) for adjusting the degree of enhancement of a first region (310) of a corrected image.

[0161] According to one embodiment, at least one processor (e.g., at least one processor (220) of FIG. 2) may be configured to display on the display (230) a fourth visual object (1610) for adjusting a second color to be changed from a first color. For example, the fourth visual object (1610) may include an adjustment bar (1612) for selecting a second color to be changed from a first color of the first region (310), or adjusting an extent to which the second color is changed from the first color. For example, the adjustment bar (1612) may determine an extent to which the first color is changed based on a second user input (1620), and the second color may be determined according to the second user input (1620) for the adjustment bar (1612).

[0162] Referring to example (1601) of FIG. 16, at least one processor (220) may be configured to display an original image and a fourth visual object (1610) on a display (230). A user may provide a second user input (1620) to adjust a second color to be changed from a first color through the fourth visual object (1610). Referring to example (1602) of FIG. 16, the fourth visual object (1610) may be changed based on the second user input (1620) for the fourth visual object (1610). For example, the fourth visual object (1610) may include, but is not limited to, an adjustment bar (1612) for indicating a second color selected by the user within a second maximum correction range and text (1611) representing the second color as a number. For example, the number may be referenced as a histogram stretching value or a scaling value of an RGB channel. At least one processor (220) can generate a corrected image based on a histogram stretching value or a scaling value of an RGB channel selected by a user, and display the corrected image through a display (230).

[0163] An electronic device (101) according to one embodiment can change a first color to a second color determined by a user's selection. By adjusting the degree of enhancement for the first region (310) according to the user's selection, a corrected image according to the user's preference can be provided.

[0164] An electronic device (101) is provided. The electronic device (101) may include at least one processor (220). The electronic device (101) may include a memory (210) including one or more storage media storing instructions. The instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to estimate, using a first artificial intelligence model, a first area (310) including a designated object (330) and a second area (320) distinct from the first area (310) from an image stored in the memory (210). The instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to change a first luminance of pixels (e.g., first pixels) included in the first region (310) to a second luminance so as to shift a luminance distribution range of the pixels based on a reference luminance (500). The instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to change a first color of at least one pixel among the pixels to a second color so as to increase a luminance deviation of the pixels. The instructions, when executed individually or collectively by the at least one processor (220), may cause the electronic device (101) to provide a corrected image for the image by changing the color of at least one pixel among the pixels to a third color based on a weighted sum of the first color and the second color.

[0165] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to detect the designated object (330) from the image using the second artificial intelligence model. The instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to provide the corrected image based on the detection of the designated object (330).

[0166] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to detect the designated object (330) based on identifying, using the second artificial intelligence model, that the area ratio of the designated object (330) to the area of ​​the first region (310) is greater than or equal to a designated ratio.

[0167] According to one embodiment, the at least one pixel that is changed to the third color may be in contact with pixels (e.g., second pixels) included in the second area (320) among the pixels.

[0168] According to one embodiment, the weight of the weighted sum may be determined based on a ratio of the number of pixels included in the first area (310) and the number of pixels included in the second area (320) that are included within a specified range from the at least one pixel that is changed to the third color.

[0169] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to obtain a segmentation mask according to the first area (310) and the second area (320) using the first artificial intelligence model. The instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to obtain an edge map based on objects in the image based on identifying an error in the segmentation mask. The instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to correct the estimated first area (310) and the second area (320) by removing the error based on the segmentation mask and the edge map.

[0170] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to compare a difference between the first luminance and the second luminance to a specified first maximum correction value. The instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to change the first luminance to the second luminance based on identifying the difference as being less than or equal to the first maximum correction value. The instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to change the first luminance based on the first maximum correction value based on identifying the difference as being less than or equal to the first maximum correction value.

[0171] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to determine the first maximum correction value based on a luminance distribution range of pixels included in an area corresponding to the first area (310) within at least one preferred image stored in the memory (210).

[0172] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to compare a difference between the first color and the second color to a specified second maximum compensation value. The instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to change the first color to the second color based on identifying the difference as being less than or equal to the second maximum compensation value. The instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to change the first color based on the second maximum compensation value based on identifying the difference as being less than the maximum compensation value.

[0173] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to determine the second maximum correction value based on a luminance deviation of pixels included within an area corresponding to the first area (310) within at least one preferred image stored within the memory (210).

[0174] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to determine the reference luminance (500) based on a luminance distribution range of pixels included in an area corresponding to the first area (310) within at least one preferred image stored in the memory (210). The instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to determine the second color based on a luminance deviation of the pixels included in an area corresponding to the first area (310) within the preferred images.

[0175] According to one embodiment, the electronic device (101) may further include a display (230) for displaying the image. The instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to display a first visual object (1510) on the display (230) for receiving a first user input (1541) for requesting generation of the corrected image. The instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to generate the corrected image based on receiving the first user input (1541) for the first visual object (1510).

[0176] According to one embodiment, the electronic device (101) may further include a display (230) for displaying the image. The instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to display a second visual object (1520) on the display (230) to indicate that the image is being corrected while the corrected image is being generated.

[0177] According to one embodiment, the electronic device (101) may further include a display (230) for displaying the image. The instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to display, on the display (230), each of a portion of the image and a portion of the corrected image based on generating the corrected image. The instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to display a third visual object (1530) for adjusting an area of ​​the portion of the image and the portion of the corrected image.

[0178] According to one embodiment, the electronic device (101) may further include a display (230) for displaying the image. The instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to display a fourth visual object (1610) on the display (230) for receiving a second user input (1620) for adjusting the second color to be changed. The instructions, when individually or collectively executed by the at least one processor (220), may cause the electronic device (101) to determine the second color based on the second user input (1620) for the fourth visual object (1610).

[0179] A method performed by an electronic device (101) including a memory (210) is provided. The method may include an operation of estimating, from an image stored in the memory (210), a first region (310) including a specified object (330) and a second region (320) distinct from the first region (310), using a first artificial intelligence model. The method may include an operation of changing a first luminance of pixels included in the first region (310) to a second luminance in order to shift a luminance distribution range of the pixels included in the first region (310) based on a reference luminance (500). The method may include an operation of changing a first color of at least one of the pixels to a second color so that a luminance deviation of the pixels increases. The method may include an operation of providing a corrected image for the image by changing the color of at least one of the pixels to a third color based on a weighted sum of the first color and the second color.

[0180] According to one embodiment, the method may further include an operation of detecting the designated object (330) from the image using a second artificial intelligence model. The method may further include an operation of providing the corrected image based on the detection of the designated object (330). According to one embodiment, the method may further include an operation of obtaining a segmentation mask according to the first region (310) and the second region (320) using the first artificial intelligence model. The method may further include an operation of obtaining an edge map based on objects in the image based on identifying a hole within the segmentation mask. The method may further include an operation of correcting the estimated first region (310) and the second region (320) by removing the hole based on the segmentation mask and the edge map.

[0181] In one embodiment, the method may further include an operation of comparing a difference between the first luminance and the second luminance with a specified first maximum correction value. The method may further include an operation of changing the first luminance to the second luminance based on identifying the difference as being less than or equal to the first maximum correction value. The method may further include an operation of changing the first luminance based on the first maximum correction value based on identifying the difference as being less than the first maximum correction value.

[0182] In one embodiment, the method may further include comparing a difference between the first color and the second color with a specified second maximum compensation value. The method may further include changing the first color to the second color based on identifying the difference as being less than or equal to the second maximum compensation value. The method may further include changing the first color based on the second maximum compensation value based on identifying the difference as being less than or equal to the maximum compensation value.

[0183] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, electronic devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0184] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0185] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0186] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (120) (e.g., the processor (120)) of a machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0187] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as a memory (130) of a manufacturer's server, an application store's server, or a relay server.

[0188] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In electronic devices, at least one processor; and A memory comprising one or more storage media storing instructions, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Using the first artificial intelligence model, a first region containing a specified object and a second region distinct from the first region are estimated from an image stored in the memory, In order to shift the luminance distribution range of pixels included in the first area based on the reference luminance, the first luminance of the pixels is changed to the second luminance, Changing the first color of at least one pixel among the pixels to a second color so as to increase the luminance deviation of the pixels, causing a corrected image for the image to be provided by changing the color of at least one pixel among the pixels to a third color based on a weighted sum of the first color and the second color; Electronic devices.

2. In paragraph 1, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Using the second artificial intelligence model, the specified object is detected from the image, causing the correction image to be provided based on the detection of the above specified object; Electronic devices.

3. In paragraph 2, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Using the second artificial intelligence model, detecting the designated object based on identifying that the ratio of the area of ​​the designated object to the area of ​​the first region is greater than or equal to the designated ratio. Electronic devices.

4. In any one of paragraphs 1 to 3, At least one pixel that is changed to the third color, Among the above pixels, the pixels that are in contact with the pixels included in the second area, Electronic devices.

5. In any one of paragraphs 1 to 4, The above weighted sum weight is, is determined based on a ratio of the number of pixels included in the first area and the number of pixels included in the second area, which are included within a specified range from the at least one pixel that is changed to the third color. Electronic devices.

6. In any one of paragraphs 1 to 5, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Using the first artificial intelligence model, a segmentation mask according to the first area and the second area is obtained, Within the above segmentation mask, based on identifying errors, an edge map based on objects within the image is obtained, By removing the error based on the segmentation mask and the edge map, the estimated first region and the second region are corrected, Electronic devices.

7. In any one of paragraphs 1 to 6, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Comparing the difference between the first luminance and the second luminance with a specified first maximum correction value, Based on identifying the difference below the first maximum correction value, changing the first luminance to the second luminance, Based on identifying the difference exceeding the first maximum compensation value, causing the first luminance to be changed based on the first maximum compensation value. Electronic devices.

8. In paragraph 7, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Causing to determine the first maximum correction value based on a luminance distribution range of pixels included in an area corresponding to the first area, within at least one preferred image stored in the memory. Electronic devices.

9. In any one of paragraphs 1 to 8, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Compare the difference between the first color and the second color with a specified second maximum compensation value, Based on identifying the difference below the second maximum correction value, changing the first color to the second color, Based on identifying the difference exceeding the maximum compensation value, causing the first color to be changed based on the second maximum compensation value, Electronic devices.

10. In paragraph 9, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Causing to determine the second maximum correction value based on the luminance deviation of pixels included in an area corresponding to the first area, within at least one preferred image stored in the memory. Electronic devices.

11. In any one of paragraphs 1 to 10, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: determining the reference luminance based on the luminance distribution range of pixels included in an area corresponding to the first area, within at least one preferred image stored in the memory; causing the second color to be determined based on the luminance deviation of the pixels included in the area corresponding to the first area within the above preferred images, Electronic devices.

12. In any one of paragraphs 1 to 11, Further comprising a display for displaying the above image, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Displaying a first visual object for receiving a first user input for requesting generation of the corrected image on the display; causing said corrected image to be generated based on receiving said first user input for said first visual object; Electronic devices.

13. In any one of paragraphs 1 to 12, Further comprising a display for displaying the above image, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: while said corrected image is being generated, causing a second visual object to be displayed on said display to indicate that said image is being corrected; Electronic devices.

14. In any one of paragraphs 1 to 13, Further comprising a display for displaying the above image, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Based on generating the above corrected image, displaying each of a part of the image and a part of the corrected image on the display, causing a third visual object to be displayed for adjusting the area of ​​said part of said image and said part of said corrected image; Electronic devices.

15. In any one of paragraphs 1 to 14, Further comprising a display for displaying the above image, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Displaying a fourth visual object on the display for receiving a second user input for adjusting the second color to be changed; causing said second color to be determined based on said second user input for said fourth visual object; Electronic devices.

Images