Electronic device for displaying image and display method using same

Abstract

Provided are an electronic device and method for displaying an image, the electronic device comprising: a display; at least one processor; and a memory for storing one or more instructions, wherein the one or more instructions, when executed individually or collectively by the at least one processor, cause the electronic device to perform operations of: acquiring an on-pixel ratio (OPR) on the basis of input image data; determining a target luminance on the basis of the OPR and a display setting luminance; performing HDR tone mapping on the input image data by using the target luminance; and controlling the display to display an image corresponding to the input image data on which the HDR tone mapping has been performed.

Description

Electronic device for displaying images and display method using the same

[0001] The present disclosure relates to display technology, and more particularly, to an electronic device for displaying an image and a display method using the same.

[0002] High Dynamic Range (HDR) is a technology that displays images with precise contrast, similar to what users actually perceive (e.g., what they see or watch). Compared to Standard Dynamic Range (SDR), HDR supports a wider color gamut and higher luminance range, resulting in more vivid and realistic images.

[0003] HDR images can be classified into HDR10, which applies the same tone mapping to the entire image sequence using static metadata, and HDR10+ or ​​Dolby Vision, which apply different tone mapping to each frame using dynamic metadata.

[0004] Meanwhile, the conventional HDR tone mapping method is mainly performed by relying only on the display's set brightness or maximum brightness value, so it does not reflect the physical characteristics of the display or the actual usage environment, and there is a problem that the brightness and color of the image are distorted differently from the content creator's intention.

[0005] An electronic device and a display method using the same are provided, which determine a target luminance based on a display setting luminance and an on-pixel ratio (OPR), and perform HDR tone mapping based on the on-pixel ratio, thereby minimizing distortion of brightness and color and displaying a high-quality HDR image that represents (e.g., reflects) the intention of a content creator.

[0006] According to one exemplary embodiment, an electronic device includes a display; at least one processor; and a memory storing one or more instructions, wherein the one or more instructions, when individually or collectively executed by the at least one processor, cause the electronic device to perform the following operations: obtaining an on-pixel ratio (OPR) based on input image data; determining a target luminance based on the on-pixel ratio and a display set luminance; performing HDR tone mapping on the input image data using the target luminance; and controlling the display to display an image corresponding to the input image data on which HDR tone mapping has been performed.

[0007] According to one exemplary embodiment, the on-pixel ratio can be obtained based on pixel values ​​of each frame of the input image data.

[0008] According to one exemplary embodiment, the one or more instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to further perform the operation of analyzing pixel values ​​of each frame of the input image data, and obtaining, based on the pixel values, at least one of a ratio of pixels that emit light among all pixels or a degree of emission.

[0009] According to one exemplary embodiment, the one or more instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to further perform an operation of identifying the target luminance as the display set luminance based on the display set luminance being less than or equal to a high brightness mode (HBM) maximum luminance.

[0010] According to one exemplary embodiment, the one or more instructions, when individually or collectively executed by the at least one processor, may further cause the electronic device to perform the operations of comparing the on-pixel ratio with a reference on-pixel ratio, and identifying the target luminance as the display set luminance based on the on-pixel ratio being equal to the reference on-pixel ratio.

[0011] According to one exemplary embodiment, the one or more instructions, when individually or collectively executed by the at least one processor, may further cause the electronic device to perform the operations of comparing the on-pixel ratio to a reference on-pixel ratio, and setting the target luminance to be lower than the display set luminance based on the on-pixel ratio being greater than the reference on-pixel ratio.

[0012] According to one exemplary embodiment, the one or more instructions, when individually or collectively executed by the at least one processor, may further cause the electronic device to perform the operations of comparing the on-pixel ratio to a reference on-pixel ratio, and setting the target luminance higher than the display set luminance based on the on-pixel ratio being less than the reference on-pixel ratio.

[0013] According to one exemplary embodiment, the one or more instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to further perform the operations of obtaining illuminance data including illuminance of a usage environment, and determining the target luminance based on the on-pixel ratio, the display set luminance, and the illuminance data.

[0014] According to one exemplary embodiment, the one or more instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to further perform the operations of analyzing a target luminance of each frame of the input image data, and compensating for the target luminance based on a first compensation method using an infinite impulse response (IIR) filter.

[0015] According to one exemplary embodiment, the one or more instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to further perform the operations of analyzing a target luminance of each frame of the input image data, and compensating for the target luminance based on a second compensation method that utilizes an average value of the target luminance of each frame of the input image data.

[0016] According to one exemplary embodiment, the one or more instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to further perform an operation of analyzing a target luminance of each frame of the input image data, and an operation of compensating for the target luminance based on a third compensation method including an incremental change of the target luminance based on a reference target luminance.

[0017] According to one exemplary embodiment, the one or more instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to further perform the operations of obtaining a tone mapping curve corresponding to the input image data, adjusting the tone mapping curve based on the on-pixel ratio, and performing HDR tone mapping for the target luminance using the adjusted tone mapping curve.

[0018] According to one exemplary embodiment, the one or more instructions, when individually or collectively executed by the at least one processor, may further cause the electronic device to perform an operation of rendering an image in which a loading effect characteristic is reflected based on the on-pixel ratio through the HDR tone mapping, and an operation of displaying the rendered image on the display.

[0019] According to one exemplary embodiment, the one or more instructions, when individually or collectively executed by the at least one processor, may further cause the electronic device to perform an operation of determining the target luminance based on the on-pixel ratio, the display set luminance, and a characteristic of the display.

[0020] According to one exemplary embodiment, a method for displaying an image may include: obtaining an on-pixel ratio (OPR) based on input image data; determining a target luminance based on the on-pixel ratio and a display setting luminance; performing HDR tone mapping on the input image data using the target luminance; and displaying an image corresponding to the input image data on which HDR tone mapping has been performed.

[0021] According to one exemplary embodiment, the on-pixel ratio can be obtained based on pixel values ​​of each frame of the input image data.

[0022] According to one exemplary embodiment, the operation of determining the target luminance may include an operation of identifying the target luminance as the display set luminance based on the display set luminance being less than or equal to a high brightness mode (HBM) maximum luminance.

[0023] According to one exemplary embodiment, the operation of determining the target luminance may include an operation of comparing the on-pixel ratio with a reference on-pixel ratio, and an operation of identifying the target luminance as the display setting luminance based on the on-pixel ratio and the reference on-pixel ratio being equal.

[0024] According to one exemplary embodiment, the operation of determining the target luminance may include an operation of comparing the on-pixel ratio with a reference on-pixel ratio, and an operation of setting the target luminance to be lower than the display set luminance based on the on-pixel ratio being greater than the reference on-pixel ratio.

[0025] According to one exemplary embodiment, the operation of determining the target luminance may include an operation of comparing the on-pixel ratio with a reference on-pixel ratio, and an operation of setting the target luminance higher than the display set luminance based on the on-pixel ratio being less than the reference on-pixel ratio.

[0026] According to one exemplary embodiment, a non-transitory computer-readable medium may store instructions that, when executed by at least one processor, cause the at least one processor to execute a method for displaying an image, the method including: obtaining an on-pixel ratio (OPR) based on input image data; determining a target luminance based on the on-pixel ratio and a display setting luminance; performing HDR tone mapping on the input image data using the target luminance; and controlling the display to display an image corresponding to the input image data on which HDR tone mapping has been performed.

[0027] The effects that can be obtained from the exemplary embodiments of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be derived and understood by those skilled in the art to which the present disclosure pertains from the following description. In other words, unintended effects resulting from implementing the exemplary embodiments of the present disclosure can also be derived by those skilled in the art from the exemplary embodiments of the present disclosure.

[0028] The above and other aspects and features of specific embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0029] FIG. 1 is a block diagram of an exemplary electronic device within a network environment, according to one or more embodiments.

[0030] FIG. 2 is a block diagram illustrating an exemplary configuration of an electronic device according to one or more embodiments.

[0031] FIG. 3 is a flowchart illustrating exemplary operation of an electronic device according to one or more embodiments.

[0032] FIG. 4 is a conceptual diagram illustrating an exemplary operation of an electronic device according to one or more embodiments to determine a target luminance.

[0033] FIG. 5 is a flowchart illustrating an exemplary operation of an electronic device according to one or more embodiments to determine a target luminance.

[0034] FIG. 6 is a diagram showing Equations 1 and 2 used by an electronic device according to one or more embodiments to calculate target luminance.

[0035] FIG. 7 is a diagram illustrating exemplary on-pixel ratios and display setting luminance according to one or more embodiments.

[0036] FIG. 8 is a conceptual diagram illustrating an exemplary operation of an electronic device according to one or more embodiments to determine a target luminance.

[0037] FIG. 9 is a conceptual diagram illustrating an exemplary operation of an electronic device according to one or more embodiments to compensate for target luminance.

[0038] FIGS. 10A, 10B, and 10C are diagrams illustrating various compensation methods by which an electronic device according to one or more embodiments compensates for target luminance.

[0039] FIG. 11 is a flowchart illustrating an exemplary operation of an electronic device performing HDR tone mapping according to one or more embodiments.

[0040] FIGS. 12a, 12b, and 12c are diagrams illustrating exemplary tone mapping curves used for HDR tone mapping by an electronic device according to one or more embodiments.

[0041] Hereinafter, one or more embodiments of this document will be described in detail with reference to the drawings so that those skilled in the art can practice them. However, this document may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used to refer to the same or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0042] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100), according to one or more embodiments.

[0043] In FIG. 1, in a network environment (100), an electronic device (101) can communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or can communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) can communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0044] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or a secondary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor)) that can operate independently or together therewith. For example, if the electronic device (101) includes a main processor (121) and a secondary processor (123), the secondary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a specified function. The secondary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0045] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0046] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0047] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0048] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0049] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0050] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0051] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0052] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0053] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0054] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0055] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0056] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0057] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).

[0058] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0059] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as a plurality of separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0060] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0061] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0062] According to one or more embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0063] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0064] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service on its own, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0065] FIG. 2 is a block diagram illustrating an exemplary configuration of an electronic device (101) according to one or more embodiments.

[0066] In FIG. 2, the electronic device (101) may include a memory (210), a communication interface (220), a display (230), and / or a processor (240).

[0067] The memory (210) can store various programs, data, and / or commands used in the electronic device (101). In addition, the memory (210) can store various pieces of information according to one or more embodiments of the present disclosure.

[0068] In one embodiment, the memory (210) may be implemented as an internal memory such as a ROM (e.g., an electrically erasable programmable read-only memory (EEPROM)) or a RAM included in at least one processor (240), or may be implemented as a separate memory from at least one processor (240). In this case, the memory (210) may be implemented as a memory embedded in the electronic device (101) or as a memory detachable from the electronic device (101) depending on the purpose of data storage. For example, data for driving the electronic device (101) may be stored in a memory embedded in the electronic device (101), and data for expanding functions of the electronic device (101) may be stored in a memory detachable from the electronic device (101).

[0069] The processor (240) (including the processor (120) of FIG. 1) may include various processing circuits and / or multiple processors. For example, the term "processor" as used in the present disclosure, including the claims, may include various processing circuits including at least one processor, wherein at least one or more of the processors may be configured to individually and / or collectively perform the various functions described in the present disclosure in a distributed manner. When "processor," "at least one processor," and "one or more processors" as used in the present disclosure are described as being configured to perform multiple functions, these terms include, but are not limited to, situations where one processor performs some of the recited functions and other processor(s) perform other parts of the recited functions, and situations where a single processor may perform all of the recited functions. Furthermore, the at least one processor may include a combination of processors that perform the various functions recited / disclosed, for example, in a distributed manner. At least one processor may execute program instructions to achieve or perform various functions.

[0070] In the case of memory embedded in the electronic device (101), it may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD)), and in the case of memory that can be detachably attached to the electronic device (101), it may be implemented in the form of a memory card (e.g., compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), extreme digital (xD), multi-media card (MMC)), or external memory that can be connected to a USB port (e.g., USB memory).

[0071] The communication interface (220) may be a component configured to allow the electronic device (101) to communicate with external devices such as an external source device or an image output device (200). The communication interface (220) may include at least one wireless communication module and / or at least one wired communication module. Each communication module may be implemented in the form of at least one hardware chip. The wireless communication module may include at least one module among a Wi-Fi module, a Bluetooth module, an infrared communication module, or other communication modules. In addition, the communication interface may include a module of Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4 th Generation), and / or 5G (5 th It may include at least one communication chip that performs communication according to various wireless communication standards such as Wireless Generation (WG).

[0072] The wired communication module may include, for example, at least one of a LAN module, an Ethernet module, a paired cable, a coaxial cable, a fiber optic cable, or an Ultra Wide-Band (UWB) module. The communication interface (220) may be implemented in various forms as described above, and may transmit and receive various signals by communicating with external devices.

[0073] The display (230) refers to a component for displaying various contents. The display (230) may be implemented as a display including a self-luminous element or a display including a non-luminous element and a backlight. For example, the display may be implemented as various types of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, an LED (Light Emitting Diodes), a micro LED, a Mini LED, a PDP (Plasma Display Panel), a QD (Quantum dot) display, and / or a QLED (Quantum dot light-emitting diodes). The display (230) may include a driving circuit and / or a backlight unit that may be implemented in a form such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, and / or an OTFT (organic TFT).

[0074] The display (230) may be implemented as a touch screen combined with a touch sensor, a flexible display, a rollable display, a 3D display, and / or a display in which multiple displays are physically connected.

[0075] At least one processor (240) controls the overall operation of the electronic device (101). Specifically, at least one processor (240) may be connected to each component of the electronic device (101) to control the overall operation of the electronic device (101). For example, at least one processor (240) may be operatively connected to a memory (210), a communication interface (220), and / or a display (230).

[0076] The processor (240) may be composed of one or more processors. At least one processor (240) may perform an operation of the electronic device (101) according to one or more embodiments by executing at least one instruction stored in the memory (210). The at least one processor (240) may include one or more of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an APU (Accelerated Processing Unit), a MIC (Many Integrated Core), a DSP (Digital Signal Processor), an NPU (Neural Processing Unit), a hardware accelerator, or a machine learning accelerator. The at least one processor (240) may control one or any combination of other components of the electronic device (101) and perform operations related to communication or data processing. The at least one processor (240) may individually or collectively execute one or more programs or instructions stored in the memory. For example, the at least one processor may perform a method according to one or more embodiments by executing one or more instructions stored in the memory.

[0077] According to one or more embodiments, when a method includes a plurality of operations, the plurality of operations may be performed by one processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to one or more embodiments, the first operation, the second operation, and the third operation may all be performed by a first processor, or the first operation and the second operation may be performed by a first processor (e.g., a general-purpose processor) and the third operation may be performed by a second processor (e.g., an artificial intelligence-specific processor).

[0078] At least one processor (240) may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When at least one processor (240) is implemented as a multicore processor, each of the multiple cores included in the multicore processor may include an internal processor memory, such as a cache memory or an on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to one or more embodiments, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to one or more embodiments.

[0079] When a method according to one or more embodiments includes a plurality of operations, the plurality of operations may be performed by one core among the plurality of cores included in the multi-core processor, or may be performed by the plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to one or more embodiments, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0080] In one or more embodiments of the present disclosure, the term processor may mean a system on a chip (SoC) in which at least one processor and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, and / or a machine learning accelerator, but the embodiments are not limited thereto. Hereinafter, for convenience of description, at least one processor (240) will be referred to as a processor (240).

[0081] The processor (240) determines a target luminance by considering the on-pixel ratio (OPR) together with the display setting luminance, and performs HDR tone mapping based on the on-pixel ratio, thereby minimizing distortion of brightness and color and displaying a high-quality HDR image that meets the intention of the content creator. For example, the processor (240) may calculate an on-pixel ratio based on input image data, determine a target luminance based on the on-pixel ratio and the display setting luminance, perform HDR tone mapping using the target luminance, and display an image on which the HDR tone mapping has been performed.

[0082] FIG. 3 is a flowchart illustrating an exemplary operation of an electronic device (101) according to one embodiment(s) of the present disclosure.

[0083] In FIG. 3, the electronic device (101) can determine a target luminance based on an on-pixel ratio together with a display setting luminance, and perform HDR tone mapping based on the on-pixel ratio. For example, the electronic device (101) can obtain an on-pixel ratio based on input image data (operation 310), determine a target luminance based on the on-pixel ratio and the display setting luminance (operation 320), perform HDR tone mapping using the target luminance (operation 330), and display an image on which the HDR tone mapping has been performed (operation 340).

[0084] According to one or more embodiments, in operation 310, the electronic device (101) may obtain an on-pixel ratio based on input image data. The electronic device (101) may calculate the on-pixel ratio based on the input image data. For example, the electronic device (101) may analyze pixel values ​​for each frame of the input image data and calculate the ratio of emitting pixels among all pixels based on the pixel values.

[0085] The electronic device (101) can analyze the pixel values ​​of each frame of input image data to determine whether each pixel is in a light-emitting state. For example, the electronic device (101) can determine whether the pixel is in a light-emitting state based on whether the luminance value of the pixel is above a certain threshold. After determining whether each pixel in the frame is in light-emitting state, the electronic device (101) can count the number of pixels that are in light-emitting state.

[0086] The electronic device (101) can calculate the total number of pixels in each frame and divide the number of light-emitting pixels by the total number of pixels to obtain the ratio of light-emitting pixels, i.e., the on-pixel ratio (OPR). For example, the electronic device (101) can obtain an independent OPR value for each frame by repeatedly performing the above operation for each frame of input image data.

[0087] According to one embodiment, the electronic device can calculate an OPR value for the frame (or image) based on a ratio of the sum of the maximum RGB (red, green, and blue) values ​​of each pixel of the frame (or image) to the maximum luminance of the frame (or image).

[0088] Additionally, the electronic device (101) can calculate a representative OPR value for multiple frames by calculating an average value for the OPR values ​​calculated from multiple frames. For example, the electronic device (101) can determine a target luminance using the representative OPR value. For example, the electronic device (101) can adjust a tone mapping curve for performing HDR tone mapping using the representative OPR value.

[0089] According to one or more embodiments, in operation 320, the electronic device (101) may determine a target luminance based on the on-pixel ratio and the display set luminance. Hereinafter, operation 320 of the electronic device (101) will be described in detail with reference to FIGS. 4 to 10c.

[0090] According to one or more embodiments, in operation 330, the electronic device (101) may generate an HDR tone mapped image by performing HDR tone mapping using the target luminance. Hereinafter, operation 330 of the electronic device (101) will be described in detail with reference to FIGS. 11 to 12c.

[0091] According to one or more embodiments, in operation 340, the electronic device (101) may display an image on which HDR tone mapping has been performed (e.g., an HDR tone-mapped image). For example, the electronic device (101) may display an image with reduced color distortion. The electronic device (101) may render the image by considering a loading effect characteristic based on an on-pixel ratio through the HDR tone mapping.

[0092] According to one or more embodiments, the electronic device (101) may analyze the on-pixel ratio and the power consumption characteristics of the display panel during the rendering process to compensate for a brightness decrease that may occur due to a loading effect. The loading effect is a phenomenon in which the average brightness of the entire screen decreases as the ratio of light-emitting pixels increases, and may occur in an AMOLED display. The electronic device (101) may determine the final target brightness of each pixel by considering the on-pixel ratio, thereby matching the brightness of the displayed image to the set target brightness. The electronic device (101) may render the image by considering the loading effect that may occur during the HDR tone mapping process, thereby preventing color distortion or brightness decrease due to the physical limitations of the display, and providing a more natural and accurate image.

[0093] FIG. 4 is a conceptual diagram illustrating an exemplary operation of an electronic device (101) according to one or more embodiments to determine a target luminance, FIG. 5 is a flowchart illustrating an exemplary operation of an electronic device (101) according to one or more embodiments to determine a target luminance, FIG. 6 is a diagram illustrating Equations 1 and 2 used by an electronic device (101) according to one or more embodiments to calculate a target luminance, and FIG. 7 is a diagram illustrating an exemplary on-pixel ratio and display setting luminance according to one or more embodiments.

[0094] The display setting brightness may be a brightness level set by the user or a default (e.g., predetermined) brightness level set by the electronic device (101). For example, the display setting brightness may be a maximum brightness target value for the electronic device (101) when playing content from input image data. For example, the display setting brightness may be set in units of nits (candelas per square meter).

[0095] In FIG. 4, the electronic device (101) can determine the maximum brightness that serves as a reference for HDR tone mapping using the display setting brightness and the on-pixel ratio. For example, the electronic device (101) can determine the maximum brightness to output content according to the input image data based on the display setting brightness and the on-pixel ratio. For example, the electronic device (101) can calculate the target brightness from the display setting brightness and the on-pixel ratio using the target brightness calculation unit (400).

[0096] As shown in FIG. 5, the electronic device (101) can determine (operation 510) whether the display set brightness is greater than the HBM maximum brightness. Here, the HBM maximum brightness may represent the maximum brightness that the electronic device (101) can output in HBM (High Brightness Mode). For example, HBM may be a mode that temporarily maximizes the brightness of the display to increase the visibility of the display in an environment with strong external lighting, such as under strong sunlight. For example, the HBM maximum brightness may refer to the maximum brightness limit that the hardware of the display can physically support so that the display can clearly display content in a specific environment.

[0097] The electronic device (101) may determine the target luminance as the display set luminance when the display set luminance is less than or equal to the HBM maximum luminance. For example, when the display set luminance is less than or equal to the HBM maximum luminance, the actual maximum luminance may be equal to the display set luminance regardless of the on-pixel ratio. For example, as shown in FIG. 7, when the display set luminance is less than or equal to the HBM maximum luminance (e.g., 1000 nit) (e.g., 500 nit, 1000 nit), the actual maximum luminance may be equal to the display set luminance regardless of the on-pixel ratio.

[0098] The electronic device (101) may compare the on-pixel ratio with a reference OPR (operations 520 and 530). Here, the reference OPR may refer to an on-pixel ratio when the actual maximum luminance matches the display setting luminance. For example, the reference OPR may represent an on-pixel ratio under the condition that the actual maximum luminance maintains the same brightness as the display setting luminance regardless of the luminance change of the display setting luminance. For example, the reference OPR may be a reference point that can exhibit optimal brightness performance in HBM mode. For example, when the on-pixel ratio is larger (e.g., greater) or smaller (e.g., less) than the reference OPR, the actual maximum luminance may be different from the display setting luminance. For example, as shown in FIG. 6, when the on-pixel ratio is larger or smaller than the reference OPR, the electronic device (101) may calculate the target luminance by applying Equation 1 or Equation 2. For example, as shown in FIG. 7, the reference OPR may be the on-pixel ratio (e.g., 25%) at which the actual maximum luminance remains the same brightness as the display set luminance regardless of changes in the display set luminance.

[0099] In one embodiment, the electronic device (101) may determine the target luminance as the display setting luminance when the on-pixel ratio and the reference OPR are equal. For example, when the on-pixel ratio and the reference OPR are equal, the actual maximum luminance may be equal to the display setting luminance.

[0100] In one embodiment, the electronic device (101) can calculate the target luminance using Equation 1 below when the on-pixel ratio is greater than the reference OPR.

[0101] Formula 1

[0102]

[0103] Here, Y is the target luminance, HBMmax is the HBM maximum luminance, X is the display setting luminance, and OPR is the on-pixel ratio.

[0104] For example, as shown in FIG. 7, when the on-pixel ratio is greater than the reference OPR, the actual maximum luminance may be smaller (e.g., less) than the display setting luminance. Accordingly, when the on-pixel ratio is greater than the reference OPR, the electronic device (101) may apply Equation 1 to calculate a target luminance that is smaller (e.g., less) than the display setting luminance by reflecting the on-pixel ratio.

[0105] In one embodiment, the electronic device (101) may cause the target luminance to be calculated using Equation 2 below when the on-pixel ratio is smaller (e.g., less) than the reference OPR.

[0106] Formula 2

[0107]

[0108] Here, Y is the target luminance, X is the display setting luminance, OPR is the on-pixel ratio, and α is the luminance correction parameter.

[0109] For example, as shown in FIG. 7, when the on-pixel ratio is smaller (e.g., less) than the reference OPR, the actual maximum luminance may be greater than the display setting luminance. Accordingly, when the on-pixel ratio is smaller (e.g., less) than the reference OPR, the electronic device (101) can apply Equation 2 to calculate a target luminance that is greater than the display setting luminance by reflecting the on-pixel ratio (e.g., taking the on-pixel ratio into consideration).

[0110] FIG. 8 is a conceptual diagram illustrating an exemplary operation of an electronic device (101) according to one or more embodiments to determine target luminance.

[0111] In FIG. 8, the electronic device (101) can receive or acquire illuminance data including illuminance of the usage environment and determine target luminance using the illuminance data. For example, as shown in FIG. 8, the electronic device (101) can determine the target luminance by additionally reflecting the illuminance data to the on-pixel ratio and the display setting luminance. For example, the electronic device (101) can calculate the target luminance from the display setting luminance, the on-pixel ratio, and the illuminance data using the target luminance calculation unit (800).

[0112] For example, the electronic device (101) can collect in real time the illuminance of the usage environment in which the electronic device (101) is used through a light sensor. The electronic device (101) can generate illuminance data of the usage environment based on the collected illuminance.

[0113] The electronic device (101) can analyze illuminance data and determine a target luminance based on the illuminance data. The electronic device (101) can adjust the target luminance based on the illuminance data. For example, when the ambient illuminance is high (e.g., under strong sunlight), the electronic device (101) can reduce the target luminance. In this case, the electronic device (101) can perform HDR tone mapping based on the reduced target luminance, thereby providing an overall upward tone mapping value for the entire image, thereby making the image appear brighter. Conversely, when the ambient illuminance is low (e.g., in a dark indoor environment), the target luminance can be increased to reduce unnecessary power consumption.

[0114] In one or more embodiments, the electronic device (101) can calculate an optimal target luminance based on illuminance data, a previously calculated on-pixel ratio (OPR), and a display set luminance. For example, the electronic device (101) can adjust the target luminance by applying a luminance correction factor based on illuminance calculated through a predefined table or algorithm.

[0115] FIG. 9 is a conceptual diagram illustrating an exemplary operation of an electronic device (101) according to one or more embodiments to compensate for target luminance, and FIGS. 10a to 10c are diagrams illustrating various compensation methods of an electronic device (101) according to one or more embodiments to compensate for target luminance.

[0116] In FIG. 9, the electronic device (101) can determine a target luminance using the display setting luminance and the on-pixel ratio, and compensate for the target luminance using at least one compensation formula. For example, the electronic device (101) can obtain (e.g., calculate or determine) the target luminance from the display setting luminance and the on-pixel ratio using the target luminance calculation unit (910). For example, the electronic device (101) can compensate for the target luminance using the target luminance compensation unit (920), thereby outputting the compensated target luminance.

[0117] In one embodiment, the electronic device (101) may compensate for target luminance using an IIR filter. For example, as shown in FIG. 10A, the electronic device (101) may analyze the target luminance for each frame of the input image data and compensate for the target luminance using the compensation equation 1 below.

[0118] Compensation 1

[0119]

[0120] Here, x[n] is the input target luminance of the current frame, and y[n] is the final target luminance of the current frame. x[nx] is the input target luminance of the previous frame, and Y[nx] is the final target luminance of the previous frame. N is the frame window to use, and M is the number of previous frames to use. b_x is the feedforward coefficient, and a_x is the feedback coefficient.

[0121] For example, the electronic device (101) can provide a change in target luminance using an IIR filter. For example, the electronic device (101) can reduce flickering caused by sudden luminance changes by calculating the final target luminance by considering not only the target luminance of the current frame but also the target luminance of the previous frame.

[0122] In one embodiment, the electronic device (101) can compensate for the target luminance using the average value of the target luminance of multiple frames. For example, as shown in FIG. 10b, the electronic device (101) can analyze the target luminance for each frame of the input image data and compensate for the target luminance using the compensation formula 2 below.

[0123] Compensation 2

[0124]

[0125] Here, y'[n] is the final target luminance of the current frame, y[nx] is the final target luminance of the previous frame, and N is the frame window to use.

[0126] For example, the electronic device (101) can determine the final target luminance by averaging the target luminance values ​​of a predetermined number of frames (N). By determining the final target luminance as the average value of the target luminance of multiple frames, the electronic device (101) can process (e.g., alleviate) rapid changes in luminance and minimize flickering in images with frequent brightness fluctuations.

[0127] In one embodiment, the electronic device (101) may compensate for the target luminance so that the change in target luminance changes incrementally (e.g., gradually). For example, as shown in FIG. 10c, the electronic device (101) may analyze the target luminance for each frame of the input image data and compensate for the target luminance using the compensation equation 3 below.

[0128] Compensation 3

[0129]

[0130] Here, y'[n] is the final target luminance of the current frame. Base is the reference target luminance. For example, Base can be the final target luminance of the previous frame (y[n-1]). For example, Base can be the average of the target luminances of multiple frames up to the previous frame. gradual_increase can be the luminance value to be gradually increased, and gradual_decrease can be the luminance value to be gradually decreased.

[0131] For example, if the difference between the current target luminance and the previous target luminance exceeds a threshold value set in the reference target luminance (Base), the electronic device (101) may adjust the display luminance in a way that gradually (e.g., gradually) increases or decreases the luminance instead of immediately reflecting the luminance change. By gradually (e.g., gradually) adjusting the target luminance through compensation, the electronic device (101) may prevent flickering that may be felt by the user. The electronic device (101) may compensate the target luminance so that the target luminance changes gradually (e.g., gradually).

[0132] FIG. 11 is a flowchart illustrating an exemplary operation of an electronic device (101) performing HDR tone mapping according to one or more embodiments, and FIGS. 12a to 12c are diagrams illustrating exemplary tone mapping curves used for HDR tone mapping by an electronic device (101) according to one or more embodiments.

[0133] In FIG. 11, the electronic device (101) can obtain a tone mapping curve corresponding to the input image data (operation 1110), adjust the tone mapping curve based on the on-pixel ratio (operation 1120), and perform HDR tone mapping for the target luminance using the adjusted tone mapping curve (operation 1130).

[0134] According to one or more embodiments, in operation 1110, the electronic device (101) may obtain an initial tone mapping curve corresponding to the input image data. For example, the initial tone mapping curve may be set based on the set luminance of the display and the dynamic range of the input image data. For example, the initial tone mapping curve may be set by adjusting the relationship between the Optical Electrical Transfer Function (OETF) and the Electrical Optical Transfer Function (EOTF).

[0135] According to one or more embodiments, in operation 1120, the electronic device (101) can adjust the initial tone mapping curve based on the calculated on-pixel ratio. For example, as shown in FIG. 12A, if the on-pixel ratio is equal to the reference OPR, the electronic device (101) can use the first tone mapping curve, in which the initial tone mapping curve remains as is. For example, if the set luminance of the display is equal to the HBM maximum luminance, or if there is no need to change the tone mapping curve due to the physical characteristics of the display, the electronic device (101) can perform HDR tone mapping for the target luminance using the first tone mapping curve.

[0136] For example, as shown in FIG. 12B, when the on-pixel ratio is greater than the reference OPR, the electronic device (101) can adjust the initial tone mapping curve to the second tone mapping curve. For example, when the on-pixel ratio is greater than the reference OPR, the second tone mapping curve can be adjusted in a direction to lower the brightness of the image by a predetermined ratio. For example, when the on-pixel ratio is greater than the reference OPR, since the ratio of emitting pixels is high and the actual brightness of the display is likely to be lower than the set brightness, the electronic device (101) can perform HDR tone mapping optimized for a high on-pixel ratio using the second tone mapping curve.

[0137] For example, as shown in FIG. 12C, when the on-pixel ratio is smaller (or less) than the reference OPR, the electronic device (101) can adjust the initial tone mapping curve to the third tone mapping curve. For example, when the on-pixel ratio is smaller (or less) than the reference OPR, the third tone mapping curve can be adjusted in a direction to increase the brightness of the image by a predetermined ratio. For example, when the on-pixel ratio is smaller (or less) than the reference OPR, the ratio of emitting pixels is low, so that the actual brightness of the display is likely to be higher than the set brightness, and therefore, the electronic device (101) can perform HDR tone mapping optimized for a low on-pixel ratio using the third tone mapping curve.

[0138] According to one or more embodiments, at operation 1130, the electronic device (101) may perform final HDR tone mapping using the adjusted tone mapping curve. For example, the electronic device (101) may adjust the brightness and color of the image to match the target luminance by illuminating each pixel of the input image data using the adjusted tone mapping curve.

[0139] In this way, the electronic device (101) of the present disclosure can determine the target luminance by considering the on-pixel ratio together with the display setting luminance, and perform HDR tone mapping based on the on-pixel ratio.

[0140] Accordingly, the electronic device (101) of the present disclosure can minimize distortion of brightness and color in a self-luminous display such as AMOLED and display high-quality HDR images that meet the intention of the content creator.

[0141] However, since this has been described above, a duplicate explanation will be omitted.

[0142] Electronic devices according to one or more embodiments disclosed herein may take various forms. The electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. In one embodiment, the electronic devices are not limited to the aforementioned devices.

[0143] It should be understood that one or more embodiments of the present disclosure and terminology used therein are not intended to limit the technical features described in this document to the specific embodiments, but rather to encompass various modifications, equivalents, or alternatives of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly dictates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among the phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0144] The term "module" as used herein may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integrally formed component, or a minimum unit or portion of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0145] One or more of the embodiments described above in the present disclosure may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0146] According to one embodiment, a method according to one or more embodiments disclosed in the present disclosure may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smart phones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0147] According to one or more embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to one or more embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In this case, according to one or more embodiments, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one or more embodiments, the operations performed by a module, program, or other component may be performed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0148] In one embodiment, an electronic device may include a display, at least one processor, and a memory storing instructions. The instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to calculate an on-pixel ratio (OPR) based on input image data, determine a target luminance based on the OPR and a display setting luminance, perform HDR tone mapping on the input image data using the target luminance, and display an image corresponding to the input image data on which the HDR tone mapping has been performed so as to control the display.

[0149] In one embodiment, the on-pixel ratio can be calculated based on pixel values ​​of each frame of the input image data.

[0150] In one embodiment, the instructions may cause the electronic device to analyze pixel values ​​for each frame of the input image data and, based on the pixel values, calculate at least one of a ratio of pixels that emit light among all pixels or a degree of emission.

[0151] In one embodiment, the instructions may cause the electronic device to determine whether a display set luminance is greater than an HBM maximum luminance, and if the display set luminance is less than or equal to the HBM maximum luminance, determine the target luminance as the display set luminance.

[0152] In one embodiment, the instructions may cause the electronic device to compare the on-pixel ratio with a reference OPR, and if the on-pixel ratio and the reference OPR are equal, determine the target luminance as the display set luminance.

[0153] In one embodiment, the instructions may cause the electronic device to compare the on-pixel ratio with a reference OPR, and if the on-pixel ratio is greater than the reference OPR, set the target luminance to be lower than the display set luminance based on the on-pixel ratio and the HBM maximum luminance. In one embodiment, the instructions may cause the electronic device to compare the on-pixel ratio with a reference OPR, and if the on-pixel ratio is greater than the reference OPR, calculate the target luminance using Equation 1 below.

[0154] Formula 1

[0155]

[0156] In one embodiment, the instructions may cause the electronic device to compare the on-pixel ratio with a reference OPR, and if the on-pixel ratio is less than the reference OPR, set the target luminance higher than the display set luminance based on the on-pixel ratio.

[0157] In one embodiment, the instructions may cause the electronic device to compare the on-pixel ratio with a reference OPR, and if the on-pixel ratio is less than the reference OPR, calculate the target luminance using Equation 2 below.

[0158] Formula 2

[0159]

[0160] In one embodiment, the instructions may cause the electronic device to receive illuminance data including illuminance of a usage environment, and to additionally reflect the illuminance data in the on-pixel ratio and the display set luminance to determine the target luminance.

[0161] In one embodiment, the commands may cause the electronic device to analyze the target luminance for each frame of the input image data and compensate for the target luminance according to a first compensation method using an IIR filter.

[0162] In one embodiment, the commands may cause the electronic device to analyze the target luminance for each frame of the input image data and compensate for the target luminance using the compensation equation 1 below.

[0163] Compensation 1

[0164]

[0165] In one embodiment, the commands may cause the electronic device to analyze the target luminance for each frame of the input image data and compensate for the target luminance according to a first compensation method using an IIR filter.

[0166] In one embodiment, the commands may cause the electronic device to analyze the target luminance for each frame of the input image data and compensate for the target luminance according to a second compensation method that uses an average value of the target luminance of a plurality of frames.

[0167] In one embodiment, the commands may cause the electronic device to analyze the target luminance for each frame of the input image data and compensate for the target luminance using the compensation equation 2 below.

[0168] Compensation 2

[0169]

[0170] In one embodiment, the commands may cause the electronic device to analyze the target luminance for each frame of the input image data and compensate for the target luminance according to a third compensation method that gradually changes the target luminance using a reference target luminance.

[0171] In one embodiment, the commands may cause the electronic device to analyze the target luminance for each frame of the input image data and compensate for the target luminance using the compensation formula 3 below.

[0172] Compensation 3

[0173]

[0174] In one embodiment, the instructions may cause the electronic device to obtain a tone mapping curve corresponding to the input image data, adjust the tone mapping curve based on the on-pixel ratio, and perform the HDR tone mapping for the target luminance using the adjusted tone mapping curve.

[0175] In one embodiment, the instructions may cause the electronic device to display an image with reduced color distortion by rendering the image with a loading effect characteristic according to the on-pixel ratio based on the HDR tone mapping.

[0176] In one embodiment, the target luminance may be determined further based on display characteristics.

[0177] A display method according to embodiments of the present disclosure may include an operation of calculating an on-pixel ratio (OPR) based on input image data, an operation of determining a target luminance based on the on-pixel ratio and display setting luminance, an operation of performing HDR tone mapping using the target luminance, and an operation of displaying an image on which the HDR tone mapping has been performed.

[0178] In one embodiment, the operation of determining the target luminance may include an operation of determining whether the display set luminance is greater than the HBM maximum luminance, and an operation of determining the target luminance as the display set luminance if the display set luminance is less than or equal to the HBM maximum luminance.

[0179] In one embodiment, the operation of determining the target luminance may include an operation of comparing the on-pixel ratio with a reference OPR, and an operation of determining the target luminance as the display setting luminance when the on-pixel ratio and the reference OPR are equal.

[0180] In one embodiment, the operation of determining the target luminance may include an operation of comparing the on-pixel ratio with a reference OPR, and an operation of calculating the target luminance using Equation 1 below when the on-pixel ratio is greater than the reference OPR.

[0181] Formula 1

[0182]

[0183] In one embodiment, the operation of determining the target luminance may include an operation of comparing the on-pixel ratio with a reference OPR, and an operation of calculating the target luminance using Equation 2 below when the on-pixel ratio is less than the reference OPR.

[0184] Formula 2

[0185]

[0186] In one embodiment, the operation of determining the target luminance may include an operation of analyzing the target luminance for each frame of the input image data, and an operation of compensating the target luminance using the compensation formula 1 below.

[0187] Compensation 1

[0188]

[0189] In one embodiment, the operation of determining the target luminance may include an operation of analyzing the target luminance for each frame of the input image data, and an operation of compensating the target luminance using the compensation formula 2 below.

[0190] Compensation 2

[0191]

[0192] In one embodiment, the operation of performing the HDR tone mapping may include an operation of obtaining a tone mapping curve corresponding to the input image data, an operation of adjusting the tone mapping curve based on the on-pixel ratio, and an operation of performing the HDR tone mapping for the target luminance using the adjusted tone mapping curve.

[0193] While the present disclosure has been illustrated and described with reference to various exemplary embodiments, it should be understood that the various exemplary embodiments are intended to be illustrative, not restrictive. Those skilled in the art will appreciate that various changes in form and detail may be made without departing from the true spirit and scope of the present disclosure, including the appended claims and their equivalents. Furthermore, it should be understood that any embodiment(s) described in the present disclosure may be used in conjunction with any other embodiment(s) described in the present disclosure.

Claims

In an electronic device (101), display (230); At least one processor (240); and It includes a memory (210) that stores one or more commands, The one or more instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An operation to obtain the on-pixel ratio (OPR) based on input image data, An operation for determining a target luminance based on the above on-pixel ratio and display setting luminance, An operation of performing HDR tone mapping on the input image data using the target luminance, and An operation for controlling the display to display an image corresponding to the input image data on which HDR tone mapping has been performed. An electronic device that causes something to be done. In the first paragraph, An electronic device in which the on-pixel ratio is obtained based on pixel values ​​of each frame of the input image data. In claim 1 or 2, The one or more instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An operation of analyzing pixel values ​​of each frame of the above input image data, and An operation of obtaining at least one of the ratio of luminous pixels among all pixels or the degree of luminous intensity based on the above pixel values. An electronic device that causes something to be done more. In any one of the first to third paragraphs, The one or more instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An operation of identifying the target luminance as the display setting luminance based on the display setting luminance being less than or equal to the HBM (high brightness mode) maximum luminance. An electronic device that causes something to be done more. In any one of the first to fourth paragraphs, The one or more instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An operation of comparing the above on-pixel ratio with the reference on-pixel ratio, and An operation of identifying the target luminance as the display setting luminance based on the above on-pixel ratio and the above reference on-pixel ratio being equal. An electronic device that causes something to be done more. In any one of the first to fifth paragraphs, The one or more instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An operation of comparing the above on-pixel ratio with the reference on-pixel ratio, and An operation of setting the target brightness lower than the display setting brightness based on the above on-pixel ratio being greater than the above reference on-pixel ratio. An electronic device that causes something to be done more. In any one of claims 1 to 6, The one or more instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An operation of comparing the above on-pixel ratio with the reference on-pixel ratio, and An operation of setting the target brightness higher than the display setting brightness based on the above on-pixel ratio being smaller than the reference on-pixel ratio. An electronic device that causes something to be done more. In any one of the first to seventh paragraphs, The one or more instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An operation of acquiring illuminance data including illuminance of the usage environment, and An operation of determining the target brightness based on the on-pixel ratio, the display setting brightness, and the illuminance data. An electronic device that causes something to be done more. In any one of claims 1 to 8, The one or more instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An operation of analyzing the target luminance of each frame of the above input image data, and An operation for compensating the target luminance based on the first compensation method using an IIR (infinite impulse response) filter. An electronic device that causes something to be done more. In any one of claims 1 to 9, The one or more instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An operation of analyzing the target luminance of each frame of the above input image data, and An operation for compensating the target luminance based on a second compensation method that uses the average value of the target luminance of each frame of the input image data. An electronic device that causes something to be done more. In any one of claims 1 to 10, The one or more instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An operation of analyzing the target luminance of each frame of the above input image data, and An operation of compensating the target luminance based on a third compensation method including an incremental change of the target luminance based on a reference target luminance. An electronic device that causes something to be done more. In any one of claims 1 to 11, The one or more instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An operation of obtaining a tone mapping curve corresponding to the above input image data, An operation of adjusting the tone mapping curve based on the on-pixel ratio, and An operation of performing HDR tone mapping for the target luminance using the adjusted tone mapping curve. An electronic device that causes something to be done more. In any one of claims 1 to 12, The one or more instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An operation of rendering an image with a loading effect characteristic reflected based on the on-pixel ratio through the HDR tone mapping, and An action of displaying the rendered image on the display. An electronic device that causes something to be done more. In any one of claims 1 to 13, The one or more instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An operation of determining the target brightness based on the on-pixel ratio, the display setting brightness, and the characteristics of the display. An electronic device that causes something to be done more. In the method of displaying an image, An operation of obtaining an on-pixel ratio (OPR) based on input image data; An operation of determining a target luminance based on the above on-pixel ratio and display setting luminance; An operation of performing HDR tone mapping on the input image data using the target luminance; and An operation of displaying an image corresponding to the input image data on which HDR tone mapping has been performed; method.

Images