Electronic device for correcting and outputting image, and control method

Abstract

This electronic device may comprise: a display that includes first pixels including a first number of first sub-pixels; and an SoC, wherein the SoC can receive an input image that includes second pixels including a second number of second sub-pixels, obtain a sampling image by sampling the input image on the basis of the first pixels when the second number of second sub-pixels is greater than the first number of first sub-pixels, obtain a first filtering image by applying first HVS filtering to the sampling image, obtain a second filtering image by applying second HVS filtering to the input image, obtain an output image by compensating for color values of the sampling image on the basis of a color difference between the first filtering image and the second filtering image, and display the output image on the display.

Description

Electronic device and control method for correcting and outputting an image

[0001] The present disclosure relates to an electronic device and a control method for correcting and outputting an image.

[0002] Thanks to recent advancements in display technology, various types of display devices are being developed for use in diverse locations, such as homes, offices, and public spaces.

[0003] For example, a display device may include three subpixels (R, G, B) for one pixel. For another example, the display device may include a structure of fewer than three subpixels for various reasons such as device lifespan, aperture ratio, and power consumption. The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0004] An electronic device according to one embodiment comprises: a display including first pixels including a first number of first subpixels; and a System on Chip (SoC); wherein the SoC receives an input image including second pixels including a second number of second subpixels, and if the second number of the second subpixels is greater than the first number of the first subpixels, the input image is sampled based on the first pixels to obtain a sampled image, a first filtered image is obtained by applying a first HVS (human visual system) filtering to the sampled image, a second filtered image is obtained by applying a second HVS filtering to the input image, and an output image is obtained by compensating the color value of the sampled image based on the color difference between the first filtered image and the second filtered image, and the output image is displayed on the display.

[0005] According to one embodiment, the SoC can compensate the sampling image by dispersing (diffusion) the color difference between the second filtered image and the first filtered image corresponding to the first pixel included in the first filtered image to the first pixel and a plurality of pixels surrounding the first pixel.

[0006] According to one embodiment, the SoC can distribute and compensate for a first color difference between the first filtered image and the second filtered image corresponding to a first subpixel included in the first pixel to the first subpixel and a plurality of first adjacent subpixels around the first subpixel, and distribute and compensate for a second color difference between the first filtered image and the second filtered image corresponding to a second subpixel included in the pixel to the second subpixel and a plurality of second adjacent subpixels around the second subpixel.

[0007] According to one embodiment, the SoC can distribute and compensate the first color difference corresponding to the first subpixel to the first subpixel and the plurality of first adjacent subpixels based on a first weight corresponding to the first subpixel and a second weight corresponding to the plurality of first adjacent pixels, and distribute and compensate the second color difference corresponding to the second subpixel to the second subpixel and the plurality of second adjacent subpixels based on a third weight corresponding to the second subpixel and a fourth weight corresponding to the plurality of second adjacent pixels.

[0008] According to one embodiment, the SoC identifies the first weight and the second weight based on at least one of the filter coefficients applied to the first HVS filtering, the location of the first subpixel and the plurality of first adjacent subpixels, the first color difference, or a preset weight, and identifies the third weight and the fourth weight based on the filter coefficients applied to the second HVS filtering, the location of the second subpixel and the plurality of second adjacent subpixels, the second color difference, or at least one of the preset weight, or a preset weight.

[0009] According to one embodiment, the SoC identifies a first brightness value corresponding to a first pixel of the first filtered image and a second brightness value corresponding to a second pixel of the second filtered image, wherein the second pixel of the second filtered image corresponds to the first pixel of the first filtered image, and identifies a color difference between the first filtered image and the second filtered image based on a first color value and a first brightness value of the first pixel of the first filtered image, and a second color value and a second brightness value of the second pixel of the second filtered image, and compensates the sampling image by dispersing the color difference to the first pixel and a plurality of pixels surrounding the first pixel.

[0010] According to one embodiment, the SoC can identify the color difference between the first filtered image and the first filtered image based on a first difference value obtained by subtracting the first brightness value from the first color value per subpixel included in the first pixel of the first filtered image and a second difference value obtained by subtracting the second brightness value from the second color value per subpixel included in the second pixel of the second filtered image.

[0011] According to one embodiment, the SoC may acquire one R subpixel value included in the sampled image based on a plurality of R (Red) subpixel values ​​included in the input image, acquire one G subpixel value included in the sampled image based on a plurality of G (Green) subpixel values ​​included in the input image, and acquire one B subpixel value included in the sampled image based on a plurality of B (Blue) subpixel values ​​included in the input image.

[0012] According to one embodiment, a first region in which color distortion occurs in the input image is predicted based on pixel information of the input image, and for the predicted region, the output image is obtained based on a second filtered image in which the color value is compensated, and for the second region of the input image, the output image is obtained based on the second filtered image in which the color value is not compensated.

[0013] According to one embodiment, the display may be implemented as a pentile display having an RGBG subpixel structure in which each pixel includes an R subpixel and a G subpixel, or includes a B subpixel and a G subpixel.

[0014] A control method for an electronic device including first pixels including a first number of first subpixels according to one embodiment comprises: receiving an input image including second pixels including a second number of second subpixels; if the second number of the second subpixels is greater than the first number of the first subpixels, sampling the input image based on the first subpixels to obtain a sampled image; applying a first human visual system (HVS) filtering to the sampled image to obtain a first filtered image; applying a second HVS filtering to the input image to obtain a second filtered image; compensating the color value of the sampled image based on the color difference between the first filtered image and the second filtered image to obtain an output image; and displaying the output image on the display.

[0015] A non-transient computer-readable medium storing instructions that cause an electronic device to perform an operation when executed by an SoC of an electronic device including a display including first pixels including a first number of first subpixels according to one embodiment, wherein the operation comprises: receiving an input image including second pixels including a second number of second subpixels; if the second number of the second subpixels is greater than the first number of the first subpixels, sampling the input image based on the first pixels to obtain a sampled image; applying a first human visual system (HVS) filtering to the sampled image to obtain a first filtered image; applying a second HVS filtering to the input image to obtain a second filtered image; compensating the color value of the sampled image based on the color difference between the first filtered image and the second filtered image to obtain an output image; and displaying the output image on the display.

[0016] The above and other aspects and features of specific embodiments of the present disclosure will become more apparent from the following description taken together with the accompanying drawings.

[0017] FIGS. 1a to 1d are drawings illustrating various pixel structures to aid in understanding the present disclosure.

[0018] Figure 2a is a diagram illustrating a case in which an RGB image is displayed on a display of an RGB subpixel structure according to one example.

[0019] FIG. 2b is a diagram illustrating a case in which an RGB image is displayed on a display of an RGBG subpixel structure according to one example.

[0020] FIG. 3a is a block diagram showing the configuration of an electronic device according to one embodiment.

[0021] FIG. 3b is a drawing for illustrating an example of an implementation of an electronic device according to one embodiment.

[0022] FIG. 4 is a flowchart illustrating a method for controlling an electronic device according to one embodiment.

[0023] FIGS. 5a and 5b are drawings for explaining a sampling image acquisition method (operation 420) according to one embodiment.

[0024] FIG. 6 is a diagram illustrating an example of a method for acquiring an output image according to one embodiment.

[0025] FIGS. 7a and FIG. 7b are drawings for illustrating an example of an HVS filter according to one embodiment.

[0026] FIG. 8 is a flowchart illustrating a method for controlling an electronic device according to one embodiment.

[0027] FIG. 9 is a diagram illustrating a method for acquiring an output image according to one embodiment.

[0028] The present disclosure will be described in detail below with reference to the attached drawings.

[0029] The terms used in the embodiments of this disclosure have been selected to be as widely used as possible, taking into account their functions within this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been selected at the applicant's discretion, and in such cases, their meanings will be described in detail in the description section of the disclosure. Therefore, the terms used in this disclosure should be defined based on their meanings and the overall content of this disclosure, rather than merely their names (such as analyzing calls, messages, schedules, etc.).

[0030] In this specification, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of the above features (e.g., numerical values, functions, actions, or components such as parts) and do not exclude the presence of additional features.

[0031] The expression "at least one of A and / or B" should be understood as representing either "A" or "B" or "A and B".

[0032] Expressions such as "first," "second," "first," or "second" used in this specification may modify various components regardless of order and / or importance, and are used only to distinguish one component from another and do not limit said components.

[0033] Where it is stated that a component (e.g., a first component) is "(operatively or communicatively) coupled with / to" or "connected to" another component (e.g., a second component), it should be understood that the component may be directly connected to the other component or connected through the other component (e.g., a third component).

[0034] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as “comprising” or “consisting of” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0035] In the embodiments, a "module" or "part" performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of "modules" or a plurality of "parts" may be integrated into at least one module and implemented by at least one processor, except for a "module" or "part" that needs to be implemented in specific hardware.

[0036] In the present disclosure, the term "user" may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0037] The various elements and areas in the drawings are depicted schematically. Accordingly, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0038] Embodiments of the present disclosure will be described in more detail below with reference to the attached drawings.

[0039] FIGS. 1a to 1d are drawings illustrating various pixel structures to aid in understanding the present disclosure.

[0040] FIG. 1a is a diagram illustrating an RGB subpixel structure in which a single pixel is formed by three subpixels according to one example. According to one example, a display may have an RGB subpixel structure in which a single pixel includes three subpixels (R, G, B) as shown in FIG. 1a. For example, each subpixel (R, G, B) may be arranged side by side horizontally.

[0041] FIG. 1b is a diagram illustrating an RGBG subpixel structure in which one pixel is formed by two subpixels according to one example. According to one example, a display may have an RGBG subpixel structure in which one pixel includes two subpixels ((R, G) or (B, G)) as shown in FIG. 1b. For example, the (R, G) pixel and the (B, G) pixel may be alternately (or alternately) WPRHD.

[0042] FIG. 1c is a diagram illustrating an RGBG subpixel structure in which a single pixel is formed by two subpixels according to one example. According to one example, a display may have an RGBG subpixel structure in which a single pixel includes two subpixels ((R, G) or (G, B)) as shown in FIG. 1c. For example, (R, G) pixels and (G, B) pixels may be provided alternately in columns. According to one example, two G subpixels may be included to provide higher resolution and color reproduction capabilities tailored to human visual characteristics that are more sensitive to green.

[0043] FIG. 1d is a diagram illustrating an RGBG subpixel structure in which one pixel is formed by one subpixel according to one example. According to one example, a display may have an RGBG subpixel structure in which one pixel is formed by one subpixel (R or G or B) as shown in FIG. 1d. For example, R pixels and G pixels may be arranged alternately in one line, and G pixels and B pixels may be arranged alternately in another line.

[0044] According to one embodiment, the input image is an RGB image composed of three color information types, R, G, and B, for each pixel, but there may be cases where the image is displayed on a subpixel structure with fewer than three subpixels, as shown in FIGS. 1b to 1d. For example, in some displays (e.g., LED displays, Micro LED displays, OLED displays), a subpixel structure with fewer than three subpixels may be used, as shown in FIGS. 1b to 1d, for various reasons such as the lifespan of pixel elements, aperture ratio, and power consumption. According to one example, a pentile display may have a subpixel structure with fewer than three subpixels, as shown in FIGS. 1b to 1d.

[0045] For convenience of explanation, the following description assumes that the display is implemented as a Pentile display with an RGBG subpixel structure in which one pixel is formed by two subpixels, as shown in FIG. 1b.

[0046] Figure 2a is a diagram illustrating a case in which an RGB image is displayed on a display of an RGB subpixel structure according to one example.

[0047] According to FIG. 2a, when an RGB image is displayed on a display with an RGB subpixel structure as illustrated in FIG. 1a, the edges of the RGB image can be clearly depicted and / or perceived by the user without color distortion because the subpixel structure is identical to the RGB image. For example, the subpixel structure may be an RGB structure and the image may be an RGB image.

[0048] FIG. 2b is a diagram illustrating a case in which an RGB image is displayed on a display of an RGBG subpixel structure according to one example.

[0049] According to FIG. 2b, when an RGB image is displayed on a Pentile display with an RGBG subpixel structure as illustrated in FIG. 1b, color fringing occurs in the edge area due to a lack of subpixels. For example, as illustrated in FIG. 2b, the edge area may be located where subpixel R and / or subpixel B are located. For example, if the edge area is located where subpixel R is located, the edge is perceived as yellow by the user, and if it is located where subpixel B is located, the edge is perceived as cyan by the user, resulting in color fringing.

[0050] Accordingly, various embodiments for converting an input image so as not to cause perceptual color distortion when the subpixel structures of the input image and the display are different will be described below.

[0051] FIG. 3a is a block diagram showing the configuration of an electronic device according to one embodiment.

[0052] According to FIG. 3a, the electronic device (100) includes a System on Chip (SoC) (110) and a display (120).

[0053] According to one embodiment, the SoC of the electronic device (100) may be a single integrated circuit that integrates various functions into a single chip. According to one example, the SoC may be designed as a custom chip for converting input images.

[0054] According to one embodiment, a display (120) of an electronic device (100) can output visualized information to a user. For example, the display (120) can be controlled by a controller to output visualized information to a user. For example, the controller may include a processor or a GPU (graphic processing unit), but is not limited thereto. The display (120) may include LED (Light Emitting Diodes), micro LED, Mini LED, OLED (Organic Light Emitting Diodes) display, LCD (Liquid Crystal Display), PDP (Plasma Display Panel), QD (Quantum dot) display and / or QLED (Quantum dot light-emitting diodes). According to one example, the display (120) may be implemented as a flat display, a curved display, a folding and / or rolling flexible display.

[0055] According to one embodiment, the electronic device (100) may be implemented as at least one of a home appliance or a user terminal equipped with a display (120). According to one example, the electronic device (100) may be implemented as a TV, but is not limited thereto, and may be implemented as various types of display devices such as a monitor, kiosk, tablet PC, digital photo frame, mobile phone, LFD (large format display), Digital Signage, DID (Digital Information Display), and video wall. However, depending on the case, it may be implemented as a video processing device (e.g., a set-top box, one connected box) that is connected to the display device to provide video.

[0056] FIG. 3b is a drawing for illustrating an example of an implementation of an electronic device according to one embodiment.

[0057] For example, according to FIG. 3b, the electronic device (100') may include at least one of a display (120), at least one processor (130), memory (140), communication circuit (150), user interface (160), sensor (170), or speaker (180).

[0058] At least one of the display (120), at least one processor (130), memory (140), communication circuit (150), user interface (160), sensor (170), or speaker (180) may be electrically and / or operably coupled with each other by an electronic component such as a communication bus.

[0059] In one embodiment, the hardware of the electronic device (100') being operatively coupled may mean that a direct or indirect connection between the hardware is established via wired or wireless means so that the second hardware is controlled by the first hardware among the hardware. Although illustrated based on different blocks, the embodiment is not limited thereto, and some of the hardware of FIG. 3b (e.g., at least a part of the processor (130), memory (140), processor (130)) may be included in the SoC (110). The type and / or number of hardware included in the electronic device (100) is not limited to that shown in FIG. 3b. For example, the electronic device (100') may include only some of the hardware components shown in FIG. 3b.

[0060] According to one embodiment, the processor (130) of the electronic device (100') may include hardware for processing data based on one or more instructions. The hardware for processing data may include, for example, an arithmetic and logic unit (ALU), a floating point unit (FPU), a field programmable gate array (FPGA), a central processing unit (CPU), a graphic processing unit (GPU), a neural processing unit (NPU), and / or an application processor (AP). The number of processors (130) may be one or more. For example, the processor (130) may have the structure of a multi-core processor such as a dual core, a quad core, or a hexa core.

[0061] The processor (130) can control the operations of the electronic device (100') by executing instructions stored in the memory (140). For example, the processor (130) may correspond to a plurality of processors that divide and collectively perform a plurality of operations among the processors.

[0062] According to one embodiment, the memory (140) of the electronic device (100) may include a hardware component for storing data and / or instructions that are input and / or output to the processor (130). Depending on the purpose of data storage, the memory (140) may be implemented in the form of a memory embedded in the electronic device (100') or in the form of a memory that is detachable from the electronic device (100). For example, data for operating the electronic device (100) may be stored in a memory embedded in the electronic device (100'), and data for the expansion function of the electronic device (100) may be stored in a memory that is detachable from the electronic device (100'). Meanwhile, the memory embedded in the electronic device (100') may be implemented as at least one of volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), non-volatile memory (e.g., OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD).In addition, the memory that can be attached to and detached from the electronic device (100') can be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.) or an external memory that can be connected to a USB port (e.g., USB memory).

[0063] According to one embodiment, within the memory (140) of the electronic device (100), one or more instructions (or commands) representing operations and / or operations to be performed on data by the processor (130) may be stored. A set of one or more instructions may be referred to as firmware, an operating system, a process, a routine, a sub-routine, and / or an application. For example, the electronic device (100) and / or the processor (130) may perform various operations when a set of a plurality of instructions distributed in the form of an operating system, firmware, a driver, and / or an application is executed. In the following, the statement that an application is installed on an electronic device (100) means that one or more instructions provided in the form of an application are stored in the memory (140) of the electronic device (100), and that the one or more applications are stored in an executable format (e.g., a file having an extension specified by the operating system of the electronic device (100')) that is executable by the processor (130) of the electronic device (100).

[0064] At least one processor (130) controls the processing of input data according to a predefined operation rule or AI model (artificial-intelligence model) stored in memory (140). The predefined operation rule or AI model is characterized by being created through learning. Being created through learning means that a predefined operation rule or AI model with desired characteristics is created by applying a learning algorithm to a number of learning data. Such learning may be performed on the device itself where the artificial intelligence according to the present disclosure is performed, or it may be performed through a separate server / system.

[0065] An AI model may be composed of multiple neural network layers. At least one layer has at least one weight value and performs the layer's operation through the result of the operation of the previous layer and at least one defined operation. Examples of neural networks include convolutional neural networks (CNN), recurrent neural networks (RNN), deep neural networks (DNN), restricted Boltzmann machines (RBM), deep belief networks (DBN), bidirectional recurrent deep neural networks (BRDNN), deep Q-networks, and Transformers; however, the neural networks in this disclosure are not limited to the aforementioned examples except where specified.

[0066] A learning algorithm is a method of training a specific target device (e.g., a robot) using a number of learning data to enable the target device to make decisions or predictions on its own. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, and the learning algorithms in this disclosure are not limited to the aforementioned examples except where specified.

[0067] According to one embodiment, the communication circuit (150) of the electronic device (100') may include hardware for supporting the transmission and / or reception of electrical signals between the electronic device (100') and an external device (e.g., a server). For example, the communication circuit (150) may communicate with an external device, an external storage medium (e.g., a USB memory stick), an external server (e.g., a web hard drive), etc., through a communication method such as Bluetooth, AP-based Wi-Fi (Wi-Fi, Wireless LAN network), Zigbee, wired / wireless LAN (Local Area Network), WAN (Wide Area Network), Ethernet, IEEE 1394, HDMI (High-Definition Multimedia Interface), USB (Universal Serial Bus), MHL (Mobile High-Definition Link), AES / EBU (Audio Engineering Society / European Broadcasting Union), Optical, Coaxial, etc. According to one example, the communication circuit (150) can communicate with other electronic devices, external servers and / or remote control devices, etc.

[0068] According to one embodiment, the user interface (160) of the electronic device (100') may be implemented as a device such as a button, touchpad, mouse, and keyboard, or as a touch screen capable of performing the display function and operation input function described above.

[0069] According to one embodiment, a sensor (170) of an electronic device (100') can sense various information. The sensor (170) can be implemented as various types of sensors. For example, the sensor (170) may include at least one sensor among a time of flight (oF) sensor, an ultrasonic sensor, a radio detection and ranging (RADAR) sensor, a photodiode sensor, a proximity sensor, a passive infrared (PIR) sensor, a pinhole sensor, a pinhole camera, an infrared human body detection sensor, a complementary metal oxide semiconductor (CMOS) image sensor, a thermal sensor, a light sensor, and a motion sensor.

[0070] The sensor (170) may include a touch sensor that detects touch actions, such as a touch film, a touch sheet, or a touch pad, but is not limited thereto.

[0071] The sensor (170) may include at least one of a camera, a microphone, a CO2 sensor, and a barometric pressure sensor. The camera may convert captured images into electrical signals and generate image data based on the converted signals. For example, the camera may include at least one of a standard (or basic) camera, a depth camera, and an ultra-wide-angle camera. The microphone is configured to receive user voice or other sounds and convert them into audio data. The CO2 sensor is a sensor for measuring carbon dioxide concentration. The barometric pressure sensor is a sensor for sensing ambient pressure.

[0072] The sensor (170) may further include at least one sensor capable of sensing ambient illuminance, ambient temperature, and the direction of incidence of light. In this case, the sensor (170) may be implemented as an illuminance sensor, a temperature sensing sensor, a light intensity sensing layer, and a camera.

[0073] The sensor (170) may further include at least one of an acceleration sensor (or gravity sensor), a geomagnetic sensor, and a gyro sensor. For example, the acceleration sensor may be a 3-axis acceleration sensor. The 3-axis acceleration sensor may measure gravitational acceleration by axis and provide raw data to the processor (140). The geomagnetic sensor or the gyro sensor may be used to obtain attitude information. Here, the attitude information may include at least one of roll information, pitch information, or yaw information.

[0074] According to one embodiment, the speaker (180) of the electronic device (100') may be implemented to output various audio data as well as various notification sounds or voice messages. The processor (130) may control the speaker (180) to output feedback or various notifications in the form of audio according to various embodiments of the present disclosure.

[0075] In addition, the electronic device (100') may further include a camera and a microphone, etc., depending on the implementation example.

[0076] The camera can be turned on and perform shooting according to a preset event. The camera can convert the captured image into an electrical signal and generate image data based on the converted signal. For example, a subject is converted into an electrical image signal through a semiconductor optical element (CCD; Charge Coupled Device), and the image signal thus converted can be amplified, converted into a digital signal, and then processed. For example, the camera may include at least one of a standard (or basic) camera and an ultra-wide-angle camera.

[0077] The microphone is configured to receive user voice or other sounds and convert them into audio data. However, according to another embodiment, the electronic device (100') can receive user voice input through an external device via a communication circuit (150).

[0078] Meanwhile, depending on the implementation example of the electronic device (100'), a speaker, a tuner, and a demodulator may be additionally included. The tuner can receive RF (Radio Frequency) broadcast signals by tuning a channel selected by the user or all previously stored channels among the RF broadcast signals received through the antenna. The demodulator may receive a digital IF signal (DIF, demodulate digital intermediate frequency) converted by the tuner, demodulate it, and perform channel decoding, etc. According to one embodiment, the input video received through the tuner may be provided to the processor (140) after being processed through the demodulator (not shown).

[0079] According to one embodiment, when an input image is received, the processor (130) can process the input image. According to one example, the image processing may be digital image processing including at least one of image enhancement, image restoration, image transformation, image analysis, image understanding, image compression, image decoding, or scaling. Although such image processing may be performed on the input image before or after processing according to various embodiments described below, for convenience of explanation, the part regarding image processing is omitted. In this disclosure, "region" is a term referring to a part of an image and means at least one pixel block or a set of pixel blocks. Also, "pixel block" means a set of adjacent pixels containing at least one pixel.

[0080] FIG. 4 is a flowchart illustrating a method for controlling an electronic device according to one embodiment.

[0081] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0082] According to one embodiment, operations 410 to 470 may be understood to be performed in the SoC (110) of the electronic device (100). However, the present disclosure is not limited thereto, and operations 410 to 470 may be implemented by other components or other devices.

[0083] According to FIG. 4, in operation 410, the method may include an operation of identifying whether a pixel of an input image contains more subpixels than the number of subpixels of a pixel of a display (120). For example, an electronic device (100) may identify whether a pixel of an input image contains more subpixels than the number of subpixels of a display (120). For example, an electronic device (100) may identify whether the number of subpixels contained in each of a plurality of pixels of an input image is greater than the number of subpixels contained in each of a plurality of pixels of a display (120).

[0084] According to one example, the input image is an RGB image, and the display (120) can be implemented as a Pentile display with an RGBG subpixel structure as shown in FIG. 1b. In this case, the number of subpixels of the input image is 3, and the number of subpixels of the display is 2, so it can be identified that the number of subpixels of the input image is greater than the number of subpixels of the display (120).

[0085] In operation 420, the method may include the operation of acquiring a sampled image based on an input image. For example, if it is identified that a pixel of the input image contains more subpixels than the number of subpixels of the display (120) (410:Y), the electronic device (100) may acquire a sampled image based on the input image.

[0086] A sampled image may be an image obtained by sampling the color information of each subpixel of an RGB image to correspond to a subpixel of a display (120). For example, the sampling may be a process for appropriately outputting the Pentile sampled image data on a Pentile structure display (120) having an RGBG subpixel structure. Hereinafter, the process of converting an RGB image into an image that can be output on a Pentile structure display (120) having an RGBG subpixel structure is referred to as Pentile sampling (or RGBG sampling), and the image obtained through the said process is referred to as a Pentile sampled image (or RGBG sampled image or RGBG image).

[0087] According to one embodiment, the electronic device (100) analyzes an RGB image to identify the R, G, and B subpixel values ​​contained in each pixel and can rearrange the R, G, and B subpixel values ​​to correspond to a subpixel pattern of a pentile structure. For example, in the case of an RGBG subpixel pattern, the R subpixels and B subpixels may be reduced and more G subpixels may be used. Accordingly, during the conversion process, only one R subpixel and one B subpixel may be left in a 2x2 block, and the rest may be filled with G subpixels. However, when converting an RGBV image into a pentile structure in this way, the position of the subpixels changes, so interpolation processing may be required to maintain the visual quality of the image. Interpolation may be a process of filling in the missing subpixel values ​​by calculating them from neighboring values. In this process, an algorithm (e.g., subpixel rendering) that supplements the color value of each subpixel with adjacent pixels may be used. However, various methods may be used to obtain a pentile sampling image from an RGB image.

[0088] In operation 430, the method may include the operation of obtaining a first filtered image by applying a first filtering to a sampled image. For example, the electronic device (100) may obtain a first filtered image by applying a first HVS filtering to a sampled image (e.g., a pentile sampled image). For example, the electronic device (100) may obtain a first filtered image by applying a first HVS filtering to an RGB image.

[0089] HVS filtering is a method that utilizes the sensitivity of the human visual system to process images or videos, emphasizing important parts (e.g., parts that humans perceive better) or reducing or removing less important parts. For example, HVS filtering can be performed by assigning higher weights to specific components and lower weights to less important components based on HVS sensitivity. In one example, a Gaussian filter can be used for HVS filtering. A Gaussian filter can apply blurring by reducing high-frequency components and emphasizing low-frequency components in an image. However, it is not mandatory to use a Gaussian filter for HVS filtering; various types of filters that provide the same or similar filtering effects can be used. For the sake of convenience in explanation, filters used for HVS filtering will be referred to as HVS filters below.

[0090] For example, by applying the first HVS filtering to a pentile sampling image, it becomes possible to obtain a perception prediction image that is actually perceived by the user from the pentile sampling image.

[0091] In operation 440, the method may include, for example, an operation of obtaining a second filtered image by applying a second HVS filtering to an input image. The electronic device (100) may obtain a second filtered image by applying a second HVS filtering to an input image. For example, the electronic device (100) may obtain a second filtered image by applying a second HVS filtering to a pentile sampling image (or a pentile subpixel structure image).

[0092] For example, by applying a second HVS filtering to an RGB image, it becomes possible to obtain a perception prediction image that is actually perceived by the user.

[0093] In operation 450, the method may include an operation of obtaining an output image by compensating the color value of a sampling image according to the color difference between a first filtered image and a second filtered image. The electronic device (100) may obtain an output image by compensating the color value of a sampling image based on the color difference (or color error) between a first filtered image and a second filtered image. For example, the electronic device (100) may obtain an output image by compensating the color value of a sampling image based on the color difference actually perceived by the user.

[0094] According to one example, the electronic device (100) can compensate the sampling image by distributing the color difference between the first filtered image and the second filtered image corresponding to a pixel included in the first filtered image to the pixel and a plurality of pixels surrounding the pixel. For example, the electronic device (100) can compensate for the perceptual brightness difference that occurs when sampling an input image into an image with a pentile subpixel structure (hereinafter, pentile sampling) by appropriately distributing it to surrounding pixels.

[0095] According to one example, the electronic device (100) can compensate the sampled image by distributing the color difference for each subpixel included in the corresponding pixel to each subpixel and to a plurality of pixels surrounding each subpixel. For example, the electronic device (100) can distribute the first color difference between the first filtered image and the second filtered image corresponding to the first subpixel to the first subpixel and to a plurality of first adjacent subpixels surrounding the first subpixel. Additionally, the electronic device (100) can distribute the second color difference between the first filtered image and the second filtered image corresponding to the second subpixel to the second subpixel and to a plurality of second adjacent subpixels surrounding the second subpixel. For example, the electronic device (100) can distribute the color difference of the R component between the first filtered image and the second filtered image corresponding to the R subpixel to the R subpixel and to a plurality of adjacent R subpixels. For example, the electronic device (100) can distribute and compensate for the color difference of the G component between the first filtered image and the second filtered image corresponding to the G subpixel to the G subpixel and a plurality of adjacent G subpixels. For example, the electronic device (100) can distribute and compensate for the color difference of the B component between the first filtered image and the second filtered image corresponding to the B subpixel to the B subpixel and a plurality of adjacent B subpixels.

[0096] According to one example, the electronic device (100) may distribute and compensate a first color difference corresponding to a first subpixel to the first subpixel and a plurality of first adjacent subpixels based on a first weight corresponding to the first subpixel and a second weight corresponding to a plurality of first adjacent pixels. According to one example, the first weight and the second weight may be the same or different. For example, let us assume a case where the first subpixel is a reference R subpixel, the first adjacent subpixel is four adjacent R subpixels, the first color difference is A, the first weight is w1, the second weight is w2, and the same weight is assigned to the adjacent R subpixels. In this case, the color difference of color difference A*w1 may be compensated to the reference R subpixel, and the color difference of A*w2 may be compensated to the four adjacent R subpixels. Here, the sum of the weights (w1 + w2*4=1) may be.

[0097] According to one example, the first weight and the second weight may be identified based on at least one of the filter coefficients applied to the first HVS filtering, the location of the first subpixel and the first adjacent subpixel, the first color difference, or the preset weight.

[0098] According to one example, the electronic device (100) may distribute and compensate a second color difference corresponding to a second subpixel to the second subpixel and a plurality of second adjacent subpixels based on a third weight corresponding to the second subpixel and a fourth weight corresponding to a plurality of second adjacent pixels. According to one example, the third weight and the fourth weight may be the same or different. According to one example, the third weight may be the same as the first weight and the fourth weight may be the same as the second weight, but is not limited thereto. For example, let us assume a case where the second subpixel is a reference G subpixel, the second adjacent subpixel is four adjacent G subpixels, the second color difference is B, the fourth weight is w3, the fourth weight is w4, and the same weight is assigned to the adjacent G subpixels. In this case, the color difference of B*w3 may be compensated to the reference G subpixel, and the color difference of B*w2 may be compensated to the four adjacent G subpixels. Here, the sum of the weights (w3 + w4*4=1) may be

[0099] According to one example, the third weight and the fourth weight may be identified based on at least one of the filter coefficients applied to the second HVS filtering, the location of the second subpixel and the second adjacent subpixel, the second color difference, or a preset weight.

[0100] For example, the second and fourth weights for adjacent subpixels may change depending on the left / right position of the adjacent subpixels.

[0101] In operation 460, the method may include an operation of displaying an output image on a display. For example, the electronic device (100) may display the output image on the display (120). For example, the electronic device (100) may control the display (120) to display the output image. For example, the electronic device (100) may display a first filtered image with a color difference compensated, obtained in operation 450, on the display (120). For example, the first filtered image with a color difference compensated may be a pentile subpixel structure image corresponding to an RGBR display structure with a color difference compensated with respect to a second filtered image.

[0102] If it is identified that the pixels of the input image do not contain more subpixels than the number of subpixels of the pixels of the display (120) (410:N), the electronic device (100) may terminate image correction processing for the RGBR display structure. In this case, the electronic device (100) may control the display (120) to display the input image without performing input image correction processing for the RGBR display structure.

[0103] FIGS. 5A and FIGS. 5B are drawings for explaining a sampling image acquisition method according to one embodiment.

[0104] According to one embodiment, the electronic device (100) can obtain a pentile sampling image by sampling an input image, for example, an RGB image (510), as shown in FIG. 5a, so as to correspond to the RGBG subpixel structure (520) of the display (120).

[0105] According to one example, the electronic device (100) can obtain one R subpixel value included in the sampled image based on a plurality of R (Red) subpixel values ​​included in the input image, obtain one G subpixel value included in the sampled image based on a plurality of G (Green) subpixel values ​​included in the input image, and obtain one B subpixel value included in the sampled image based on a plurality of B (Blue) subpixel values ​​included in the input image. For example, the electronic device (100) can obtain a pentile sampled image through sub-pixel rendering as illustrated in FIG. 5b.

[0106] According to one embodiment, sub-pixel rendering is a technique for rendering an image by using sub-pixels independently without calculating on a pixel-by-pixel basis. According to one example, an electronic device (100) may assign weights to each sub-pixel value of neighboring pixels included in an RGB image in order to obtain the pixel value of a reference sub-pixel (Ro(x), Go(x), Bo(x+1), Go(x+1), ..) in a pentile sampling image. For example, the electronic device (100) may render a pentile sampling image using weights as shown in Equation 1 below.

[0107] [Mathematical Formula 1]

[0108] Ro(x) = Wr1 * R(x-1) + Wr2 * R(x) + Wr3 * R(x+1)

[0109] Go(x) = Wg1 * G(x-1) + Wg2 * G(x) + Wg3 * G(x+1)

[0110] Bo(x+1) = Wb1 * B(x) + Wb2 * B(x+1) + Wb3 * B(x+2)

[0111] Go(x+1) = Wg1 * G(x) + Wg2 * G(x+1) + Wg3 * G(x+2)

[0112] Here, Ro, Bo, and Go represent the subpixel values ​​of the Pentile structure resulting from subpixel rendering, x represents the pixel location, and Wr, Wg, and Wb represent the weights (or filter coefficients) for the R, G, and B subpixels, respectively. For example, the sum of the weights for each color can be 1.

[0113] According to one example, the subpixels used for sampling (or filtering) may include subpixels located immediately to the reference position (x) and on both sides (x-1, x+1), as shown in FIG. 5B. However, this is not limited thereto, and subpixels located at positions (x-2, x+2) or further away from the reference position (x) may also be used for filtering. According to one example, the number of subpixels used for sampling may be three, as shown in FIG. 5B, but more or fewer subpixels may be used.

[0114] For example, the image processing effect may vary depending on the weight of the sub-pixel rendering. For instance, the image processing effect may differ depending on how the weight for sub-pixel rendering is set. For instance, the smaller the difference between the weight of the center position and the weight of the surrounding position, the greater the color distortion reduction effect, while the side effect of the image becoming blurred may become stronger. Similarly, the wider the surrounding position for sampling is set, the greater the color distortion reduction effect, while the side effect of the image becoming blurred may also become stronger. Accordingly, the electronic device (100) can set the weight by considering the characteristics of each region of the input image.

[0115] FIG. 6 is a diagram illustrating an example of a method for acquiring an output image according to one embodiment.

[0116] According to one embodiment, at least one of the pentile sampling module (610), the first HVS filtering module (621), the second HVS filtering module (622), the color error calculation module (630), and the color error dispersion module (640) illustrated in FIG. 6 may be implemented with at least one software, at least one hardware, and / or a combination thereof. For example, the pentile sampling module (610), the first HVS filtering module (621), the second HVS filtering module (622), the color error calculation module (630), and the color error dispersion module (640) may be implemented to use a predefined algorithm, a predefined formula, and / or an artificial intelligence model. The pentile sampling module (610), the first HVS filtering module (621), the second HVS filtering module (622), the color error calculation module (630), and the color error dispersion module (640) may be included within the electronic device (100), but may be distributed to at least one external device according to one example.

[0117] According to one embodiment, the input image is an RGB image, and image processing can be performed on each subpixel R, G, and B included in the RGB image. However, for convenience of explanation in FIG. 6, an image (61) (hereinafter referred to as the R input image) containing the value of a specific subpixel, for example, an R subpixel, as shown in FIG. 6 is assumed. For example, it is assumed that the R component of the input 3x3 pixels included in the R input image (61) contains 180, 180, and 90 in all three rows.

[0118] According to one embodiment, the pentile sampling module (610) can convert an R input image (61) into an R pentile image (62) of a pentile structure through pentile sampling. For example, in the R pentile image (62), 0 may mean that the corresponding subpixel does not exist on the display. For example, when the R input image (61) undergoes pentile sampling, the R subpixel value of the pentile structure in the R pentile image (62) may be doubled from 180 to 360. This is because, since the number of R subpixels in the pentile structure is only half that of the RGB structure, the brightness of individual R subpixels must be doubled to maintain the same brightness on the display. However, this is merely an example, and the pentile sampling module (610) can obtain the pentile image (62) through subpixel rendering as described in FIG. 5.

[0119] According to one embodiment, the first HVS filtering module (622) can obtain a first filtered image (64) by HVS filtering the R Pentile image (62). For example, assuming the HVS filter is a 3x3 average filter, the 3x3 pixel R Pentile image (62) can become a 1 pixel image having 160 through HVS filtering.

[0120] According to one embodiment, the second HVS filtering module (621) can obtain a second filtered image (63) by HVS filtering the R input image (61). For example, if the HVS filter is assumed to be a 3x3 average filter that takes the average value of a 3x3 area, the R input image (61) of 3x3 pixels can become a 1 pixel image having 150 through HVS filtering.

[0121] According to one example, since HVS filtering reflects the characteristic of the human eye averaging the ambient brightness, the human eye can perceive a 3x3 pixel R input image (61) as having a brightness of 150 and a 3x3 pixel R Pentile image (62) as having a brightness of 160.

[0122] However, in FIG. 6, for convenience of explanation, the HVS filter is assumed to be a 3x3 average filter, but since the actual human eye is closer to a Gaussian filter, a Gaussian filter can be used for HVS filtering. For example, as shown in FIG. 7a, the Gaussian distribution can have a form in which the weight is large at 0 on the x-axis and decreases as it goes toward the + / - part, and if such a Gaussian distribution is applied to a 3x3 mask (70), the center of the mask (70) has a large weight and the weight decreases as it goes toward the edges of the mask (70). However, the values ​​shown in FIG. 7a are examples, and the filtering values ​​will, of course, vary depending on the sigma value of the Gaussian function. According to one example, the electronic device (100) can apply the Gaussian mask (70) to the 3x3 pixel R input image (61) and the 3x3 pixel R Pentile image (62), respectively, as shown in FIG. 7b. For example, since characteristics vary depending on various factors such as viewing distance, appropriate filter sizes and / or coefficients can be used for HVS filtering to suit the purpose.

[0123] According to one embodiment, the color error calculation module (630) can calculate a color error based on the first filtered image (64) and the second filtered image (63). For example, the color error calculation module (630) can calculate a color error of 10 based on "color value of the first filtered image (64) - color value of the second filtered image (63)". This means that in the embodiment illustrated in FIG. 6, when pentile sampling is applied, the brightness of the R input image appears to increase by 10 to the user's eye. However, since it is assumed that only the R component is handled in the example, it may mean that the red color appears to be 10 more intense to the user's eye.

[0124] According to one embodiment, the color error dispersion module (640) can obtain an output image by dispersing and compensating the R Pentile image (62) based on the calculated color error. For example, the color error dispersion module (640) can appropriately distribute the difference to surrounding pixels to compensate for the perceptual brightness difference that occurs before and after Pentile sampling. For example, according to FIG. 6, since there are 5 locations where subpixels exist within a 3x3 area in the R Pentile image (62), the color error dispersion module (640) can divide the brightness difference of 10 into 5 equal parts and subtract 2 from each subpixel. However, since it is assumed that only the R component is handled in this example, the color error dispersion module (640) can subtract the pixel value of the R subpixel by 2.

[0125] According to one embodiment, a method for dispersing brightness differences may include at least one of the following: dispersing evenly among surrounding pixels, dispersing by assigning weights according to HVS filter coefficients, assigning a higher weight to a central location than to surrounding locations, assigning a higher weight to surrounding locations than to a central location, assigning weights only to a central location or surrounding locations, and assigning a higher weight to a location among surrounding locations that has a smaller color error.

[0126] In the above-described embodiment, only the R subpixel was described for convenience of explanation, but the same method can be applied to the G subpixel and B subpixel.

[0127] FIG. 8 is a flowchart illustrating a method for controlling an electronic device according to one embodiment.

[0128] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0129] According to one embodiment, operations 810 to 890 can be understood as being performed in the SoC (110) of the electronic device (100). For operations 810 to 890 that overlap with operations 410 to 470 shown in FIG. 4, a detailed description will be omitted.

[0130] According to one embodiment, the color difference (or color error) according to the method described in FIG. 4 includes a luminance component. According to one example, the electronic device (100) can calculate only the pure color difference by excluding the luminance component from the color difference. According to one example, the electronic device (100) identifies a first luminance value corresponding to a pixel of the first filtered image and a second luminance value corresponding to said pixel of the second filtered image, and can identify the color difference between the first filtered image and the first filtered image based on the first color value and the first luminance value of the pixel of the first filtered image, and the second color value and the second luminance value of the pixel of the second filtered image. FIG. 8 describes the corresponding embodiment in detail.

[0131] According to FIG. 8, in operation 810, the electronic device (100) can identify whether the pixels of the input image contain more subpixels than the number of subpixels of the display (120).

[0132] If it is identified that the pixels of the input image contain more subpixels than the number of subpixels of the display (120) (810:Y), in operation 820, the electronic device (100) can acquire a sampling image based on the input image. For example, the electronic device (100) can acquire a pentile sampling image.

[0133] In operation 830, the electronic device (100) can obtain a first filtered image by applying a first HVS filtering to a sampling image (e.g., a pentile sampling image).

[0134] In operation 840, the electronic device (100) can obtain a second filtered image by applying a second HVS filtering to the input image.

[0135] In operation 850, the method may include an operation of obtaining a first difference value based on a first luminance value and a first color value for each subpixel included in a pixel of the first filtered image. For example, the electronic device (100) may obtain a first difference value by subtracting a first brightness value (or a first luminance value) from a first color value for each subpixel included in a pixel of the first filtered image. For example, the first brightness value may be a brightness value corresponding to a pixel of the first filtered image. For example, the electronic device (100) may obtain a brightness value corresponding to a pixel according to a preset brightness calculation formula. For example, the electronic device (100) may calculate a brightness value Y according to an RGB_to_Y() function such as Y=0.2126×R+0.7152×G+0.0722×B. Different weights are assigned to R, G, and B in this way because the sensitivity perceived by the human eye differs for each color component. However, this is not limited to this, and brightness values ​​can also be calculated by assigning equal weights to R, G, and B.

[0136] In operation 860, the method may include an operation of obtaining a second difference value based on a second luminance value and a second color value for each subpixel included in a pixel of the second filtered image. For example, the electronic device (100) may obtain a second difference value by subtracting a second brightness value (or a second luminance value) from a second color value for each subpixel included in a pixel of the second filtered image. For example, the second brightness value may be a brightness value corresponding to a pixel of the second filtered image. The second brightness value may be calculated in the same way as the first brightness value.

[0137] According to one example, the electronic device (100) can obtain a first difference value of a first filtered image and a second difference value of a second filtered image based on the following mathematical formula 2.

[0138] [Mathematical Formula 2]

[0139] Y1 = RGB_to_Y(R1, G1, B1)

[0140] R1' = R1 - Y1

[0141] G1' = G1 - Y1

[0142] B1' = B1 - Y1

[0143] Y2 = RGB_to_Y(R2, G2, B2)

[0144] R2' = R2 - Y2

[0145] G2' = G2 - Y2

[0146] B2' = B2 - Y2

[0147] Here, Y1 may be a brightness value of the first filtered image, and Y2 may be a brightness value of the second filtered image. R1, G1, and B1 may be subpixel values ​​of the first filtered image, and R2, G2, and B2 may be subpixel values ​​of the second filtered image. R1', G1', and B1' may be first difference values ​​of the first filtered image, and R2', G2', and B2' may be second difference values ​​of the second filtered image.

[0148] According to one example, the electronic device (100) can identify the first difference value (R1', G1', B1') of the first filtered image as the color value of a subpixel from which the brightness component is excluded, and the second difference value (R2', G2', B2') of the second filtered image as the color value of a subpixel from which the brightness component is excluded.

[0149] In operation 870, the method may include an operation of identifying a first filtered image and a color difference between the first filtered image and the first filtered image based on a first difference value and a second difference value. The electronic device (100) may identify a color difference between the first filtered image and the first filtered image based on a first difference value and a second difference value.

[0150] According to one example, the electronic device (100) can obtain a first filtered image and a color difference between the first filtered image based on the following mathematical formula 3.

[0151] [Mathematical Formula 3]

[0152] ColorError_R = R2' - R1'

[0153] ColorError_G = G2' - G1'

[0154] ColorError_B = B2' - B1'

[0155] Here, R1', G1', and B1' may be color values ​​of subpixels from which the brightness component is excluded in the first filtered image. R2', G2', and B2' may be color values ​​of subpixels from which the brightness component is excluded in the second filtered image. ColorError_R, ColorError_G, and ColorError_B may be the first filtered image and the color difference corresponding to the R, G, and B subpixels of the first filtered image.

[0156] In operation 880, the method may include an operation of obtaining an output image by compensating the color value of a sampling image based on the color difference between a first filtered image and a second filtered image. The electronic device (100) may obtain an output image by compensating the color value of a sampling image based on the color difference between a first filtered image and a second filtered image.

[0157] According to one example, the electronic device (100) can compensate the sampling image by dispersing the color difference between the first filtered image and the second filtered image to the corresponding pixel and a plurality of pixels surrounding the corresponding pixel. According to one example, the electronic device (100) can compensate the sampling image by dispersing the color difference for each subpixel included in the corresponding pixel to each subpixel and a plurality of pixels surrounding each subpixel. Since the method of dispersing the color difference is the same or similar to the method described in operation 450, a detailed description is omitted.

[0158] In operation 890, the electronic device (100) can display the output image on the display (120).

[0159] If it is identified that the pixels of the input image do not contain more subpixels than the number of subpixels of the pixels of the display (120) (810:N), the electronic device (100) may terminate image correction processing for the RGBR display structure. In this case, the electronic device (100) may control the display (120) to display the input image without input image correction processing for the RGBR display structure.

[0160] FIG. 9 is a diagram illustrating a method for acquiring an output image according to one embodiment.

[0161] According to one embodiment, when a pentile sampling image is obtained through sub-pixel rendering, color distortion can be reduced, but the more strongly the color distortion is reduced, the more the sharpness of the input image is reduced (blur). Accordingly, according to one embodiment, an electronic device (100) can predict an area where color distortion occurs in an input image based on pixel information of the input image, and obtain an output image based on a sampling image with color values ​​compensated for the predicted area and based on a sampling image with color values ​​not compensated for the remaining area.

[0162] According to FIG. 9, the electronic device (100) can predict (930) an area containing pixels where color distortion occurs based on an input image and obtain a color fringing map containing pixel information of the area. For example, the color fringing map may include location information of pixels where color distortion occurs and / or information about the degree of color distortion.

[0163] According to one embodiment, the electronic device (100) can obtain an output image (e.g., a pentile subpixel structure image) by processing the area containing pixels where color distortion occurs based on a color fringing map differently from the remaining area, and then muxing (940) the processed image.

[0164] According to one example, the electronic device (100) may apply sub-pixel rendering and / or color difference compensation, as illustrated in FIGS. 5a and 5b, only to areas containing pixels where color distortion occurs. According to one example, the electronic device (100) may determine different weights for sub-pixel rendering for areas containing pixels where color distortion occurs based on information contained in a color fringing map. For the remaining areas, the electronic device (100) may simply perform an operation of converting to a Pentile structure. This is because there is no need to cause blurring through sub-pixel rendering for areas where color distortion does not occur. Accordingly, by not performing unnecessary processing in areas where color distortion does not occur, a decrease in clarity can be prevented.

[0165] As described above, by applying subpixel rendering and / or color difference compensation only to areas containing pixels where color distortion occurs, blur caused by sampling (e.g., Pentile sampling) can be prevented more effectively.

[0166] According to the various embodiments described above, in order to reduce color distortion occurring in a Pentile structure display, color distortion reduction performance can be improved while preventing a decrease in clarity by predicting how much color distortion will occur from a cognitive perspective and distributing compensation to surrounding pixels. In addition, a decrease in clarity can be prevented by not performing unnecessary processing in areas where color distortion does not occur.

[0167] The methods according to the various embodiments of the present disclosure described above may be implemented in the form of an application that can be installed on an existing electronic device. However, they are not limited thereto, and the methods according to the various embodiments of the present disclosure described above may be performed using a deep learning-based artificial neural network (or deep artificial neural network), that is, a learning network model.

[0168] The methods according to the various embodiments of the present disclosure described above can be implemented by software upgrades or hardware upgrades alone for existing electronic devices.

[0169] The various embodiments of the present disclosure described above may also be performed through an embedded server equipped in an electronic device or an external server of the electronic device.

[0170] According to a specific example of the present disclosure, the various embodiments described above may be implemented as software comprising instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include an electronic device (e.g., electronic device (A)) according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions. When instructions are executed by a processor, the processor may perform a function corresponding to the instructions directly or by using other components under the control of the processor. Instructions may include code generated or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" means only that the storage medium does not contain a signal and is tangible, and does not distinguish whether data is stored semi-permanently or temporarily in the storage medium.

[0171] Additionally, according to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed online in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created in a storage medium such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0172] Additionally, each component (e.g., module or program) according to the various embodiments described above may be composed of a single or multiple entities, and some of the aforementioned sub-components may be omitted, or other sub-components may be further included in the various embodiments. Generally or additionally, some components (e.g., module or program) may be integrated into a single entity to perform the functions performed by each of the respective components prior to integration in the same or similar manner. The operations performed by the module, program, or other components according to the various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations added.

[0173] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

Claims

1. In an electronic device, A display comprising first pixels including a first number of first subpixels; and Includes SoC (System on Chip); and The above SoC is, Receive an input image including second pixels including a second number of second subpixels, and If the second number of the second subpixels is greater than the first number of the first subpixels, the input image is sampled based on the first subpixels to obtain a sampled image, and A first filtered image is obtained by applying a first HVS (human visual system) filtering to the above-mentioned sampling image, and A second filtered image is obtained by applying a second HVS filtering to the above input image, and Based on the color difference between the first filtered image and the second filtered image, the color value of the sampling image is compensated to obtain an output image, and An electronic device that displays the above output image on the above display.

2. In Paragraph 1, The above SoC is, An electronic device that compensates the sampling image by dispersing (diffusion) the color difference between the second filtering image and the first filtering image corresponding to the first pixel included in the first filtering image to the first pixel and a plurality of pixels surrounding the first pixel.

3. In Paragraph 2, The above SoC is, The first color difference between the first filtered image and the second filtered image corresponding to the first subpixel included in the first pixel is distributed and compensated to the first subpixel and a plurality of first adjacent subpixels surrounding the first subpixel, and An electronic device that distributes and compensates for a second color difference between a first filtered image and a second filtered image corresponding to a second subpixel included in the pixel to a plurality of second adjacent subpixels surrounding the second subpixel.

4. In Paragraph 3, The above SoC is, The first color difference corresponding to the first subpixel is distributed and compensated to the first subpixel and the plurality of first adjacent subpixels based on a first weight corresponding to the first subpixel and a second weight corresponding to the plurality of first adjacent pixels, and An electronic device that distributes and compensates the second color difference corresponding to the second subpixel to the second subpixel and the plurality of second adjacent subpixels based on a third weight corresponding to the second subpixel and a fourth weight corresponding to the plurality of second adjacent pixels.

5. In Paragraph 4, The above SoC is, Identifying the first weight and the second weight based on at least one of the filter coefficients applied to the first HVS filtering, the positions of the first subpixel and the plurality of first adjacent subpixels, the first color difference, or a preset weight, or a preset weight, and An electronic device that identifies the third weight and the fourth weight based on at least one of the filter coefficients applied to the second HVS filtering, the positions of the second subpixel and the plurality of second adjacent subpixels, the second color difference, or the preset weight, or the preset weight.

6. In Paragraph 1, The above SoC identifies a first brightness value corresponding to a first pixel of the first filtered image and a second brightness value corresponding to a second pixel of the second filtered image, wherein the second pixel of the second filtered image corresponds to the first pixel of the first filtered image, and Identifying the color difference between the first filtered image and the second filtered image based on the first color value and the first brightness value of the first pixel of the first filtered image, and the second color value and the second brightness value of the second pixel of the second filtered image. An electronic device that compensates the sampling image by dispersing the color difference to the first pixel and a plurality of pixels surrounding the first pixel.

7. In Paragraph 6, The above SoC is, An electronic device for identifying a color difference between a first filtered image and a second filtered image based on a first difference value obtained by subtracting a first brightness value from a first color value for each subpixel included in a first pixel of the first filtered image and a second difference value obtained by subtracting a second brightness value from a second color value for each subpixel included in a second pixel of the second filtered image.

8. In Paragraph 1, The above SoC is, An electronic device that obtains one R subpixel value included in the sampling image based on a plurality of R (Red) subpixel values ​​included in the input image, obtains one G subpixel value included in the sampling image based on a plurality of G (Green) subpixel values ​​included in the input image, and obtains one B subpixel value included in the sampling image based on a plurality of B (Blue) subpixel values ​​included in the input image.

9. In Paragraph 1, The above SoC is, Predicting a first region where color distortion occurs in the input image based on pixel information of the input image, and An electronic device that obtains an output image based on a sampling image in which the color value is compensated for the first region and based on a sampling image in which the color value is not compensated for the second region of the input image.

10. In Paragraph 1, The above display is, An electronic device implemented as a pentile display having an RGBG subpixel structure in which each pixel includes an R subpixel and a G subpixel, or a B subpixel and a G subpixel.

11. A method for controlling an electronic device comprising a display including first pixels including a first number of first subpixels, The operation of receiving an input image including second pixels including a second number of second subpixels; If the second number of the second subpixels is greater than the first number of the first subpixels, the operation of sampling the input image based on the first subpixels to obtain a sampled image; An operation to obtain a first filtered image by applying a first HVS (human visual system) filtering to the above sampling; The operation of obtaining a second filtered image by applying a second HVS filtering to the above input image; An operation to obtain an output image by compensating the color value of the sampling image based on the color difference between the first filtered image and the second filtered image; and A control method comprising the operation of displaying the output image on the display.

12. In Paragraph 11, The operation of acquiring the above output image is, A control method comprising: compensating the sampling image by dispersing (diffusion) the color difference between the second filtering image and the first filtering image corresponding to the first pixel included in the first filtering image to the first pixel and a plurality of pixels surrounding the first pixel.

13. In Paragraph 12, The operation of compensating the above-mentioned sampling image is, An operation of distributing and compensating for the first color difference between the first filtered image and the second filtered image corresponding to the first subpixel included in the first pixel to the first subpixel and a plurality of first adjacent subpixels surrounding the first subpixel; and A control method comprising: an operation of distributing and compensating for the second color difference between the first filtered image and the second filtered image corresponding to the second subpixel included in the pixel to the second subpixel and a plurality of second adjacent subpixels surrounding the second subpixel.

14. In Paragraph 13, The operation of compensating the above-mentioned sampling image is, An operation of distributing and compensating the first color difference corresponding to the first subpixel to the first subpixel and the plurality of first adjacent subpixels based on a first weight corresponding to the first subpixel and a second weight corresponding to the plurality of first adjacent pixels; and A control method comprising: an operation of distributing and compensating the second color difference corresponding to the second subpixel to the second subpixel and the plurality of second adjacent subpixels based on a third weight corresponding to the second subpixel and a fourth weight corresponding to the plurality of second adjacent pixels.

15. A non-transient computer-readable medium storing instructions that cause an electronic device to perform an operation when executed by a System on Chip (SoC) of an electronic device including a display including first pixels including a first number of first subpixels, The above operation is, The operation of receiving an input image including second pixels including a second number of second subpixels; If the second number of the second subpixels is greater than the first number of the first subpixels, the operation of sampling the input image based on the first subpixels to obtain a sampled image; The operation of obtaining a first filtered image by applying a first HVS (human visual system) filtering to the above-mentioned sampling image; The operation of obtaining a second filtered image by applying a second HVS filtering to the above input image; An operation to obtain an output image by compensating the color value of the sampling image based on the color difference between the first filtered image and the second filtered image; and A control method comprising the operation of displaying the output image on the display.

Images