Image processing device and method

Abstract

This display device may comprise a memory for storing at least one instruction, wherein the at least one instruction, when executed by at least one processor, causes the display device to: obtain a first screen capture image having a first data format; change a data format for the first screen capture image from the first data format to a second data format including a transparency element; adjust a value of the transparency element for a video region associated with a first video image in the first screen capture image to transparently process the video region; generate a second composite image on the basis of the first video image or a second video image following the first video image, and a second screen capture image including the transparently processed video region; and perform blur processing on the second composite image.

Description

Image processing device and method

[0001] The present disclosure relates to an image processing device and method.

[0002] Recent display devices such as TVs provide users with graphical information, such as program guide information, along with video images based on broadcast signals.

[0003] When presenting video and graphic images together, it may be necessary to apply image processing effects, such as blur, to effectively convey the desired information to the user. In these cases, it's necessary to consider image processing methods that can apply the desired effects despite performance constraints, such as hardware and software limitations.

[0004] According to one embodiment of the present disclosure, a display device includes a display, at least one processor, and a memory storing at least one instruction, wherein the at least one instruction, when executed by the at least one processor, causes the display device to: obtain a first screen capture image having a first data format by capturing a first composite image generated based on a first video image and at least one graphic image using a screen capture method, wherein the first data format does not include a transparency element representing transparency, change a data format for the first screen capture image from the first data format to a second data format including the transparency element, and adjust a value of a transparency element for a video area associated with the first video image in the first screen capture image based on data of the second data format, thereby processing the video area to be transparent, and generate a second composite image based on a second screen capture image including the first video image or a second video image following the first video image and the processed-transparent video area, and an area in the second composite image in which the first video image or the second video image is displayed is within the processed-transparent video area. It is included, and can perform blur processing on the second composite image, and control the display to display at least a portion of the blurred second composite image.

[0005] A method of a display device according to one embodiment of the present disclosure comprises: obtaining a first screen capture image having a first data format by capturing a first composite image generated based on a first video image and at least one graphic image using a screen capture method, wherein the first data format does not include a transparency element indicating transparency; changing a data format for the first screen capture image from the first data format to a second data format including the transparency element; adjusting a value of a transparency element for a video area associated with the first video image in the first screen capture image based on data of the second data format, thereby making the video area transparent; generating a second composite image based on a second screen capture image including the first video image or a second video image following the first video image and the made-transparent video area, wherein an area in the second composite image in which the first video image or the second video image is displayed is included within the made-transparent video area; An operation of performing blur processing on the second composite image and displaying at least a portion of the blurred second composite image can be performed.

[0006] According to one embodiment of the present disclosure, a display device includes a display, at least one processor, and a memory storing at least one command, wherein the at least one command, when executed by the at least one processor, causes the display device to: generate a composite image including a graphic image and a video image disposed in a transparent area; receive a command to perform an application that supports blur processing; blur the composite image based on the command; generate a display image in which an image related to the application is superimposed on at least a portion of the blurred composite image; and control the display to display the display image.

[0007] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings.

[0008] FIG. 1 illustrates a block configuration of a display device according to an embodiment of the present disclosure.

[0009] FIG. 2 illustrates a block configuration of a display device according to one embodiment of the present disclosure.

[0010] FIG. 3 is a diagram for explaining a procedure for processing video data and graphic data by a display device according to one embodiment of the present disclosure.

[0011] FIG. 4 is a drawing for explaining a capture operation of a display device according to one embodiment of the present disclosure.

[0012] FIG. 5 is a diagram illustrating an image obtained through a capture operation of a display device according to one embodiment of the present disclosure.

[0013] FIG. 6 illustrates changes in an output image according to execution of an application supporting image effect processing according to one embodiment of the present disclosure.

[0014] FIG. 7 is a diagram for explaining a blur processing procedure according to one embodiment of the present disclosure.

[0015] FIG. 8 is a drawing for explaining a blur processing result according to a blur processing procedure in a multi-window case according to one embodiment of the present disclosure.

[0016] FIG. 9 is a drawing for explaining a blur processing result according to a blur processing procedure in a multi-window case according to one embodiment of the present disclosure.

[0017] FIG. 10 is a diagram for explaining a blur processing procedure according to one embodiment of the present disclosure.

[0018] FIG. 11 is a diagram for explaining a blur processing procedure according to one embodiment of the present disclosure.

[0019] FIG. 12 shows changes in an image resulting from processing a screen capture image in a blur processing procedure according to one embodiment of the present disclosure.

[0020] FIG. 13 is a diagram illustrating an image obtained in a blur processing procedure according to one embodiment of the present disclosure.

[0021] FIG. 14 is a flowchart illustrating a method of a display device according to one embodiment of the present disclosure.

[0022] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0023] At this time, it will be understood that each block of the processing flow diagrams and combinations of the flow diagrams can be performed by computer program instructions.

[0024] Additionally, each block may represent a module, segment, or portion of code that contains one or more executable instructions for performing a specific logical function(s). It should also be noted that in some alternative implementation examples, the functions described in the blocks may occur out of order. For example, two blocks depicted in succession may actually be executed substantially concurrently, or the blocks may sometimes be executed in reverse order, depending on their respective functions.

[0025] Here, the term '~ part' used in this embodiment means software or hardware components such as FPGA (field programmable gate array) or ASIC (application specific integrated circuit), and the '~ part' performs certain roles. However, the '~ part' is not limited to software or hardware. The '~ part' may be configured to be on an addressable storage medium or may be configured to reproduce one or more packet processing devices. Therefore, as an example, the '~ part' includes components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and '~ parts' may be combined into a smaller number of components and '~ parts' or further separated into additional components and '~ parts'. In addition, the components and '~parts' may be implemented to play one or more central processing units (CPUs) within the device or secure multimedia card. In addition, in the embodiment, the '~parts' may include one or more packet processing devices.

[0026] FIG. 1 illustrates a block configuration of a display device according to an embodiment of the present disclosure.

[0027] The display device (100) of FIG. 1 may be, but is not limited to, a smartphone, a tablet PC, a PC, a smart TV, a mobile phone, a personal digital assistant (PDA), a laptop, a media player, a micro server, a digital broadcasting terminal, a navigation device, a kiosk, a home appliance, and other mobile or non-mobile computing devices. The display device (100) may perform various computing functions, such as real-time video viewing and communication. In the following description, the display device (100) is described assuming that it is a TV or a monitor, but this is merely an example, and the embodiments of the present disclosure may be equally applied to electronic devices having a display function.

[0028] According to one embodiment, the display device (100) may include a processor (e.g., including a processing circuit) (110), a memory (120), an image input interface (e.g., including an image input circuit) (130), a display (140), and a communication interface (e.g., including a communication circuit) (150).

[0029] According to one embodiment, the memory (120) is a storage medium used by the display device (100) and can store data such as at least one command (121) (e.g., at least one command corresponding to at least one program) or setting information. The program may include an operating system (OS) program and various application programs.

[0030] According to one embodiment, the memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read only memory (ROM), an electrically erasable programmable ROM (EEPROM), a programmable ROM (PROM), a magnetic memory, a magnetic disk, and an optical disk.

[0031] According to one embodiment, the video input interface (130) can receive video data through a tuner, an input / output interface, or a communication interface (150). The video input interface (130) can include at least one of the tuner and the input / output interface. The tuner can select only the frequency of a broadcast channel to be received by the display device (100) from among many radio wave components through amplification, mixing, resonance, etc. of a broadcast signal received wirelessly or wiredly. The broadcast signal can include video, audio, and additional data (e.g., an electronic program guide (EPG)). The tuner can receive broadcast channels (or, viewing images) from various broadcast sources such as terrestrial broadcasting, cable broadcasting, satellite broadcasting, and Internet broadcasting. The tuner can be implemented as an integrated unit with the display device (100) or as a separate tuner electrically connected to the display device (100). The above input / output interface may include at least one of an HDMI (high definition multimedia interface) input port, a component input jack, a PC input port, and a USB input jack, which can receive image data from an external device of the display device (100) under the control of the processor (110). It is obvious to those skilled in the art that the above input / output interface may be added, deleted, and / or changed depending on the performance and structure of the display device (100).

[0032] According to one embodiment, the display (140) may perform functions for outputting information in the form of numbers, characters, images, and / or graphics. The display (140) may include at least one hardware module for outputting. The at least one hardware module may include, for example, at least one of a liquid crystal display (LCD), a light emitting diode (LED), a light emitting polymer display (LPD), an organic light emitting diode (OLED), an active matrix organic light emitting diode (AMOLED), or a flexible LED (FLED). The display (140) may display a screen corresponding to data received from the processor (110). The display (140) may be referred to as an 'output unit', a 'display unit', or other terms having an equivalent technical meaning thereto.

[0033] According to one embodiment, the communication interface (150) may provide a wired / wireless communication interface that enables communication with an external device. The communication interface (150) may include at least one of a wired Ethernet, a wireless LAN communication interface, and a short-range communication interface. The wireless LAN communication interface may include, for example, Wi-Fi, and may support the wireless LAN standard (IEEE802.11x) of the Institute of Electrical and Electronics Engineers (IEEE). The wireless LAN communication interface may be wirelessly connected to an AP (Access Point) under the control of the processor (110). The short-range communication interface may wirelessly perform short-range communication with an external device under the control of the processor (110). Short-range communication may include Bluetooth, Bluetooth low energy, infrared communication (IrDA: infrared data association), ultra wideband (UWB), WiFi Direct, and near field communication (NFC). The above external devices may include server devices that provide video services, etc., and mobile terminals (e.g., phones, tablets, etc.).

[0034] According to one embodiment, the processor (110) may execute at least one instruction (121) stored in the memory (120) to perform calculations or data processing related to control and / or communication of at least one other component of the display device (100). The processor (110) may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), a micro controller unit (MCU), a sensor hub, a supplementary processor, a communication processor, an application processor, an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA), and may have multiple cores.

[0035] FIG. 2 illustrates a block configuration of a display device according to one embodiment of the present disclosure.

[0036] Referring to FIG. 2, the display device (200) may include a processor (110), a memory (120), an image input interface (130), a display (140), a communication interface (150), and a capture unit (e.g., including a capture circuit) (210). Compared to the display device (100) of FIG. 1, the display device (200) of FIG. 2 may further include a capture unit (210). For a description of the processor (110), the memory (120), the image input interface (130), the display (140), and the communication interface (150) of the display device (200) of FIG. 2, reference may be made to the description of FIG. 1. Therefore, any duplicate description will be omitted.

[0037] According to one embodiment, the processor (100) may generate a video image based on input video data input through the video input interface (130). According to one embodiment, the video image may correspond to a video frame.

[0038] In one embodiment, the video image may be an image corresponding to a received video signal (e.g., a broadcast signal). For example, the video image may be a content image itself, generated by converting a video signal corresponding to the content into an image. For example, the video image may be a dynamic image, comprising an image of an object moving within the screen over time, but is not limited thereto.

[0039] According to one embodiment, the processor (100) may generate a graphic image based on input graphic data input through the image input interface (130). According to one embodiment, the graphic image may correspond to a graphic frame.

[0040] According to one embodiment, the graphic image may be an image displayed together with a video image. As an example, the graphic image may be a static image that constitutes an image in which an object within the screen does not move over time, but is not limited thereto. According to one embodiment, the graphic image may be an image generated (or graphically processed) by a processor (110) (e.g., GPU) of the display device. According to one embodiment, the graphic image may include at least one piece of graphic information. The graphic information may include, for example, at least one of a user interface, menu information, setting information, user guide information, electronic program guide information, or notification information.

[0041] According to one embodiment, the capture unit (210) can capture an image. For example, the capture unit (210) can capture a video image and / or a graphic image (e.g., a static image).

[0042] According to one embodiment, the capture unit (210) may store data of a captured image (hereinafter, “capture image”) in a memory (e.g., a capture buffer). For example, the capture unit (210) may store data of a captured video image (hereinafter, “capture video image”) in a video capture buffer, and store data of a captured graphic image (hereinafter, “capture graphic image”) in a graphic capture buffer.

[0043] According to one embodiment, in the display device (200), the capture unit (210) can allocate a capture buffer for each captured image. For example, the display device (200) can allocate a video capture buffer for each captured video image, and can allocate a graphic capture buffer for each captured graphic image.

[0044] According to one embodiment, the capture unit (210) may include hardware components and / or software components. For example, the capture unit (210) may include a hardware component (e.g., a video capture driver) for capturing video images and a software component for capturing graphic images.

[0045] In one embodiment, when capturing video images using hardware components (e.g., a video capture driver), the capture unit (210) cannot capture all video images streaming in real time due to hardware performance limitations. For example, when video images are streamed at 60 FPS (frames per second), the capture unit (210) may capture video images at a capture performance lower than 60 times per second (e.g., a capture performance of 15 times per second). In this case, the display device may interpolate video images that were not captured (e.g., 45 video images per second) through an interpolation function.

[0046] An example of the operation of the capture unit (210) is exemplarily described below with reference to FIGS. 4 and 5.

[0047] FIG. 3 is a diagram illustrating a procedure for a display device to process video data and graphic data, respectively, through a video path and a graphic path, according to one embodiment of the present disclosure. The video path may be an image processing path for a video frame sequence, and the graphic path may be an image processing path for a static image.

[0048] Referring to FIG. 3, a display device (e.g., the display device (100) of FIG. 1 or the display device (200) of FIG. 2) can generate a plurality of images, generate a composite image based on the generated plurality of images, and output the composite image. For example, the display device can generate a video image and a graphic image using input video data and input graphic data, respectively, generate a composite image based on the generated video image and graphic image, and output the composite image. According to one embodiment, the size of the composite image can be the same as the size of the screen (screen) displayed (or output) through the display (140).

[0049] In one embodiment, the video image may be an image corresponding to a received video signal (e.g., a broadcast signal). For example, the video image may be a content image itself, generated by converting a video signal corresponding to the content into an image. For example, the video image may be a dynamic image, comprising an image of an object moving within the screen over time.

[0050] According to one embodiment, the graphic image may be an image displayed together with a video image. For example, the graphic image may be a static image that constitutes an image in which an object on the screen does not move over time. For example, the graphic image may be an image generated by a processor (110) (e.g., a GPU) of the display device. According to one embodiment, the graphic image may include at least one piece of graphic information. The graphic information may include, for example, at least one of a user interface, menu information, setting information, user guide information, electronic program guide information, or notification information.

[0051] According to one embodiment, the display device (200) can generate and / or process video images and graphic images, respectively, through separate paths or pipelines. For example, the display device (200) can generate and / or process video images on a video plane through a video path (310), and can generate and / or process graphic images on a graphic plane through a graphic path (320).

[0052] According to one embodiment, the video plane may be a plane for supporting generation and / or processing of a video image. As an example, the video plane may support generation and / or processing of a video image having a first data format (e.g., a YUV data format). Here, the first data format may be a data format that does not include a component indicating transparency (hereinafter, referred to as a transparency element). As an example, the first data format may be a YUV data format, but is not limited thereto. Here, the YUV data format is one of the video data formats and includes a Y (luma) component indicating brightness information, a U (Cb: chroma blue) component indicating blue difference information, and a V (Cr: chroma red) component indicating red difference information.

[0053] According to one embodiment, the graphics plane may be a plane for supporting the generation and / or processing of graphic images. For example, the graphics plane may support the processing and / or generation of graphic images having a second data format different from the first data format of the video plane. Here, the second data format may be a data format including a transparency element. For example, the second data format may be, but is not limited to, an RGBA data format. Here, the RGBA data format is one of the data formats representing color information, and includes an R (red) component representing a red color component, a G (green) component representing a green color component, a B (blue) component representing a blue color component, and an A (alpha) component representing transparency. The range of the value of each component may be, for example, from 0 to 255. For example, the range of the alpha value may be, for example, from 0 to 255, where 0 represents complete transparency and 255 represents complete opacity.

[0054] In one embodiment, a video plane containing a video image may have a different layer than a graphics plane containing a graphic image. For example, the video plane may be a sublayer of the graphics plane. For example, when generating a composite image, the video image (or video plane) may be positioned below the graphic image (or graphics plane).

[0055] Generation and / or processing of video images via the video path (310) can be performed by a video input interface (e.g., including a video input circuit) (131) and a video processing unit (111).

[0056] According to one embodiment, the video input interface (131) may be included in the image input interface (130).

[0057] According to one embodiment, the video input interface (131) can obtain input video data. For example, the video input interface (131) can obtain input data from a video signal received from a broadcast server. The input video data may include, but is not limited to, data for generating at least one video image (e.g., video pixel data).

[0058] According to one embodiment, the video processing unit (111) may be included in the processor (110). In the present disclosure, the operation of the video processing unit (111) may be understood as the operation of the processor (110).

[0059] According to one embodiment, the video processing unit (111) may generate a video image based on input video data. For example, the video processing unit (111) may generate at least one video image on a video plane based on video pixel data.

[0060] According to one embodiment, when it is necessary to generate multiple video images, the video processing unit (111) can generate video images on multiple video planes, respectively. When video images are generated on multiple video planes, different image settings (e.g., resolution, size / position, size / position of a window containing the video image, etc.) can be applied to each video plane. The multiple video planes can have different layers.

[0061] According to one embodiment, the video processing unit (111) may include a scaler (111a) and / or a quality processing unit (111b). A video image generated on a video plane by the video processing unit (111) may be processed by the scaler (111a) and / or the quality processing unit (111b).

[0062] According to one embodiment, the scaler (111a) may perform scaling processing on a video image. The scaling processing may include processing for adjusting the size and / or position of the video image. As an example, the scaling processing may include processing for adjusting the size or resolution of the video image. For example, the scaling processing may include processing for enlarging, reducing, increasing, or reducing the resolution of the video image. As an example, the scaling processing may include processing for adjusting the position of the video image. As an example, the scaling processing may include processing for adjusting the position of a video image whose size or resolution has been adjusted.

[0063] According to one embodiment, the scaler (111a) may perform scaling processing on the video image so that an area where the video image is displayed in the composite image corresponds to a portion of a graphic image (e.g., a transparent area) associated with the video image. For example, the scaler (111a) may perform scaling processing on the video image so that an area where the video image is displayed in the composite image matches a portion of a graphic image associated with the video image. For example, the scaler (111a) may perform scaling processing on the video image so that an area where the video image is displayed in the composite image is included in a portion of a graphic image associated with the video image. The graphic image associated with the video image may be, for example, a graphic image used to generate the composite image together with the video image.

[0064] According to one embodiment, the scaler (111a) may perform scaling processing on a video image based on scaling setting information received from the graphics processing unit (112) (or GPU (112a)). The scaling setting information may include a scaling factor and / or a position adjustment value. The scaling factor may be a ratio indicating how much to adjust the size of the video image. For example, the scaling factor may be a value expressing a relative size change with respect to the original size. If the scaling factor is less than 1, size reduction may occur, and if the scaling factor is greater than 1, size enlargement may occur. For example, if the scaling factor is 2, the scaler (111a) may enlarge the video image by twice its original size.

[0065] According to one embodiment, the image quality processing unit (111b) may perform image quality processing on a video image (or a scaled video image). The image quality processing may include operations for improving the image quality of the video image. For example, the image quality processing may include, but is not limited to, contrast and brightness adjustment operations, color correction operations, sharpness enhancement operations, tone mapping operations, filter and effect application operations, and / or noise removal operations.

[0066] In the embodiment of FIG. 3, the operation of the image quality processing unit (111b) is exemplified as being performed after the operation of the scaler (111a), but the embodiment is not limited thereto. For example, if necessary, the operation of the image quality processing unit (111b) may be performed before the operation of the scaler (111a). Furthermore, if necessary, at least one of the operations of the scaler (111a) and the image quality processing unit (111b) for a specific video image may be omitted.

[0067] The generation and processing of graphic images through a graphic path (320) can be performed by a graphic input interface (132) and a graphic processing unit (112).

[0068] According to one embodiment, the graphical input interface (132) may be included in the video input interface (130).

[0069] According to one embodiment, the graphic input interface (132) may obtain input graphic data. For example, the graphic input interface (132) may receive input graphic data generated by at least one application of the display device. The input graphic data may include, for example, data for generating at least one graphic image (e.g., graphic pixel data).

[0070] According to one embodiment, the graphics processing unit (112) may be included in the processor (110). In the present disclosure, the operation of the graphics processing unit (112) may be understood as the operation of the processor (110). According to one embodiment, the graphics processing unit (112) may include a GPU (112a).

[0071] According to one embodiment, the graphics processing unit (112) (or GPU (112a)) may generate a graphic image based on input graphic data. For example, the graphics processing unit (112) (or GPU (112a)) may generate at least one graphic image on a graphic plane based on graphic pixel data.

[0072] According to one embodiment, when multiple graphic images need to be generated (e.g., the multi-window case of FIG. 8 or 9), the graphics processing unit (112) (or GPU (112a)) can generate the graphic images on multiple graphic planes, respectively. When the graphic images are generated on multiple graphic planes, different image settings (e.g., resolution, size / position, size / position of a window including the graphic image, etc.) can be applied to each graphic plane. The multiple graphic planes can have different layers.

[0073] According to one embodiment, the graphics processing unit (112) (or GPU (112a)) may generate scaling setting information for a video image associated with a graphics image and transmit the scaling setting information to a scaler (111a).

[0074] According to one embodiment, the display device can generate a composite image based on a video image and a graphic image. For example, the display device can generate the composite image by mixing the video image and the graphic image using a mixer (113). For example, the display device can generate the composite image by overlaying the video image and the graphic image. For example, the display device can generate the composite image by placing a video image (e.g., a scaled video image) under a graphic image. In this case, the area where the scaled video image is displayed can correspond to a portion of the graphic image (e.g., a transparent area). Therefore, even if the video image is placed under the graphic image in the composite image, the video image can be displayed to a user of the display device.

[0075] According to one embodiment, the display (140) can display the composite image as an output image (141). The output image (141) is an image visually recognized by a user of the display device and may be an image included on the screen of the display (140).

[0076] FIG. 4 is a diagram illustrating a capture operation of a display device according to one embodiment of the present disclosure. FIG. 5 is a diagram illustrating an image acquired through a capture operation of a display device according to one embodiment of the present disclosure.

[0077] Referring to FIGS. 4 and 5, a display device (e.g., display device (100) of FIG. 1 or display device (200) of FIG. 2) can perform image capture. According to one embodiment, the capture operation of FIG. 4 can be performed by a capture unit (e.g., capture unit (210) of FIG. 2) or a processor (e.g., processor (110) of FIGS. 1 / 2) of the display device.

[0078] According to one embodiment, the display device can perform capture of a video image (hereinafter, video capture) (410) and / or capture of a graphic image (hereinafter, graphic capture) (420).

[0079] In one embodiment, video capture (410) may be performed on a video path (e.g., video path (310) of FIG. 3) (or a video plane), and graphics capture (420) may be performed on a graphics path (e.g., graphics path (320) of FIG. 3) (or a graphics plane).

[0080] In one embodiment, video capture (410) may include a main / sub scaler capture operation (411), a post capture operation (412), and / or a screen capture operation (413). In one embodiment, graphics capture (420) may include a graphics window capture operation (421).

[0081] According to one embodiment, the captured image obtained through the video capture (410) may have a data format of a video image (e.g., a YUV data format). For example, the captured image obtained through the main / sub scaler capture operation (411), the post capture operation (412), and the screen capture operation (413) may have a YUV data format. According to one embodiment, the captured image obtained through the graphics capture (420) may have a data format of a graphics image (e.g., an RGBA data format). For example, the captured image obtained through the graphics window capture operation (421) may have an RGBA data format.

[0082] According to one embodiment, the display device can perform at least one of a main / sub scaler capture operation (411), a post capture operation (412), or a screen capture operation (413) on the video path. For example, the display device can bypass the main / sub scaler capture operation (411) and the post capture operation (412) on the video path and perform the screen capture operation (413). For example, the display device can bypass the main / sub scaler capture operation (411) and the screen capture operation (413) on the video path and perform the post capture operation (412).

[0083] In one embodiment, the display device may optionally perform a graphic window capture operation (421) on the graphic path. For example, if a screen capture operation (413) is performed on the video path, the display device may not perform the graphic window capture operation (421) on the graphic path.

[0084] In one embodiment, the main / sub scaler capture may be a capture method for a video image before scaling processing is performed by a scaler (e.g., scaler (111a) of FIG. 3). The video image before scaling processing may be, for example, a video image itself generated based on input video data. Since the video image is a video image before scaling processing is performed, it may be, for example, a video image displayed on the entire screen area.

[0085] According to one embodiment, in the main / sub scaler capture operation (411), the display device can obtain a scaler capture image by capturing a video image before scaling processing is performed using the main / sub scaler capture method. An example of the scaler capture image may be the scaler capture image (511) of FIG. 5. As illustrated in FIG. 5, the scaler capture image (511) may include a video image displayed on the entire area of ​​the screen.

[0086] According to one embodiment, data of a scaler capture image may be stored in a buffer (hereinafter, “scaler buffer”) corresponding to a main / sub scaler capture operation (411). The scaler capture image stored in the scaler buffer may be used for scaling processing by the scaler. In the present disclosure, the main / sub scaler capture may be referred to as a scaler capture.

[0087] In one embodiment, post-capture may be a capture method for a video image after scaling processing has been performed by a scaler (e.g., scaler (111a) of FIG. 3). The video image after scaling processing may be, for example, a video image generated by adjusting the position and / or size of a video image generated based on input video data. Since the video image is a video image after scaling processing has been performed, it may be, for example, a video image displayed in a portion of the entire screen area.

[0088] According to one embodiment, in the post-capture operation (412), the display device may obtain a post-capture image by capturing a video image after scaling processing is performed using a post-capture method. An example of the post-capture image may be the post-capture image (512) of FIG. 5. As illustrated in FIG. 5, the post-capture image (512) may include a video image displayed in a portion of the screen.

[0089] According to one embodiment, data of a post-capture image may be stored in a buffer (hereinafter, “post-capture buffer”) corresponding to a post-capture operation (412). The post-capture image stored in the post-capture buffer may be used for composite processing with a graphic image.

[0090] In one embodiment, a graphic window capture may be a capture method for a graphic image (or a window containing a graphic image).

[0091] According to one embodiment, in a graphic window capture operation (421), the display device may obtain a graphic window capture image by capturing a graphic image (or a window including a graphic image) using a graphic window capture method. An example of the graphic window capture image may be the graphic window capture image (521) of FIG. 5. As illustrated in FIG. 5, the graphic window capture image may correspond to a graphic image included in one window (or graphic plane).

[0092] According to one embodiment, data of a graphic window capture image may be stored in a buffer (hereinafter, graphic window capture buffer) corresponding to a graphic window capture operation (421). The graphic window capture image stored in the graphic window capture buffer may be used for composite processing with a post image.

[0093] Meanwhile, when multiple graphic images are provided on a single screen through multiple windows (or graphic planes) (e.g., the multi-window case of FIG. 8 or 9), due to performance limitations of the window system, the display device can capture only one window through the graphic window capture operation (421). For example, the display device can capture only the window corresponding to the layer immediately below the window (or graphic plane) in which information of a specific application (e.g., a blur app) associated with the graphic window capture operation (421) is displayed. In other words, windows located in layers below the captured window cannot be captured. That is, in the multi-window case, when the graphic window capture operation (421) is used, an image in which graphic images are synthesized cannot be captured.

[0094] In one embodiment, screen capture may be a method for capturing a composite image generated based on a video image and a graphic image. In one embodiment, screen capture may be a method for capturing the entire screen. For example, screen capture may be a method for capturing the entire screen including the composite image. The composite image may be generated, for example, by mixing a video image and at least one graphic image associated with the video image using a mixer (e.g., mixer (113) of FIG. 3 ).

[0095] According to one embodiment, in the screen capture operation (413), the display device may obtain a screen capture image by capturing a composite image using a screen capture method. In the present disclosure, the screen capture image may also be referred to as a screenshot image. An example of the screen capture image may be the screen capture image (513) of FIG. 5. As illustrated in FIG. 5, the screen capture image (513) may include a composite image (or an entire screen image including the composite image) generated by synthesizing a post-capture image (512) and a graphic window capture image (521). In the composite image, an area where a scaled video image is displayed may correspond to a transparent area of ​​the graphic window.

[0096] According to one embodiment, data of a screen capture image may be stored in a buffer (hereinafter, referred to as a screen capture buffer) corresponding to a screen capture operation (413). The screen capture image stored in the screen capture buffer may be post-processed and used for compositing with a post-capture image. The post-processing of the screen capture image and the compositing with the post-capture image are described below with reference to FIGS. 10 to 13.

[0097] FIG. 6 illustrates changes in an output image according to execution of an application supporting image effect processing according to one embodiment of the present disclosure.

[0098] In one embodiment, the image effect processing may be blur processing. In one embodiment, the blur processing may include an operation of applying a blur effect (hereinafter, a blur effect application operation) and / or an operation of outputting information associated with a blur app (hereinafter, a blur output operation). The blur effect may be, for example, an effect that softens or blurs at least a portion of an image. For example, the blur effect may be an effect that blurs an entire area of ​​an image (e.g., the first image (610)) or a screen (screen) to which the blur effect is applied. Through this blur effect, details within an image can be hidden, or matters displayed on a blurred image can be emphasized.

[0099] Hereinafter, various embodiments of the present disclosure will be described using blur processing as an example of image effect processing, but the present disclosure is not limited thereto. For example, various embodiments of the present disclosure can also be applied to image blurring processes such as filtering, mosaic effect processing, and averaging effect processing. In the present disclosure, an application that supports blur processing may be referred to as a "blur app" or "blur application."

[0100] In one embodiment, a display device (e.g., display device (100) of FIG. 1 or display device (200) of FIG. 2) may initiate blur processing based on an event associated with blur processing being identified. For example, the display device may initiate blur processing in response to a blur app being executed.

[0101] According to one embodiment, blur processing may be performed by a processor (e.g., processor (110) of FIG. 1 / 2) or a GPU (e.g., GPU (112a) of FIG. 3) of the display device.

[0102] Referring to FIG. 6, the first image (610) corresponds to a composite image generated based on a video image (611) and a first graphic image (612).

[0103] The second image (620) corresponds to an image to which a blur effect has been applied to the first image (610). According to one embodiment, the display device can apply a blur effect to the image by mixing pixel values ​​of the image with surrounding pixel values ​​and setting them as new pixel values.

[0104] According to one embodiment, when a blur app is executed while a first image (610) is displayed, the display device may apply a blur effect to the first image (610) to generate a second image (620). For example, in response to the blur app being executed while the first image (610) is displayed, the display device may apply a blur effect to the first image (610) to generate a second image (620). In the present disclosure, the second image (620) to which the blur effect is applied may be referred to as a blur image.

[0105] The third image (630) corresponds to an image that displays a second graphic image (613) associated with a blur app on a second image (620). For example, if the blurred second image (620) is normally generated, the display device may display a second graphic image (631) associated with the blur app on the second image. According to one embodiment, the second graphic image (631) may include at least one piece of graphic information. The at least one piece of graphic information may include at least one piece of menu information, setting information (e.g., graphic window setting information), user guide information, electronic program guide information, or notification information. According to one embodiment, the graphic information may be generated by a processor (e.g., a processor (110) of FIG. 1 / 2) (or a GPU (e.g., a GPU (112a) of FIG. 3)).

[0106] FIG. 7 is a diagram illustrating a blur processing procedure according to an embodiment of the present disclosure. FIG. 8 is a diagram illustrating a blur processing result according to a blur processing procedure in a multi-window case according to an embodiment of the present disclosure. FIG. 9 is a diagram illustrating a blur processing result according to a blur processing procedure in a multi-window case according to an embodiment of the present disclosure.

[0107] Referring to FIG. 7, in operation 710, a display device (e.g., display device (100) of FIG. 1 or display device (200) of FIG. 2) may perform video capture (e.g., video capture (410) of FIG. 4) for a first video image. As an example, the display device may perform a post-capture operation (e.g., post-capture operation (412) of FIG. 4) for a scaled first video image to obtain a post-capture image (e.g., post-capture image (512) of FIG. 5).

[0108] In operation 720, the display device may perform a graphic capture for the graphic image (e.g., a graphic capture (420) of FIG. 4). For example, the display device may perform a graphic window capture for the graphic image (e.g., a graphic window capture operation (421)) to obtain a graphic window capture image (e.g., a graphic window capture image (521) of FIG. 5). Operation 720 may be performed in parallel with operation 710. For example, operation 720 may be performed simultaneously with or together with operation 710.

[0109] In operation 730, the display device may generate a first composite image based on the post-capture image and the graphic window capture image. For example, the display device may generate the first composite image by mixing the post-capture image and the graphic window capture image using a mixer (e.g., mixer (113) of FIG. 3). For example, the display device may generate the first composite image by overlaying the post-capture image and the graphic window capture image. For example, the display device may generate the composite image by positioning the post-capture image below the graphic window capture image. In this case, an area in which a video image is displayed in the post-capture image may correspond to a portion of the graphic image (e.g., a transparent area). Accordingly, even if the video image is positioned below the graphic image in the composite image, the video image can be displayed to a user of the display device. The first composite image may be, for example, the same image as the screen capture image (513) of FIG. 5.

[0110] In operation 740, the display device may perform an operation of applying a blur effect to the first composite image. In operation 750, the display device may perform a blur output operation on the first composite image to which the blur effect has been applied. For a description of the blur effect application operation and the blur output operation, as well as a description of the resulting image, refer to the description of FIG. 6. Therefore, any duplicate description will be omitted.

[0111] Thereafter, the display device may repeatedly perform operations 710 to 750 for video image(s) following the first video image. The video image(s) following the first video image may be, for example, video image(s) set to be displayed later in time than the first video image. For example, the video image(s) following the first video image may belong to the same video stream as the first video image and may be video image(s) set to be displayed later in time than the first video image. At this time, the graphic window capture image obtained through the above-described operation 720 may be used identically for the associated video image(s) without change. In this way, the graphic image obtained through a single graphic window capture may be repeatedly used for the associated video images to perform blur processing on the real-time video image. The associated video images may each be, for example, video images set to be sequentially displayed on a single screen together with the graphic window capture image.

[0112] Meanwhile, as described above, the graphic capture method (e.g., the graphic window capture method) can only capture one window (or graphic plane) due to limitations in software performance (e.g., the window system performance). For example, the display device can only capture the window (or graphic plane) corresponding to the layer immediately below the layer associated with the blur app through the graphic window capture. In other words, windows located in layers below the captured window cannot be captured.

[0113] For example, as in FIG. 8, multiple graphic images may be set to be provided on a single screen through multiple windows (or graphic planes) (multi-window case). For example, a first window (810) providing information related to a blur app, a second window (820) located in a layer below the first window (810) and providing a notification, a third-first window (831) located in a layer below the second window (820) and providing a TV viewer, a third-second window (832) providing a text viewer, and a fourth window (840) located in a layer below the third-first window (831) and the third-second window (832) and providing a multi-view may be set to be provided on a single screen. In this case, the display device can capture only the second window (820) (or, a graphic image (e.g., a graphic image including a notification) included in the second window (820)) located in the layer immediately below the first window (810) among the multiple windows. At this time, the second window (820) is a window displayed in a partial area, not the entire area of ​​the screen. Therefore, when a blur effect is applied to the window, the blur effect is not applied to the entire screen as originally intended, but only to a partial area of ​​the entire screen. Therefore, the blur processing result originally intended in the multi-window case cannot be provided.

[0114] For example, as in FIG. 9, multiple graphic images may be set to be provided on the screen through multiple windows (or graphic planes) (multi-window case). For example, a first window (910) providing information related to a blur app, a second window (920) located in a layer below the first window (910) and providing a menu, a third window (930) located in a layer below the second window (920) and providing a guide app including a transparent hole (931), and a fourth window (940) located in a layer below the third window (930) and providing a TV viewer may be set to be provided on one screen. In this case, the display device may capture only the second window (920) located in the layer immediately below the first window (910) among the multiple windows (or a graphic image (e.g., a graphic image providing a menu) included in the second window (920)). At this time, the second window (920) is a window that is displayed not on the entire screen area, but on a portion of the screen area. Therefore, when applying a blur effect to this window, the blur effect is not applied to the entire screen area as originally intended, but only to a portion of the entire screen area. Therefore, the originally intended blur processing result cannot be provided in the multi-window case.

[0115] In this way, when applying blur processing using a synthetic image based on a captured graphic window image, it cannot provide the intended blurring result for the entire screen in a multi-window scenario. Therefore, a new blurring procedure is needed that can provide the intended blurring result for the entire screen even in a multi-window scenario. This new blurring procedure can, for example, utilize a screen capture image.

[0116] FIG. 10 is a diagram illustrating a blur processing procedure according to an embodiment of the present disclosure. FIG. 11 is a diagram illustrating a blur processing procedure according to an embodiment of the present disclosure. FIG. 12 illustrates changes in an image resulting from processing a screen capture image in a blur processing procedure according to an embodiment of the present disclosure. FIG. 13 is a diagram illustrating an image obtained in a blur processing procedure according to an embodiment of the present disclosure.

[0117] In one embodiment, a display device (e.g., display device (100) of FIG. 1 or display device (200) of FIG. 2) may initiate a blur processing procedure based on an event related to blur processing being identified. For example, the display device may initiate operation 1010 in response to a blur app being executed.

[0118] Referring to FIGS. 10 and 11, at operations 1010 and 1110, the display device may perform a screen capture operation (e.g., screen capture operation (413) of FIG. 4) for the first composite image.

[0119] In one embodiment, operations 1010 and 1110 may be performed before the blur app (or graphic information related to the blur app) is displayed on the screen. For example, operations 1010 and 1110 may be performed after the blur app is launched, but before the blur app is displayed on the screen. If a screen capture is performed after the blur app is displayed on the screen, it is not possible to apply a blur effect to lower layers of the blur app because the blur app is included in the screen capture image along with the first composite image.

[0120] According to one embodiment, the display device can obtain a screen capture image (e.g., a screen capture image (513) of FIG. 5, a screen capture image (1210) of FIG. 12) by capturing a first composite image using a screen capture method. Data of the obtained screen capture image can be stored in a screen capture buffer.

[0121] According to one embodiment, the first composite image may be generated based on the first video image and at least one graphic image. For example, the first composite image may be generated by mixing the first video image and at least one graphic image. For example, the first composite image may be generated by overlaying the first video image and at least one graphic image.

[0122] According to one embodiment, the first video image may be a scaled video image. For example, the first video image may be a video image scaled by a scaler (e.g., scaler (111a) of FIG. 3). For a description of the scaling process, refer to the description of FIG. 3. Therefore, any duplicate description will be omitted.

[0123] In one embodiment, at least one graphic image may be associated with the first video image. For example, the at least one graphic image may be a graphic image configured to be displayed on a single screen together with the first video image (or each video image within a first group of video images that includes the first video image).

[0124] According to one embodiment, the at least one graphic image may include a first graphic image (e.g., a graphic image of the second window (920) of FIG. 9) and a second graphic image (e.g., a graphic image of the third window (930) of FIG. 9).

[0125] In one embodiment, the first graphic image and the second graphic image may be generated on different graphic planes. For example, the first graphic image may be generated on a first graphic plane, and the second graphic image may be generated on a second graphic plane that is a sub-graphic plane of the first graphic plane.

[0126] In one embodiment, the first graphic image and the second graphic image may be provided on different windows. For example, the first graphic image may be provided through a first window, and the second graphic image may be provided through a second window. In one embodiment, the first window and the second window may be present on the same graphic plane or on different graphic planes.

[0127] According to one embodiment, the screen capture image may include a video area corresponding to the first video image and a graphic area corresponding to at least one graphic image associated with the first video image. For example, as illustrated in FIG. 12 , the first image (1210) may include a video area (1211) corresponding to the first video image and a graphic area (1212) corresponding to at least one graphic image associated with the first video image.

[0128] In one embodiment, the screen capture image may have a first data format. The first data format may be a data format that does not include a transparency element indicating transparency. As an example, the first data format may be, but is not limited to, a YUV data format.

[0129] In operation 1011, the display device may perform post-processing on the screen capture image. In one embodiment, the post-processing operation on the screen capture image may include an operation for making a video area within the screen capture image transparent.

[0130] In one embodiment, the video area within the screen capture image may be associated with the first video image. For example, the video area within the screen capture image may be identical to the area where the first video image is displayed. For example, the video area within the screen capture image may include the area where the first video image is displayed.

[0131] According to one embodiment, when the screen capture image has a first data format that does not include a transparency element, the display device may perform operations 1111 and 1112 of FIG. 11.

[0132] In operation 1111, the display device may change the data format for the screen capture image from the first data format to a second data format including a transparency element. For example, before performing transparency processing on the video area, the display device may change the data format for the screen capture image from the first data format to a second data format including a transparency element. The second data format may be a data format including a transparency element. As an example, the second data format may be, but is not limited to, an RGBA data format. An example of a format-converted screen capture image may be the same as the second image (1220) of FIG. 12. Since only the format of the data is changed through the processing of operation 1111, the first image (1210) and the second image (1220) correspond to visually identical images.

[0133] In operation 1112, the display device may make a video area within a screen capture image transparent. For example, after the data format is changed, the display device may identify a video area within the screen capture image based on data of the second data format, and may adjust a value of a transparency element for the video area within the screen capture image to make the video area transparent. For example, the display device may make the video area transparent by adjusting an alpha value for pixels corresponding to the video area in the data of the second data format for the screen capture image to 0. An example of a screen capture image in which a video area is identified may be the third image (1230) of FIG. 12. An example of a screen capture image including a transparentized video area may be the fourth image (1240) including a transparentized video area (1241) of FIG. 12.

[0134] According to one embodiment, when the screen capture image has a second data format including a transparency element, the display device can make the video area transparent by identifying a video area within the screen capture image based on data of the second data format without changing the data format, and adjusting a value of the transparency element for the video area within the screen capture image. For example, the display device can make the video area transparent by adjusting an alpha value for pixels corresponding to the video area in data of the second data format for the screen capture image to 0.

[0135] In one embodiment, data of a post-processed screen capture image may be stored in a screen capture buffer. For example, data of a screen capture image in which a video area has been made transparent may be stored in the screen capture buffer.

[0136] In operations 1020 and 1120, the display device may perform a video capture (e.g., video capture (410) of FIG. 4) for a second video image. In one embodiment, the second video image may be the same video image as the first video image used to obtain the screen capture image, or may be a video image subsequent to the first video image (e.g., one of the video images within a first video image group that includes the first video image). The second video image may be, for example, a graphic image configured to be displayed on a single screen together with at least one graphic image associated with the first video image.

[0137] According to one embodiment, the second video image may be a scaled video image. For example, the second video image may be a video image scaled by a scaler (e.g., scaler (111a) of FIG. 3). For a description of the scaling process, refer to the description of FIG. 3. Therefore, a duplicate description will be omitted.

[0138] According to one embodiment, the display device can obtain a first video capture image (e.g., the first video capture image (1310) of FIG. 13) by capturing a second video image using a video capture method. For example, the display device can obtain a first post-capture image (e.g., the post-capture image (512) of FIG. 5) by capturing a second video image using a post-capture method. Data of the obtained first video capture image (e.g., the first post-capture image) can be stored in a video capture buffer (e.g., the post-capture buffer). The data of the first video capture image stored in the video capture buffer can be used, together with data of a post-processed (e.g., a video area of ​​which is processed to be transparent) screen capture image stored in a screen capture buffer, to generate a composite image based on the first video capture image and the post-processed screen capture image.

[0139] In operations 1030 and 1130, the display device may perform composite processing on the first video capture image and the post-processed screen capture image.

[0140] According to one embodiment, the display device can generate a second composite image based on a first video capture image (e.g., a first post-capture image) and a post-processed screen capture image (e.g., a screen capture image with a video region made transparent). For example, the display device can generate the second composite image based on the first post-capture image and the screen capture image with a video region made transparent. For example, the display device can generate the second composite image by synthesizing (or combining) the first post-capture image and the screen capture image with a video region made transparent. For example, the display device can generate the second composite image by mixing the first post-capture image and the screen capture image with a video region made transparent. For example, the display device can generate the second composite image by overlaying the first post-capture image and the screen capture image with a video region made transparent.

[0141] For example, as illustrated in FIG. 13, the display device can generate the second composite image (1330) by placing the first post-capture image (1310) below the screen capture image (1320) in which the video area is made transparent. In this case, the area in which the video image is displayed in the post-capture image (1310) can correspond to a portion (e.g., the transparent area) of the screen capture image (1320) in which the video area is made transparent. Accordingly, even if the video image is placed below the graphic image in the composite image (1330), the video image can be displayed to the user of the display device.

[0142] In one embodiment, a layer of a video plane (or window) including a first video capture image (e.g., a first post-capture image) may be a lower layer of a layer of a plane (or window) including a post-processed screen capture image (e.g., a screen capture image with a video area made transparent). In one embodiment, a scaled second video image within the first video capture image (e.g., the first post-capture image) may be positioned within a video area made transparent within the second composite image. In this way, even if the first video capture image is set as a lower layer of the post-processed screen capture image, the scaled second video image within the first video capture image may be displayed to a user of the display device through the video area made transparent.

[0143] In operations 1040 and 1140, the display device may perform a blur effect application operation on the second composite image (1330) to generate a second composite image (1340) to which the blur effect is applied. In operations 1050 and 1150, the display device may perform a blur output operation on the second composite image (1340) to which the blur effect is applied. The display device may output a blur app image (1350) that displays graphic information associated with a blur app that supports the blur processing on the second composite image (1340) to which the blur effect is applied. According to one embodiment, at least a portion of the blurred second composite image (1340) may be displayed on the display. For a description of the blur effect application operation and the blur output operation and a description of the resulting image, refer to the description of FIG. 6. Therefore, redundant descriptions are omitted.

[0144] Thereafter, the display device can repeat operations 1020 to 1050 (or operations 1120 to 1150) for video image(s) following the second video image. The video image(s) following the second video image may be, for example, video image(s) that are set to be displayed later in time than the second video image. For example, the video image(s) following the second video image may be video image(s) that belong to the same video stream as the first video image and the second video image and are set to be displayed later in time than the first video image and the second video image. In this case, the display device can use the post-processed screen capture image (or screen capture image in which the video area is processed as transparent) obtained through the above-described operation 1011 (or operations 1111 and 1112) in the same manner for the associated video image(s) without change. The associated video image(s) may be, for example, a graphic image that follows the second video image and is configured to be displayed on a single screen together with at least one graphic image associated with the first video image (or the second video image). In this way, a screen capture image obtained through a single screen capture can be repeatedly used for the associated video images to perform blur processing on the real-time video image.

[0145] Meanwhile, unlike the blur processing procedure using the graphic window capture method, in the blur processing procedure using the screen capture method, a screen capture image that captures the entire screen, including a composite image generated based on the video window and the associated graphic window, is used for blur processing. This screen capture image can include all graphic images of the multi-window. Therefore, even when multiple graphic images are set to be provided on one screen through multiple windows (or graphic planes) (multi-window case), the blur effect can be applied to the entire screen rather than just one window. Therefore, even in the multi-window case, the originally intended blur effect can be applied to the entire screen.

[0146] For example, as in FIG. 8, a first window (810) providing information related to a blur app, a second window (820) located in a layer below the first window (810) and providing a notification, a third-first window (831) located in a layer below the second window (820) and providing a TV viewer, a third-second window (832) providing a text viewer, and a fourth window (840) located in a layer below the third-first window (831) and the third-second window (832) and providing a multi-view may be set to be provided through one screen. In this case, the screen capture image may include an entire screen image including all of the graphic images of the second window (820), the third-first window (831), the third-second window (832), and the fourth window (840). Therefore, when a blur effect is applied to the corresponding screen capture image, the blur effect may be applied to the entire screen as originally intended. Therefore, it can provide the originally intended blur processing results even in multi-window cases.

[0147] For example, as shown in FIG. 9, a first window (910) providing information related to a blur app, a second window (920) located in a layer below the first window (910) and providing a menu, a third window (930) located in a layer below the second window (920) and providing a guide app including a transparent hole (931), and a fourth window (940) located in a layer below the third window (930) and providing a TV viewer can be set to be provided on a single screen. In this case, the screen capture image can include an entire screen image including all of the graphic images of the second window (920), the third window (930), and the fourth window (940). Therefore, when a blur effect is applied to the corresponding screen capture image, the blur effect can be applied to the entire screen as originally intended. Therefore, even in a multi-window case, the originally intended blur processing result can be provided.

[0148] FIG. 14 is a flowchart illustrating a method of a display device according to one embodiment of the present disclosure.

[0149] Referring to FIG. 14, a display device (e.g., display device (100) of FIG. 1 or display device (200) of FIG. 2) may obtain a screen capture image having a first data format by capturing a first composite image generated based on a first video image and at least one graphic image using a screen capture method (14010). According to one embodiment, the first data format may not include a transparency element indicating transparency.

[0150] The display device can change the data format for the screen capture image from the first data format to a second data format including the transparency element (14020).

[0151] The display device can process the video area as transparent by adjusting the value of the transparency element for the video area associated with the first video image in the screen capture image based on the data of the second data format (14030).

[0152] The display device may generate a second composite image based on a screen capture image including the first video image or a second video image following the first video image and the transparently processed video area (14040). In one embodiment, an area in the second composite image where the first video image or the second video image is displayed may be included within the transparently processed video area.

[0153] The display device can perform blur processing on the second composite image (14050).

[0154] According to one embodiment, the at least one graphic image includes a first graphic image and a second graphic image, and a window including the first graphic image and a window including the second graphic image may be located in different layers.

[0155] According to one embodiment, obtaining the screen capture image may be performed before a blur application associated with the blur processing or graphic information associated with the blur application is displayed on the screen.

[0156] According to one embodiment, the first data format may be a YUV data format, and the second data format may be an RGBA data format.

[0157] According to one embodiment, in order to process the video area as transparent, the display device can adjust the alpha value for pixels corresponding to the video area in the data of the second data format to 0.

[0158] According to one embodiment, the second video image corresponds to a post-capture image obtained by capturing a scaled video image for synthesis with the at least one graphic image using a video capture method, and the second video image may have the first data format.

[0159] In one embodiment, the second composite image may be generated by positioning the second video image in a lower layer of a layer in which a screen capture image including the transparently processed video area is positioned.

[0160] According to one embodiment, to perform blur processing on the second composite image, the display device may apply a blur effect to the second composite image and output graphic information associated with a blur application that supports the blur processing on the second composite image to which the blur effect is applied.

[0161] According to one embodiment, after performing blur processing on the second composite image, the display device can generate a third composite image by synthesizing a third video image following the second video image and a screen capture image including the transparently processed video area, and perform blur processing on the third composite image.

[0162] According to one embodiment, the at least one graphic image may be set to be sequentially displayed through one screen with each of the first video image, the second video image, and the third video image.

[0163] According to one embodiment, the graphic image includes at least one graphic information, and the at least one graphic information is included within a graphic area located outside the video area within the screen capture image, and may include at least one of user interface, menu information, setting information, user guide information, electronic program guide information, or notification information.

[0164] In one embodiment, the first video image and the graphic image may have different data formats, and the first video image and the screen capture image may have the same data format.

[0165] According to one embodiment, the display device can identify an event related to the blur processing, and in response to the event related to the blur processing being identified, obtain a screen capture image having the first data format.

[0166] In the specific embodiments of the present disclosure described above, components included in the present disclosure are expressed in the singular or plural form, depending on the specific embodiment presented. However, the singular or plural expressions are selected to suit the presented situation for convenience of explanation, and the present disclosure is not limited to singular or plural components. Components expressed in the plural form may be composed of singular elements, or components expressed in the singular form may be composed of plural elements.

[0167] While the detailed description of this disclosure has described specific embodiments, it should be understood that various modifications are possible without departing from the scope of this disclosure. Therefore, the scope of this disclosure should not be limited to the described embodiments, but should be defined not only by the scope of the claims described below, but also by equivalents thereof.

Claims

1. In the display device, display, at least one processor, and A memory storing at least one instruction, wherein the at least one instruction, when executed by the at least one processor, causes the display device to: A first composite image generated based on a first video image and at least one graphic image is captured using a screen capture method, thereby obtaining a first screen capture image having a first data format, wherein the first data format does not include a transparency element indicating transparency, Changing the data format for the first screen capture image from the first data format to a second data format including the transparency element, By adjusting the value of the transparency element for the video area associated with the first video image in the first screen capture image based on the data of the second data format, the video area is processed to be transparent, A second composite image is generated based on the first video image or the second video image following the first video image and the second screen capture image including the transparently processed video area, and an area in the second composite image where the first video image or the second video image is displayed is included in the transparently processed video area, Performing blur processing on the second composite image, and A display device configured to control the display to display at least a portion of the blurred second composite image.

2. In paragraph 1, A display device, wherein the at least one graphic image includes a first graphic image and a second graphic image, and a window including the first graphic image and a window including the second graphic image are located in different layers.

3. In any one of paragraphs 1 and 2, The above first data format is a YUV data format, and the above second data format is an RGBA data format. A display device, wherein, to make the video area transparent, the at least one command, when executed by the at least one processor, causes the display device to: adjust alpha values ​​for pixels corresponding to the video area in the data of the second data format to 0.

4. In paragraph 1, A display device wherein the second video image corresponds to a post-capture image obtained by capturing a scaled video image for compositing with the at least one graphic image using a video capture method, and the second video image has the first data format.

5. In paragraph 3, A display device wherein the second composite image is generated by positioning the second video image in a lower layer of the layer containing the second screen capture image.

6. In the first paragraph, to perform blur processing on the second composite image, the at least one command, when executed by the at least one processor, causes the display device to: Apply a blur effect to the above second composite image, A display device that outputs graphic information associated with a blur application that supports the blur processing on the second composite image to which the blur effect is applied.

7. In the first paragraph, after performing blur processing on the second composite image, the at least one command, when executed by the at least one processor, causes the display device to: A third composite image is generated by synthesizing a third video image following the second video image and a screen capture image including the transparently processed video area, Perform blur processing on the third composite image above, A display device, wherein the at least one graphic image is set to be sequentially displayed through one screen with each of the first video image, the second video image, and the third video image.

8. In paragraph 1, The first video image and the at least one graphic image have different data formats, A display device, wherein the first video image and the first screen capture image have the same data format.

9. In the first paragraph, the at least one command, when executed by the at least one processor, causes the display device to: Identify events related to the above blur processing, A display device, which obtains the first screen capture image based on the identification of an event related to the blur processing.

10. In the method of the display device, An operation of obtaining a first screen capture image having a first data format by capturing a first composite image generated based on a first video image and at least one graphic image using a screen capture method, wherein the first data format does not include a transparency element indicating transparency; An action of changing a data format for the first screen capture image from the first data format to a second data format including the transparency element; An operation of processing the video area as transparent by adjusting the value of a transparency element for the video area associated with the first video image in the first screen capture image based on data of the second data format; An operation of generating a second composite image based on a second screen capture image including the first video image or a second video image following the first video image and the transparently processed video area, wherein an area in the second composite image where the first video image or the second video image is displayed is included within the transparently processed video area; Performing blur processing on the second composite image, and A method comprising the action of displaying at least a portion of the blurred second composite image.

11. In paragraph 10, A method wherein at least one graphic image comprises a first graphic image and a second graphic image, and a window comprising the first graphic image and a window comprising the second graphic image are located in different layers.

12. In clause 10 or 11, The above first data format is a YUV data format, and the above second data format is an RGBA data format. A method wherein the operation of making the above video area transparent includes an operation of adjusting the alpha value for pixels corresponding to the video area in the data of the second data format to 0.

13. In any one of paragraphs 10 to 12, A method according to claim 1, wherein the operation of obtaining the first screen capture image is performed before a blur application associated with the blur processing or graphic information associated with the blur application is displayed on the screen.

14. In paragraph 10, A method wherein the second composite image is generated by positioning the second video image in a sublayer of a layer containing the second screen capture image.

15. In the 10th paragraph, the operation of performing blur processing on the second composite image is: An operation of applying a blur effect to the second composite image; and A method comprising an operation of outputting graphic information associated with a blur application supporting the blur processing on the second composite image to which the blur effect is applied.

Images