Display device for image quality processing using look-up table, and method therefor

Abstract

The present disclosure provides a display device for image quality processing using a look-up table (LUT), and a method therefor. The display device performs LUT generation and image quality processing by using a first mode generating, on the basis of data transmitted from a decoder to a buffer queue and stored therein, a LUT for image quality processing, switches to a second mode when it is identified that a first mode switching condition is satisfied while using the first mode, so as to perform LUT generation and image quality processing by using the second mode generating, on the basis of data received from the decoder, a LUT for image quality processing, and switches back to the first mode when it is identified that a second mode switching condition is satisfied while using the second mode.

Description

Display device and method for image quality processing using a lookup table

[0001] The present disclosure relates to a display device and method for image processing.

[0002] Recently, various techniques are being utilized in digital image processing technology to maximize the user experience. One such method is image quality enhancement using look-up tables (LUTs). Through this LUT-based enhancement, it is possible to improve the visual quality of video content while simultaneously increasing processing speed.

[0003] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0004] According to one embodiment of the present disclosure, a display device may be provided. The display device may include: a display; a memory for storing at least one instruction; and at least one processor for executing at least one instruction stored in the memory. The at least one processor may generate a first Look Up Table (LUT) for image quality processing for a first frame based on data stored in a buffer queue using a first mode, wherein the data includes first metadata corresponding to the first frame transmitted from a decoder to the buffer queue or information related to the first metadata, switch to a second mode based on identifying that a first mode switching condition related to state information of the buffer queue is satisfied while using the first mode, and generate a second LUT for image quality processing for the second frame based on second metadata corresponding to the second frame received from the decoder using the second mode.

[0005] According to one embodiment, at least one processor may switch to the first mode based on identifying that a second mode switching condition related to the state information of the buffer queue is satisfied while using the second mode.

[0006] According to one embodiment, the first mode switching condition is satisfied when the capacity of data accumulated in the buffer queue and / or the number of metadata indices accumulated in the buffer queue exceeds a first threshold, and the second mode switching condition may be satisfied when the capacity of data accumulated in the buffer queue and / or the number of metadata indices accumulated in the buffer queue is less than a second threshold.

[0007] According to one embodiment, the first mode switching condition may be satisfied if the value obtained by dividing the capacity of data output from the buffer queue during a unit time by the capacity of data input to the buffer queue during the unit time exceeds a fifth threshold.

[0008] According to one embodiment, at least one processor may periodically check the status information of the buffer queue or check it based on the occurrence of a specific event. The status information may include at least one of the capacity of data accumulated in the buffer queue, the number of indices of metadata accumulated in the buffer queue, the capacity of data input into the buffer queue per unit time, or the capacity of data output from the buffer queue per unit time.

[0009] According to one embodiment, when at least one processor performs an operation to generate the second LUT using the second mode, it may allocate more processor resources or assign a higher processor priority to the operation to generate the LUT compared to when it performs an operation to generate the first LUT using the first mode.

[0010] According to one embodiment, at least one processor can secure storage space of the buffer queue by processing data accumulated in the buffer queue while using the second mode.

[0011] According to one embodiment, when at least one processor uses a second mode, it can perform the second LUT generation operation in response to a rendering request.

[0012] According to one embodiment, the first LUT and the second LUT may be used for at least one of a gamma correction process, a color mapping process, a color correction process, a tone mapping process, a white balance adjustment process, or a dithering process.

[0013] According to one embodiment of the present disclosure, a method for operating a display device may be provided. The method for operating a display device may include: generating a first LUT for image quality processing for a first frame based on data stored in a buffer queue using a first mode, wherein the data includes first metadata corresponding to the first frame transmitted from a decoder to the buffer queue or information related to the first metadata; switching to a second mode based on identifying that a first mode switching condition related to state information of the buffer queue is satisfied while using the first mode; and / or generating a second LUT for image quality processing for a second frame based on second metadata corresponding to the second frame transmitted from the decoder using the second mode.

[0014] According to one embodiment, the method of operating the display device may further include the operation of switching to the first mode based on identifying that a second mode switching condition related to the state information of the buffer queue is satisfied while using the second mode.

[0015] According to one embodiment, a method for operating a display device may further include an operation of periodically checking the status information of the buffer queue.

[0016] According to one embodiment, a method for operating a display device may further include an operation of checking the status information of the buffer queue based on the occurrence of a specific event.

[0017] According to one embodiment, the operation of generating the second LUT using the second mode may include: an operation of allocating more processor resources or assigning a higher processor priority to the operation for generating the LUT compared to the case where the first mode is used; and / or an operation of securing storage space in the buffer queue by processing data accumulated in the buffer queue.

[0018] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0019] FIG. 1 is a block diagram for explaining the configuration of a display device of the present disclosure according to one embodiment of the present disclosure.

[0020] FIG. 2 is a drawing for explaining the operation of a first mode according to one embodiment of the present disclosure.

[0021] FIG. 3a is a diagram illustrating a case according to one embodiment of the present disclosure in which the speed at which data is input into a buffer queue is faster than the speed at which it is output.

[0022] FIG. 3b is a diagram showing data input and output to a buffer queue according to one embodiment of the present disclosure.

[0023] FIG. 4 is a drawing for explaining the operation of a second mode according to one embodiment of the present disclosure.

[0024] FIG. 5 is a drawing for explaining the operation of a display device according to one embodiment of the present disclosure.

[0025] FIG. 6 is a flowchart illustrating the operation of a display device switching to a second mode or maintaining the first mode while using a first mode, according to one embodiment of the present disclosure.

[0026] FIG. 7 is a flowchart illustrating the operation of a display device switching to a first mode or maintaining a second mode while using a second mode, according to one embodiment of the present disclosure.

[0027] FIG. 8 is a flowchart for explaining the operation of a display device including mode selection according to one embodiment of the present disclosure.

[0028] Hereinafter, embodiments of the present disclosure are described in detail with reference to the drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and should be understood to include various modifications, equivalents, or substitutions of the embodiments described herein, rather than being limited to the embodiments described herein. It should be understood that various modifications of the present disclosure are possible by those skilled in the art without departing from the gist of the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

[0029] In relation to the description of the drawings, identical or similar reference numerals may be used for identical or similar components. Additionally, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0030] Meanwhile, various elements and areas in the drawings are depicted schematically, and the technical concept of the present disclosure is not limited by the relative sizes, spacing, or arrangements depicted in the attached drawings. The display device of the present disclosure is not limited to the configuration and / or operation shown in the drawings and may include all other configurations capable of performing the same or similar functions by executing at least one command.

[0031] The individual components depicted in the drawings are not necessarily physically separated but are separated to aid in the explanation and understanding of the present disclosure. The present disclosure may include configurations in which the individual components shown in the drawings are merged, modified, or have some components deleted and / or added. Likewise, the operations depicted in the drawings are illustrative for the purpose of explanation and understanding, and the present disclosure may be modified by a person skilled in the art by merging the individual operations depicted in the drawings, changing the order, or deleting and / or adding parts of the operations.

[0032] FIG. 1 illustrates a block configuration of a display device according to one embodiment of the present disclosure.

[0033] The display device (100) of FIG. 1 may be a smartphone, tablet PC, PC, smart TV, mobile phone, PDA (personal digital assistant), laptop, media player, micro server, digital broadcasting terminal, navigation, kiosk, home appliance, and other mobile or non-mobile computing device, but is not limited thereto. Additionally, the display device (100) may perform various computing functions such as real-time video viewing and communication. In the following description, the display device (100) is described on the premise that it is a TV or a monitor, but this is merely an example and the embodiments of the present disclosure may be equally applied to other electronic devices having display functions.

[0034] According to one embodiment, the display device (100) may include at least one processor (110), an image input unit (120), a memory (130), and a display (140).

[0035] According to one embodiment, the memory (130) includes at least one storage medium used by the display device (100) and can store data such as at least one instruction (131) or setting information corresponding to at least one program. The program may include an operating system (OS) program and various application programs. When the instruction (131) stored in the memory (130) is executed by at least one processor (110), it can cause the display device (100) to perform at least one operation.

[0036] In one embodiment, the memory (130) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), RAM (random access memory, RAM), SRAM (static random access memory), ROM (read only memory, ROM), EEPROM (electrically erasable programmable ROM), PROM (programmable ROM), magnetic memory, a magnetic disk, and an optical disk.

[0037] According to one embodiment, a buffer queue may be statically and / or dynamically allocated on memory (130). The buffer queue is a storage space for temporarily storing data and may have, for example, a First-In-First-Out structure.

[0038] According to one embodiment, the buffer queue may store data for accessing data stored in another part of the memory (130). For example, the buffer queue may store an index for accessing data stored in another part of the memory (130). The index may serve to point to a memory address or a specific data block, thereby enabling the data to be searched and processed quickly.

[0039] According to one embodiment, the video input unit (120) may receive video data through a tuner, an input / output unit, and / or a communication unit. The video input unit (120) may include at least one of the tuner and the input / output unit. The tuner may select only the frequency of the broadcast channel to be received by the display device (100) from among many radio wave components by tuning, mixing, resonance, etc., a broadcast signal received via wired or wireless means. The broadcast signal may include video, audio, and additional data (e.g., EPG (Electronic Program Guide)). The tuner may receive broadcast channels (or viewing video) from various broadcast sources such as terrestrial broadcasting, cable broadcasting, satellite broadcasting, internet broadcasting, etc. The tuner may be implemented as an integral unit with the display device (100) or as a separate tuner electrically connected to the display device (100). The above input / output unit may include at least one of an HDMI (High Definition Multimedia Interface) input port, a component input jack, a PC input port, and a USB input jack, which can receive video data from an external device of the display device (100) under the control of the processor (110). The above communication unit may provide a wired / wireless communication interface that enables communication with an external device and may include at least one of a wired Ethernet, a wireless LAN communication unit, and a short-range communication unit. Depending on the performance and structure of the display device (100), the above input / output unit may be added, deleted, and / or changed.

[0040] According to one embodiment, the display (140) can perform functions for outputting information in the form of numbers, characters, images, and / or graphics. The display (140) may include at least one hardware module for output. The at least one hardware module may include, for example, at least one of an LCD (Liquid Crystal Display), an LED (Light Emitting Diode), an LPD (Light Emitting Polymer Display), an OLED (Organic Light Emitting Diode), an AMOLED (Active Matrix Organic Light Emitting Diode), or an FLED (Flexible LED). The display (140) can display a screen corresponding to data received from the processor (110). The display (140) may be referred to as an 'output unit', a 'display unit', or other terms having an equivalent technical meaning.

[0041] According to one embodiment, the processor (110) can perform control, computation, and / or data processing of at least a portion of the display device (100) by executing at least one instruction (131) stored in memory (130). The processor (110) may include at least one processing circuit that executes at least one instruction stored in memory and may be referred to as 'at least one processor (110)'.

[0042] According to one embodiment, at least one processor (110) may include various processing circuits and / or multiple processors. One or more of the at least one processor (110) may be configured to perform various functions described in the present disclosure individually and / or collectively. Where in the present disclosure, "processor," "at least one processor," or "one or more processors" is described as being configured to perform various functions, these terms may cover, for example, a situation in which one processor performs some of the cited functions and other processor(s) perform other parts of the cited functions, and may also cover, but are not limited to, a situation in which a single processor can perform all of the cited functions. Additionally, at least one processor (110) may include a combination of processors performing the cited / disclosed various functions in a distributed manner, for example. At least one processor (110) may execute program instructions to achieve or perform various functions.

[0043] According to one embodiment, at least one processor (110) may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), a micro controller unit (MCU), a sensor hub, a supplementary processor, a communication processor, an application processor, an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA), and may have multiple cores.

[0044] According to one embodiment, at least one processor (110) may select and use at least one of a plurality of different modes by executing at least one instruction (131), or may use a different mode by switching the currently used mode to another mode. The different modes may include different operation steps and / or sequences up to the generation of a LUT for image processing. Using a specific mode may be understood as generating a LUT for image processing according to a specific operation step and / or sequence. In the present disclosure, a mode may be referred to as a LUT generation mode.

[0045] In the following description, the entity using the mode may be expressed as a “display device,” “at least one processor,” and / or “processor,” or omitted.

[0046] According to one embodiment, the different modes may include a first mode that includes an operation using a buffer queue and a second mode that does not include an operation using a buffer queue. The operation using a buffer queue may mean, for example, an operation including at least one of an operation to create a buffer queue or allocate storage space, an operation to store data in a buffer queue, and an operation to utilize data stored in a buffer queue. The operation to utilize data stored in a buffer queue may, for example, include an operation in which at least one processor (110) accesses data stored in another part of memory (130) using an index stored in a buffer queue.

[0047] The above LUT refers to a reference table used to quickly obtain output values ​​for input data, which stores pre-calculated values. For example, in image quality processing technology, an LUT may be a table capable of correcting the color and brightness of an input video signal in real time based on pre-defined color mapping information.

[0048] According to one embodiment, the input data may include metadata. Metadata may include various additional information related to the video, such as the shooting environment, camera settings, and lighting conditions. In the video, corresponding metadata may exist for each frame.

[0049] According to one embodiment, at least one processor (110) can generate a suitable LUT for a corresponding frame using each metadata by executing at least one instruction (131) and apply it to image quality processing. For example, dynamically applying a LUT based on metadata enables image quality improvement suitable for the characteristics of each frame, and can be effective, for example, in HDR (High Dynamic Range) content or high-resolution video of 4K or higher.

[0050] According to one embodiment, the image quality processing may include one or more of gamma correction processing, color mapping processing, color correction processing, tone mapping processing, white balance adjustment processing, dithering processing, or other processing for image quality improvement.

[0051] FIG. 2 is a drawing for explaining the configuration and operation when a display device (e.g., the display device (100) of FIG. 1) according to one embodiment of the present disclosure uses a first mode.

[0052] Referring to FIG. 2, a display device (e.g., the display device (100) of FIG. 1) may include an image input unit (120), a decoder (210), a buffer queue (220), an LUT generation unit (230), an image quality processing unit (240), and / or a display (140). According to one embodiment, the decoder (210), the LUT generation unit (230), and / or the image quality processing unit (240) may be implemented as at least one processor (e.g., the processor (110) of FIG. 1). According to one embodiment, the buffer queue (220) may be included in a memory (e.g., the memory (130) of FIG. 1).

[0053] According to one embodiment, the decoder (210) can decode input video data frame by frame. According to one embodiment, the decoding may include the operation of decoding a compressed video stream to reconstruct pixel data included in each frame.

[0054] According to one embodiment, the buffer queue (220) may store at least a portion of the data decoded by the decoder (210). The at least portion of the decoded data may include metadata for each frame or indices to facilitate and / or speed access to the metadata.

[0055] According to one embodiment, the LUT generation unit (230) can analyze the metadata for image quality processing tailored to image characteristics and perform an LUT generation operation for image quality processing based on the metadata.

[0056] According to one embodiment, the image quality processing unit (240) can perform image quality processing by applying the LUT generated by the LUT generation unit (230). For example, the image quality processing unit (240) can perform various processing such as color correction processing and brightness adjustment processing according to the LUT.

[0057] Referring to FIG. 2, according to one embodiment, a display device using a first mode (e.g., the display device (100) of FIG. 1) can perform at least one of the following operations:

[0058] - The operation of receiving video information from the video input unit (120) and transmitting it to the decoder (210).

[0059] - The operation of transferring and storing all or part of the data corresponding to each frame (hereinafter referred to as the 'first frame') obtained by decoding in the decoder (210) to the buffer queue (220).

[0060] - An operation to generate a first LUT for image quality processing for a first frame using data stored in a buffer queue (220) in the LUT generation unit (230).

[0061] - An operation to perform image quality processing for the first frame by applying the first LUT in the image quality processing unit (240).

[0062] - An operation of rendering the data of the first frame that has undergone image processing and outputting it to the display (140).

[0063] According to one embodiment, the timing of the operation of generating the first LUT in the LUT generation unit (230) may be a time prior to the rendering request for the first frame.

[0064] According to one embodiment, the timing of the operation of performing image quality processing for the first frame in the image quality processing unit (240) may generally be a time before rendering the first frame, but is not limited thereto. For example, depending on the embodiment, it may be a time during rendering and / or a time after rendering.

[0065] According to one embodiment, data corresponding to a first frame obtained by decoding may include frame data for the first frame, first metadata for the first frame, and / or other information related to said first metadata. Data stored in a buffer queue may include said first metadata and / or other information related to said first metadata. Other information related to said first metadata may include indices to facilitate and / or speed access to said first metadata stored in memory (130).

[0066] According to one embodiment, the first mode may have an advantage in terms of stability of image processing, for example, compared to other modes that do not use a buffer queue (220). For example, by storing and / or indexing metadata in advance, stable processing may be possible even in situations where input data is missing or received irregularly.

[0067] According to one embodiment, the first mode may have an advantage in overall process performance compared to other modes that do not use the buffer queue (220), for example, by efficiently using processor resources. For example, processor resources can be used efficiently by reducing the computational burden of the LUT generation unit (230) by reusing the same LUT when similar metadata is repeated.

[0068] FIG. 3a is a diagram illustrating an example of a buffer overflow occurring during the use of a first mode according to an embodiment of the present disclosure. A buffer overflow refers to, for example, a phenomenon in which data exceeding the capacity that the buffer can accommodate accumulates in the buffer.

[0069] According to one embodiment, when data (310) for a service such as a real-time streaming service is input to a decoder (210), decoding buffering occurs temporarily in large quantities, and if the rate at which data is input to the buffer queue (220) is faster than the rate at which data is output, data may continue to accumulate in the buffer queue (220). Then, if a buffer overflow occurs, the image quality processing unit (240) may not properly match the LUT corresponding to each frame (320), and thus the image quality processing performance may be degraded.

[0070] FIG. 3b is a diagram showing the input and output of a buffer queue (220) according to one embodiment of the present disclosure. Frame@k in FIG. 3b refers to the k-th frame. LUT@k refers to the LUT corresponding to Frame@k. In the embodiment of FIG. 3b, the generation of one LUT for one frame (e.g., a one-to-one correspondence between the frame and the LUT) is described as an example. However, depending on the embodiment, one LUT may be generated for multiple frames or multiple LUTs may be generated for one frame as needed.

[0071] In FIG. 3b, i decoded data corresponding to a total of i frames (330) from the nth frame to the n+i-1th frame are input into a buffer queue (220). Then, a total of j data from the nth frame to the n+j-1th frame are output to generate a corresponding LUT (340). In general, since the LUT corresponding to each frame must be matched one-to-one, i and j have the same value. However, in situations where a large amount of temporary decoding buffering occurs, such as when data (310) for a service like a real-time streaming service is input into the decoder (210), i > j, and data can accumulate up to ij. If the value of ij is greater than the capacity of the buffer, a buffer overflow occurs, so the LUT corresponding to each frame is not properly matched (320), and the image quality processing performance may be degraded.

[0072] Referring to FIG. 3b, according to one embodiment, the amount of data accumulated in the buffer queue (220) can be measured indirectly through the difference between the number of input frames (330) i and the number of output LUTs (340) j without directly checking the accumulated data.

[0073] According to one embodiment, the amount of data accumulated in the buffer queue (220) can be indirectly measured by performing a calculation after checking at least one of the input of the buffer queue (220), the output of the buffer queue (220), the input of the decoder (210), the output of the decoder (220), the input of the LUT generation unit (230), or the output of the LUT generation unit (230).

[0074] FIG. 4 is a drawing for explaining the configuration and operation when a display device (e.g., the display device (100) of FIG. 1) uses a second mode according to one embodiment of the present disclosure.

[0075] Referring to FIG. 4, a display device (e.g., the display device (100) of FIG. 1) may include an image input unit (120), a decoder (210), a buffer queue (220), an LUT generation unit (230), an image quality processing unit (240), and / or a display (140). According to one embodiment, the decoder (210), the LUT generation unit (230), and / or the image quality processing unit (240) may be implemented as at least one processor (e.g., the processor (110) of FIG. 1). According to one embodiment, the buffer queue (210) may be included in a memory (e.g., the memory (130) of FIG. 1).

[0076] Referring to FIG. 4, according to one embodiment, a display device (100) using a second mode can perform at least one of the following operations:

[0077] - The operation of receiving video information from the video input unit (120) and transmitting it to the decoder (210).

[0078] - The operation of transmitting all or part of the data corresponding to each frame (hereinafter referred to as the 'second frame') obtained by decoding in the decoder (210) to the LUT generation unit (230).

[0079] - An operation to generate a second LUT for image quality processing for the second frame using data received from the LUT generation unit (230).

[0080] - An operation to perform image quality processing for the second frame by applying the second LUT in the image quality processing unit (240).

[0081] - An action of rendering the data of the second frame, which has been processed for image quality, and outputting it to the display (140).

[0082] For example, when using the second mode, unlike when using the first mode, the decoder (210) can directly transmit the data corresponding to each frame obtained by decoding to the LUT generation unit (230) without transmitting and storing it in the buffer queue (220).

[0083] For example, when using the second mode, unlike when using the first mode, the LUT generation unit (230) can generate an LUT for a frame using data corresponding to each frame directly transmitted from the decoder (210) without using data stored in the buffer queue (220). Through this, the LUT can be generated in real time.

[0084] According to one embodiment, the timing of the operation to generate the second LUT in the LUT generation unit (230) may be a time after the rendering request for the second frame.

[0085] According to one embodiment, the timing of the operation of performing image quality processing for the second frame in the image quality processing unit (240) may generally be a time before rendering the second frame, but is not limited thereto. For example, it may be a time during rendering and / or a time after rendering.

[0086] According to one embodiment, the data received from the LUT generation unit (230) may include second metadata for the second frame and / or other information related to the second metadata.

[0087] According to one embodiment, the second mode may have the advantage that, when compared to another mode (e.g., the first mode) that uses a buffer queue (220), the problem of image quality processing performance degradation due to the buffer overflow described above does not occur.

[0088] According to one embodiment, the display device (100) can secure storage space in the buffer queue (220) by processing data accumulated in the buffer queue (220) while using the second mode. The processing may include at least one of processing to transfer the accumulated data to a subsequent module (e.g., LUT generation unit (230)), data compression processing, data dropping processing, and data merging processing.

[0089] According to one embodiment, the second mode may have the advantage of performing operations for LUT generation more quickly by, for example, allocating more processor resources to the LUT generation unit (230) or assigning a higher process priority compared to other modes (e.g., the first mode) that use a buffer queue (220). Specifically, in situations where image quality processing must be performed in real time by reflecting the characteristics of each frame that changes rapidly over time, such as real-time streaming, image quality processing can be performed without delay.

[0090] FIG. 5 is a drawing for explaining the configuration and operation of a display device (e.g., the display device (100) of FIG. 1) according to one embodiment of the present disclosure.

[0091] Referring to FIG. 5, a display device (e.g., the display device (100) of FIG. 1) may include an image input unit (120), a decoder (210), a buffer queue (220), an LUT generation unit (230), an image quality processing unit (240), a mode control unit (510), and / or a display (140). In the embodiment of FIG. 5, the display device (100) may further include a mode control unit (510) compared to the embodiment of FIG. 2 and / or FIG. 4. According to one embodiment, the decoder (210), the LUT generation unit (230), the image quality processing unit (240), and / or the mode control unit (510) may be implemented by at least one processor (e.g., the processor (110) of FIG. 1). According to one embodiment, the buffer queue (210) may be contained in a memory (e.g., the memory (130) of FIG. 1).

[0092] According to one embodiment, the mode control unit (510) may identify (511) whether a mode switching condition is satisfied and issue a mode switching command (512) based on the satisfaction of the condition. According to one embodiment, the mode switching command (512) may be transmitted to another part of at least one processor, for example, to a LUT generation unit (230).

[0093] According to one embodiment, the display device (100) can select and use either a first mode or a second mode in accordance with a mode switching command (512) of a mode control unit (510).

[0094] Referring to FIG. 5, according to one embodiment, the data path may vary depending on the mode. Specifically, when using the first mode, the entire data path may include the data path of the first mode of FIG. 5. When using the second mode, the entire data path may include the data path of the second mode of FIG. 5. In the present disclosure, the data path may be referred to as a signal processing path or an image signal processing path.

[0095] Referring to FIG. 5, according to one embodiment, the data path of the first mode may include a path traversing a decoder (210), a buffer queue (220), and / or an LUT generation unit (230). In the data path of the first mode, the decoder (210) decodes input data and transmits it to the buffer queue (220), where the received data is stored in the buffer queue (220), and the LUT generation unit (230) may perform an LUT generation operation using all or part of the data stored in the buffer queue (220).

[0096] Referring to FIG. 5, according to one embodiment, the data path of the second mode may include a path that passes through the decoder (210) and / or the LUT generation unit (230) but does not pass through the buffer queue (220). For example, in the second mode, LUT generation can be performed without using the buffer queue (220). For example, in the data path of the second mode, the decoder (210) decodes the input data and transmits it directly to the LUT generation unit (230), and the LUT generation unit (230) can perform an LUT generation operation using all or part of the received data.

[0097] According to one embodiment, the mode switching condition (511) may include a first mode switching condition, which is a condition for switching from a first mode to a second mode, and a second mode switching condition, which is a condition for switching from a second mode to a first mode. The mode switching condition (511) may be associated with state information of at least a portion of a display device (e.g., the display device (100) of FIG. 1). The mode switching condition (511) may be appropriately set to select an appropriate mode according to the state information.

[0098] According to one embodiment, the mode switching condition (511) may be associated with at least one of the following information: processor load information, memory usage information, data processing speed information of the decoder (210), data input speed information of the LUT generation unit (230), and data processing speed information of the LUT generation unit.

[0099] According to one embodiment, the mode switching condition (511) may be related to the state information of the buffer queue (220). The state information of the buffer queue (220) may include at least one of the capacity of data accumulated in the buffer queue (220), the number of indices of metadata accumulated in the buffer queue (220), or the rate at which data is input into the buffer queue (220) and the rate at which data is output from the buffer queue (220).

[0100] According to one embodiment, the first mode switching condition is satisfied when the capacity of data accumulated in the buffer queue (220) exceeds a first threshold, and the second mode switching condition is satisfied when the capacity of data accumulated in the buffer queue (220) is less than a second threshold. The first threshold may be the maximum capacity of the buffer queue (220) or a capacity less than or equal to it. The second threshold may be a capacity less than or equal to the first threshold.

[0101] According to one embodiment, the first mode switching condition is satisfied when the number of metadata indices accumulated in the buffer queue (220) exceeds a third threshold, and the second mode switching condition may be satisfied when the number of metadata indices accumulated in the buffer queue (220) is less than a fourth threshold. The third threshold may be the maximum number of indices that can be stored in the buffer queue (220) or less. The fourth threshold may be a number less than or equal to the third threshold.

[0102] According to one embodiment, the first mode switching condition may be satisfied if the value obtained by dividing the capacity of data input into the buffer queue (220) during a unit time by the capacity of data output from the buffer queue (220) during the unit time exceeds a fifth threshold. For example, the satisfaction of the first mode switching condition may be determined by measuring the input / output speed of data during a common unit time (e.g., the last 5 seconds). The unit time and the fifth threshold can be appropriately selected by a person skilled in the art.

[0103] According to one embodiment, the first mode switching condition may be satisfied if the value obtained by dividing the number of frames decoded during a unit time by the number of LUTs corresponding to each frame generated during the unit time exceeds a sixth threshold. For example, the satisfaction of the first mode switching condition may be determined by measuring the decoding speed and the LUT generation speed during a common unit time (e.g., the last 5 seconds). The unit time and the sixth threshold can be appropriately selected by a person skilled in the art.

[0104] At least one of the first, second, third, fourth, fifth, and sixth thresholds may not be a fixed value. For example, the thresholds may be dynamically set according to various variables such as memory usage, processor utilization, data dwell time in the buffer queue (220), and frame rate.

[0105] According to one embodiment, the first mode switching condition and / or the second mode switching condition may be satisfied after a certain period of time has elapsed since the last mode was switched. The certain period of time can be appropriately set by a person skilled in the art.

[0106] According to one embodiment, the first mode switching condition and / or the second mode switching condition may be satisfied by an external input (e.g., user input).

[0107] According to one embodiment, the mode control unit (510) may monitor the status information of the buffer queue (220) to determine whether the mode switching condition is satisfied (511). The status information of the buffer queue (220) may include at least one of the capacity of data accumulated in the buffer queue (220), the number of indices of metadata accumulated in the buffer queue (220), the rate at which data is input into the buffer queue (220), and the rate at which data is output from the buffer queue (220).

[0108] In one embodiment, the monitoring may mean checking the status information of the buffer queue (220) at regular time intervals or based on the occurrence of a specific event. The specific event may include at least one of various events, such as, for example, an event in which new data is input into the buffer queue (220), an event in which LUT generation is completed by the LUT generation unit (230), or an event in which a processor resource or memory usage threshold is exceeded.

[0109] FIG. 6 is a flowchart illustrating a process for determining whether a display device (e.g., the display device (100) of FIG. 1) is used to switch modes while using a first mode, according to one embodiment of the present disclosure. FIG. 6 illustrates that the first mode switching condition is related to the state information of a buffer queue (e.g., the buffer queue (220) of FIG. 2), but the present disclosure is not limited thereto. For example, the contents of the embodiment of FIG. 6 may also be applied when the first mode switching condition is related to the state information of another part of the display device (100) of the present disclosure.

[0110] Referring to FIG. 6, in operation 610, the display device (100) can monitor the status information of the buffer queue (220) while using the first mode. Operation 610 can be performed, for example, by a processor (e.g., the processor (110) of FIG. 1) or a mode control unit (e.g., the mode control unit (510) of FIG. 5).

[0111] In operation 620, the display device (100) can identify whether a first mode switching condition is satisfied. Operation 620 may be performed, for example, by a processor (110) or a mode control unit (510). The first mode switching condition is a condition related to the state information of the buffer queue (220), for example, and may be related to the possibility of the occurrence of the buffer overflow described above. Specifically, for example, the first mode switching condition may be a condition satisfied when the amount of data accumulated in the buffer queue (220) exceeds a first threshold.

[0112] In operation 632, the display device (100) may switch to a second mode when the first mode switching condition is satisfied. In this case, in operation 640, the display device (100) may generate a LUT using the second mode, which is the switched mode. Operation 632 may be performed, for example, by a processor (110) or a mode control unit (510).

[0113] In operation 631, the display device (100) may maintain the first mode if the first mode switching condition is not satisfied. In this case, in operation 640, the display device (100) may generate a LUT using the maintained first mode. Operation 631 may be performed, for example, by a processor (110) or a mode control unit (510).

[0114] FIG. 7 is a flowchart illustrating a process for determining whether a display device (e.g., the display device (100) of FIG. 1) is using a second mode and whether to switch modes according to one embodiment of the present disclosure. FIG. 7 illustrates, as an example, that the second mode switching condition is related to the state information of a buffer queue (220), but the present disclosure is not limited thereto. For example, the contents of the embodiment of FIG. 7 may also be applied when the second mode switching condition is related to the state information of another part of the display device (100) of the present disclosure.

[0115] Referring to FIG. 7, in operation 710, the display device (100) can monitor the status information of the buffer queue (220) while using the second mode. Operation 710 can be performed, for example, by a processor (e.g., the processor (110) of FIG. 1) or a mode control unit (e.g., the mode control unit (510) of FIG. 5).

[0116] In operation 720, the display device (100) can identify whether the second mode switching condition is satisfied. Operation 720 may be performed, for example, by a processor (110) or a mode control unit (510). The second mode switching condition is a condition related to the state information of the buffer queue (220), for example, and may be related to the possibility of the occurrence of the buffer overflow described above. For example, the second mode switching condition may be a condition satisfied when the amount of data accumulated in the buffer queue (220) becomes less than a second threshold.

[0117] In operation 732, the display device (100) may switch to the first mode when the second mode switching condition is satisfied. In this case, in operation 740, the display device (100) may generate a LUT using the first mode, which is the switched mode. Operation 732 may be performed, for example, by a processor (110) or a mode control unit (510).

[0118] In operation 731, the display device (100) may maintain the second mode if the second mode switching condition is not satisfied. In this case, in operation 740, the display device (100) may generate a LUT using the maintained second mode. Operation 731 may be performed, for example, by a processor (110) or a mode control unit (510).

[0119] FIG. 8 is a flowchart for explaining the operation of a display device (e.g., the display device (100) of FIG. 1) according to one embodiment of the present disclosure, based on mode selection. In FIG. 8, for convenience of explanation, the mode selection operation (820) is depicted as being performed at a time preceding decoding, but is not limited thereto. Since the mode selection depends, for example, on a mode switching command based on satisfying a mode switching condition, it is not limited to a specific time among the image quality processing steps for each frame.

[0120] Referring to FIG. 8, in operation 810, the image input unit (e.g., image input unit (120) of FIG. 1) of the display device (e.g., display device (100) of FIG. 1) can receive image data frame by frame.

[0121] In operation 820, the mode control unit (e.g., mode control unit (510) of FIG. 5) of the display device (e.g., display device (100) of FIG. 1) may select a mode to use by going through an operation illustrated in FIG. 6 or FIG. 7, for example. As a result of operation 820, the display device (100) may use a first mode or a second mode.

[0122] When the first mode is used as a result of operation 820, the display device (100) may sequentially perform the following operations: operation 831 in which a decoder (e.g., decoder (210) of FIG. 2) decodes image data to obtain frame data and metadata corresponding to each frame; operation 840 in which a buffer queue (e.g., buffer queue (220) of FIG. 2) is created within a memory (e.g., memory (130) of FIG. 1) and at least a portion of the data obtained by decoding is stored in the buffer queue (220); operation 851 in which an LUT generation unit (e.g., LUT generation unit (230) of FIG. 2) generates a LUT corresponding to each frame based on the data stored in the buffer queue (220); operation 860 in which an image quality processing unit (e.g., image quality processing unit (240) of FIG. 2) performs image quality processing by matching and applying the LUT and frame data corresponding to each frame; and display each frame for which image quality processing and rendering are completed (e.g., FIG. 1 Operation 870 that outputs to the display (140)).

[0123] When using the second mode as a result of operation 820, the display device (100) may sequentially perform the following operations: operation 832 in which a decoder (e.g., decoder (210) of FIG. 2) decodes image data to obtain frame data and metadata corresponding to each frame; operation 852 in which an LUT generation unit (e.g., LUT generation unit (230) of FIG. 2) receives the decoded data obtained from the decoder (210) and generates an LUT corresponding to each frame; operation 860 in which an image quality processing unit (e.g., image quality processing unit (240) of FIG. 2) matches and applies the LUT and frame data corresponding to each frame to perform image quality processing; and operation 870 in which each frame for which image quality processing and rendering are completed is output to a display (e.g., display (140) of FIG. 1).

[0124] According to one embodiment of the present disclosure, a display device may include an image input unit, a display, a memory including at least one storage medium in which data and at least one instruction are stored, and at least one processor capable of executing said at least one instruction. The at least one processor may control a method of operation of the display device by executing said at least one instruction.

[0125] According to one embodiment, a method of operation of a display device may be provided. The method of operation of the display device may include at least one operation. The at least one operation may include an operation of decoding input image data to obtain data for each frame, an operation of performing an operation to generate a LUT for image quality processing for each frame using all or part of the data for each frame obtained by decoding, an operation of performing image quality processing using the generated LUT for each frame, an operation of rendering each frame for which image quality processing has been performed, and an operation of outputting each frame for which rendering has been completed to a display.

[0126] According to one embodiment, a buffer queue for temporarily storing data in the memory may be created and / or allocated by the at least one processor executing at least one instruction. Storage space in the buffer queue may be secured by processing accumulated data as the at least one processor executes at least one instruction.

[0127] According to one embodiment, the at least one processor can select and use one of different modes corresponding to different operation methods and sequences related to image processing of a display device by executing at least one instruction.

[0128] According to one embodiment, the different modes may include the following modes: a first mode of decoding an input image to obtain first metadata corresponding to each frame, storing the first metadata or data related to the first metadata in a buffer queue in the memory, generating a first LUT corresponding to each frame using the data stored in the buffer queue, and performing image quality processing for each frame by applying the first LUT. A second mode of decoding an input image to obtain second metadata corresponding to each frame, generating a second LUT corresponding to each frame using the second metadata, and performing image quality processing for each frame by applying the second LUT.

[0129] According to one embodiment, the at least one processor may determine whether one or more mode switching conditions are satisfied in order to determine whether to switch to a mode different from the currently used mode by executing at least one instruction. The one or more mode switching conditions may be related to state information of one or more components of the display device. For example, the one or more mode switching conditions may be related to state information of the buffer queue.

[0130] According to one embodiment, the at least one processor can perform image quality processing using the first mode by executing at least one instruction, and, based on identifying that a first mode switching condition is satisfied while using the first mode, switch to the second mode and perform image quality processing using the second mode.

[0131] According to one embodiment, the at least one processor can perform image quality processing using the first mode by switching to the first mode based on identifying that a second mode switching condition is satisfied while using the second mode by executing at least one instruction.

[0132] According to one embodiment, the first LUT and / or the second LUT may include a LUT for image quality processing. The image quality processing may include one or more of gamma correction processing, color mapping processing, color correction processing, tone mapping processing, white balance adjustment processing, and dithering processing.

[0133] The first mode switching condition above refers to a condition for switching from the first mode to the second mode. According to one embodiment, the first mode switching condition may be one of the following conditions, or a condition in which two or more of the following conditions are combined by AND and / or OR. (a) The capacity of information accumulated in the buffer queue exceeds a first threshold. (b) The number of metadata indices accumulated in the buffer queue exceeds a second threshold. (c) The value obtained by dividing the capacity of data input to the buffer queue by the capacity of data output from the buffer queue during the same unit time exceeds a third threshold.

[0134] The above second mode switching condition refers to a condition for switching from the second mode to the first mode. According to one embodiment, the second mode switching condition may be one of the following conditions, or a condition in which two or more of the following conditions are combined by AND and / or OR. (a) The capacity of information accumulated in the buffer queue becomes less than a fourth threshold. (b) The number of metadata indices accumulated in the buffer queue becomes less than a fifth threshold. (c) The time of continuous use of the second mode exceeds a sixth threshold.

[0135] According to one embodiment, the at least one processor may check the status information of the buffer queue at regular time intervals or / or based on the occurrence of a specific event. According to one embodiment, the status information may include at least one of the capacity of data accumulated in the buffer queue, the number of indices of metadata accumulated in the buffer queue, the capacity of data input into the buffer queue per unit time, and the capacity of data output from the buffer queue per unit time.

[0136] According to one embodiment, when the at least one processor performs an operation to generate the second LUT using the second mode, it can increase the speed of the operation to generate the LUT by allocating more processor resources to the operation to generate the LUT or / or assigning a higher processor priority compared to when the processor performs an operation to generate the first LUT using the first mode.

[0137] According to one embodiment, when the at least one processor uses a second mode, the at least one processor can perform an LUT generation operation corresponding to a frame in response to a rendering request corresponding to each frame.

[0138] According to one embodiment, while the at least one processor is using the second mode, it may be configured to secure storage space in the buffer queue by processing information accumulated in the buffer queue.

[0139] According to one embodiment of the present disclosure, a method for operating a display device may be provided. The method for operating a display device may include: generating a first LUT for image quality processing for a first frame based on data stored in a buffer queue using a first mode, wherein the data includes first metadata corresponding to the first frame transmitted from a decoder to the buffer queue or information related to the first metadata; switching to a second mode based on identifying that a first mode switching condition related to state information of the buffer queue is satisfied while using the first mode; and generating a second LUT for image quality processing for a second frame based on second metadata corresponding to the second frame transmitted from the decoder using the second mode.

[0140] According to one embodiment, a method for operating a display device may further include: switching to the first mode based on identifying that a second mode switching condition related to the state information of the buffer queue is satisfied while using the second mode.

[0141] According to one embodiment, a method for operating a display device may further include an operation of periodically checking the status information of the buffer queue.

[0142] According to one embodiment, a method for operating a display device may further include an operation of checking the status information of the buffer queue based on the occurrence of a specific event.

[0143] According to one embodiment, the operation of generating the second LUT using the second mode may include: an operation of allocating more processor resources or assigning a higher processor priority to the operation for generating the LUT compared to the case where the first mode is used; and / or an operation of securing storage space in the buffer queue by processing data accumulated in the buffer queue.

[0144] Meanwhile, the various embodiments described above may be implemented in software containing instructions stored on a device-readable storage medium, included in a computer program product in the form of a device-readable storage medium or distributed online through an application store, or implemented within a recording medium readable by a computer or similar device using software, hardware, or a combination thereof.

[0145] Each component according to the various embodiments described above may be composed of a single or multiple entities, and some auxiliary components may be omitted or additionally included. Some components may be integrated into a single entity to perform the same or similar functions as those performed by each corresponding component prior to integration.

[0146] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0147] The term “part” as used in the various embodiments of this document (e.g., embodiments of FIG. 4 and / or FIG. 5) may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, the “part” may be implemented in the form of an application-specific integrated circuit (ASIC).

[0148] The operations according to the various embodiments described above may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

Claims

1. In a display device, display; Memory including a storage medium for storing at least one instruction; and At least one processor including a processing circuit that executes at least one instruction stored in the memory; and The above at least one processor is: Using a first mode, a first LUT (Look Up Table) for image quality processing for a first frame is generated based on data stored in a buffer queue, and the data includes first metadata corresponding to the first frame transmitted from a decoder to the buffer queue or information related to the first metadata. Based on identifying that a first mode switching condition related to the state information of the buffer queue is satisfied while using the first mode, switching to a second mode, and A second LUT for image quality processing for a second frame is configured to be generated based on second metadata corresponding to a second frame received from the decoder, using the second mode above. Display device.

2. In Paragraph 1, The above-mentioned at least one processor is, Configured to switch to the first mode based on identifying that a second mode switching condition related to the state information of the buffer queue is satisfied while using the second mode, Display device.

3. In Paragraph 2, The above first mode switching condition is satisfied when the capacity of data accumulated in the buffer queue exceeds a first threshold, and The above second mode switching condition is satisfied when the capacity of data accumulated in the buffer queue becomes less than the second threshold value. Display device.

4. In Paragraph 2, The above first mode switching condition is satisfied when the number of metadata indices accumulated in the buffer queue exceeds a third threshold, and The above second mode switching condition is satisfied when the number of metadata indices accumulated in the buffer queue becomes less than the fourth threshold. Display device.

5. In Paragraph 1 or 2, The above first mode switching condition is satisfied when the value obtained by dividing the capacity of data output from the buffer queue during a unit time by the capacity of data input to the buffer queue during the unit time exceeds a fifth threshold value. Display device.

6. In any one of paragraphs 1 through 5, The above at least one processor is configured to periodically check the status information of the buffer queue, and The above state information includes at least one of the capacity of data accumulated in the buffer queue, the number of indices of metadata accumulated in the buffer queue, the capacity of data input into the buffer queue per unit time, or the capacity of data output from the buffer queue per unit time. Display device.

7. In any one of paragraphs 1 through 5, The above at least one processor is configured to check the state information of the buffer queue based on the occurrence of a specific event, and The above state information includes at least one of the capacity of data accumulated in the buffer queue, the number of indices of metadata accumulated in the buffer queue, the capacity of data input into the buffer queue per unit time, or the capacity of data output from the buffer queue per unit time. Display device.

8. In any one of paragraphs 1 through 7, The above at least one processor is configured to allocate more processor resources or assign a higher processor priority to the operation for generating the LUT when performing the operation for generating the second LUT using the second mode compared to when performing the operation for generating the first LUT using the first mode. Display device.

9. In any one of paragraphs 1 through 8, The above-mentioned at least one processor is, Configured to secure storage space of the buffer queue by processing data accumulated in the buffer queue while using the second mode, Display device.

10. In any one of paragraphs 1 through 9, The above at least one processor performs an operation to generate the second LUT based on a rendering request for the second frame while using the second mode, Display device.

11. A method for operating a display device, An operation to generate a first LUT for image quality processing for a first frame based on data stored in a buffer queue using a first mode, wherein the data includes first metadata corresponding to the first frame transmitted from a decoder to the buffer queue or information related to the first metadata; An operation to switch to a second mode based on identifying that a first mode switching condition related to the state information of the buffer queue is satisfied while using the first mode; and The operation of generating a second LUT for image quality processing for the second frame based on second metadata corresponding to the second frame transmitted from the decoder using the second mode; method.

12. In Paragraph 11, The operation of switching to the first mode based on identifying that a second mode switching condition related to the state information of the buffer queue is satisfied while using the second mode; further comprising method.

13. In Paragraph 12, The above first mode switching condition is satisfied when the capacity of data accumulated in the buffer queue exceeds a first threshold, and The above second mode switching condition is satisfied when the capacity of data accumulated in the buffer queue becomes less than the second threshold value. method.

14. In Paragraph 12, The above first mode switching condition is satisfied when the number of metadata indices accumulated in the buffer queue exceeds a third threshold, and The above second mode switching condition is satisfied when the number of metadata indices accumulated in the buffer queue becomes less than the fourth threshold. method.

15. In Paragraph 12, The above first mode switching condition is satisfied when the value obtained by dividing the capacity of data output from the buffer queue during a unit time by the capacity of data input to the buffer queue during the unit time exceeds a fifth threshold value. method.

Images