Display device and operation method thereof

Abstract

In order to improve the image quality of a region of interest of an image to the greatest extent, a display device according to an embodiment of the present invention can extract a region of interest and a detail image from the entire region of an input image, extract one or more sub-images from the extracted region of interest, merge the detail image and the one or more sub-images to generate a merged image, and display the merged image. A mixture curve is generated from a map gain of the generated merge map and a local mapping curve generated on the basis of local region information of each of a plurality of local regions constituting the entire region, and tone mapping of the input image is performed on the basis of the generated mixture curve.

Description

Technical Field

[0001] The present invention relates to a display device and a driving method thereof. The present invention relates to a display device and a driving method thereof for enhancing the effect of dynamic tone mapping. Background Art

[0002] Digital images are usually composed of three channels of RGB. Generally, SDR (Standard Dynamic Range) images can present brightness in the range of 0-255 (8 bits) for each channel.

[0003] However, because SDR images have a narrow range of brightness that can be displayed in each channel, dark areas and bright areas may not be displayed correctly. For example, in images with a significant difference in brightness between dark and bright areas, such as photos taken under backlight, the balance between light and dark may be skewed to one side. As a result, there may be a significant difference between the scene seen with the naked eye and the SDR image of that scene.

[0004] Compared to such SDR images, images presented in 16 or 32 bits in each channel are called HDR (High Dynamic Range) images. Since HDR images have a wider brightness presentation capability, they have the advantage of being able to present images more realistically.

[0005] However, ordinary display devices cannot display 16-bit or 32-bit grayscales, so the 16-bit or 32-bit HDR image needs to be converted into a digital image with a limited dynamic range.

[0006] Tone mapping is used in the process of converting HDR images into SDR images or LDR (Low Dynamic Range) images. It can refer to a technology that compresses the brightness area of the HDR image into the brightness area of the SDR image or LDR image while improving the brightness-to-dark ratio. Summary of the Invention

[0007] Problems to be solved by the invention

[0008] The present invention aims to improve the image quality of the region of interest of an image to the greatest extent possible.

[0009] The present invention provides a display device and a driving method thereof, which can further improve the stereoscopic effect of an area of interest of an image and the user's sense of immersion.

[0010] Technical solutions to the problem

[0011] The display device of an embodiment of the present invention can extract a region of interest and a detail image from the entire area of an input image, extract one or more sub-images from the extracted region of interest, merge the detail image and the one or more sub-images to generate a merged image, generate a mixing curve based on the image gain of the generated merged image and a local mapping curve generated based on local area information of each of the multiple local areas constituting the entire area, and perform tone mapping of the input image based on the generated mixing curve.

[0012] The operation method of the display device according to the embodiment of the present invention may include: extracting a region of interest and a detail image from the entire region of an input image; extracting one or more sub-images from the extracted region of interest;

[0013] A step of merging the detail image and the one or more sub-images to generate a merged image; a step of generating a blending curve based on the image gain of the generated merged image and a local mapping curve generated based on local area information of each of the plurality of local areas constituting the entire area; and a step of performing tone mapping of the input image based on the generated blending curve.

[0014] Effects of the Invention

[0015] According to the display device and the driving method thereof according to the embodiments of the present invention, the brightness-darkness ratio in the region of interest of an image can be improved, and the stereoscopic effect of the image and the user's sense of immersion can be enhanced.

[0016] According to an embodiment of the present invention, the image quality of the HDR image region of interest to the viewer can be improved by performing tone mapping on the region of interest, thereby maximizing the presence and stereoscopic effect of the HDR image and significantly improving the image quality competitiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a block diagram showing the configuration of a display device according to an embodiment of the present invention.

[0018] Figure 2 FIG. 4 is a block diagram of a remote control device according to an embodiment of the present invention.

[0019] Figure 3 An example of the actual configuration of a remote control device according to an embodiment of the present invention is shown.

[0020] Figure 4 An example of using a remote control device according to an embodiment of the present invention is shown.

[0021] Figure 5 is a flowchart showing a method for driving a display device according to an embodiment of the present invention.

[0022] Figure 61 is a diagram showing an example of an image subjected to HDR processing according to an embodiment of the present invention.

[0023] Figure 7 According to an embodiment of the present invention, Figure 6 A diagram of the detail map of the image shown.

[0024] Figure 8 FIG. 1 is a diagram illustrating application of gain adjustment to a local area based on detail map information according to an embodiment of the present invention.

[0025] Figure 9 FIG. 1 is a diagram illustrating a change in curvature of a display device according to an embodiment of the present invention.

[0026] Figure 10 FIG. 1 is a diagram illustrating an example of a local contrast curve applied to a local area according to an embodiment of the present invention.

[0027] Figure 11 FIG. 1 is a diagram illustrating an example of a local contrast curve applied to each APL according to an embodiment of the present invention.

[0028] Figure 12 FIG. 1 is a diagram illustrating a case where different sharpness levels are applied to the same APL according to an embodiment of the present invention.

[0029] Figure 13 FIG. 1 is a diagram illustrating a local contrast curve at each APL according to a change in curvature and a case where different sharpness levels are applied at the same APL according to an embodiment of the present invention.

[0030] Figure 14 This is a flowchart for explaining an operating method of a display device according to another embodiment of the present invention.

[0031] Figure 15 FIG. 1 is a diagram illustrating a process of extracting a plurality of sub-maps of an object detected from an HDR image and obtaining a gain of a merged map from the extracted sub-maps according to an embodiment of the present invention.

[0032] Figure 16 FIG. 4 is a diagram illustrating a process of generating a blending curve by multiplying a map gain by a local area contrast curve according to an embodiment of the present invention.

[0033] Figure 17a and Figure 17b FIG. 1 is a diagram illustrating a method of identifying an object and applying a feature map of elements of the object to verify whether to generate a tone mapping curve according to an embodiment of the present invention.

[0034] Figure 18is a diagram illustrating a process of generating an output image through tone mapping based on object recognition of an input HDR image according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following describes embodiments of the present invention in more detail with reference to the accompanying drawings. The suffix "part" used in the following description of the components is given or used interchangeably for the sake of convenience and does not have different meanings or functions.

[0036] Terms including ordinal numbers such as first, second, etc. may be used to describe various components, but the components are not limited by the terms. The terms are only used to distinguish one component from other components.

[0037] In the following description, it is assumed that the display device is a smart TV (TV) that supports a broadcast receiving function, but the display device may also include a smart phone, etc. That is, the display device of the embodiment of the present invention does not necessarily include Figure 1 Multiple components shown.

[0038] Figure 1 This is a block diagram showing the configuration of a display device according to an embodiment of the present invention.

[0039] Reference Figure 1 The display device 100 may include a broadcast receiving unit 130 , an external device interface unit 135 , a storage unit 140 , a user input interface unit 150 , a control unit 170 , a wireless communication unit 173 , a display unit 180 , an audio output unit 185 , and a power supply unit 190 .

[0040] The broadcast receiving unit 130 may include a tuner 131 , a demodulator 132 , and a network interface unit 133 .

[0041] The tuner 131 may select a specific broadcast channel according to a channel selection instruction and may receive a broadcast signal of the selected specific broadcast channel.

[0042] The demodulator 132 may separate the received broadcast signal into a video signal, an audio signal, and a data signal related to the broadcast program, and may restore the separated video signal, audio signal, and data signal into a form that can be output.

[0043] The network interface unit 133 may provide an interface for connecting the display device 100 to a wired / wireless network including the Internet. The network interface unit 133 may transmit and receive data with other users or other electronic devices via the connected network or another network linked to the connected network.

[0044] The network interface unit 133 can access a predetermined web page through the connected network or another network linked to the connected network. In other words, it can access a predetermined web page through the network and send or receive data with the corresponding server.

[0045] Furthermore, the network interface unit 133 can receive content or data provided by content providers or network operators. Specifically, the network interface unit 133 can receive content such as movies, advertisements, games, VOD (Video on Demand), broadcast signals, and related information provided by content providers or network operators via the network.

[0046] In addition, the network interface unit 133 can receive firmware update information and update files provided by the network operator, and can send data to the Internet, content providers, or network operators.

[0047] The network interface unit 133 may select and receive a desired application from among applications open to the public through a network.

[0048] The external device interface unit 135 may receive an application or an application directory in an adjacent external device and transmit the application or application directory to the control unit 170 or the storage unit 140 .

[0049] The external device interface unit 135 can provide a connection path between the display device 100 and an external device. The external device interface unit 135 can receive one or more of images and audio output from an external device connected to the display device 100 wirelessly or by wire, and transmit the received audio to the control unit 170. The external device interface unit 135 can include a plurality of external input terminals. The plurality of external input terminals can include RGB terminals, one or more HDMI (High Definition Multimedia Interface) terminals, and color difference component terminals.

[0050] An image signal of an external device input through the external device interface unit 135 may be output through the display 180 . An audio signal of an external device input through the external device interface unit 135 may be output through the audio output unit 185 .

[0051] The external device that can be connected to the external device interface unit 135 can be any one of a set-top box, a Blu-ray player, a DVD player, a game console, a sound bar, a smartphone, a PC, a USB memory, and a home theater, but this is just an example.

[0052] In addition, a portion of content data stored in the display apparatus 100 may be transmitted to other users pre-registered with the display apparatus 100 or to a user or electronic device selected from other electronic devices or a selected electronic device.

[0053] The storage unit 140 may store programs for various signal processing and control in the control unit 170 , and may also store image, voice, or data signals after signal processing.

[0054] In addition, the storage unit 140 may perform a function for temporarily storing an image, voice, or data signal input from the external device interface unit 135 or the network interface unit 133 , and may also store information on a prescribed image through a channel memory function.

[0055] The storage unit 140 may store applications or an application directory input from the external device interface unit 135 or the network interface unit 133 .

[0056] The display device 100 can play content files (video files, still image files, music files, text files, application files, etc.) stored in the storage unit 140 and provide them to the user.

[0057] The user input interface 150 can transmit user input signals to the control unit 170, or transmit signals from the control unit 170 to the user. For example, the user input interface 150 can receive and process control signals such as power on / off, channel selection, and screen setting from the remote control device 200 using various communication methods such as Bluetooth, WB (UltraWideband), ZigBee, RF (Radio Frequency), or infrared (IR) communication, or transmit control signals from the control unit 170 to the remote control device 200.

[0058] In addition, the user input interface unit 150 may transmit a control signal input from a local key (not shown) such as a power key, a channel key, a volume key, and a setting value to the control unit 170 .

[0059] The image signal processed by the control unit 170 can be input to the display 180 and displayed as an image corresponding to the image signal. In addition, the image signal processed by the control unit 170 can be input to an external output device through the external device interface unit 135.

[0060] The audio signal processed by the control unit 170 may be outputted as an audio signal to the audio output unit 185. In addition, the audio signal processed by the control unit 170 may be inputted to an external output device through the external device interface unit 135.

[0061] In addition, the control unit 170 can control the overall operation within the display device 100 .

[0062] In addition, the control unit 170 can control the display device 100 using user instructions input through the user input interface unit 150 or an internal program, and can download applications or application directories required by the user to the display device 100 by accessing the network.

[0063] The control unit 170 can output the channel information selected by the user and the like together with the processed image or voice signal through the display unit 180 or the audio output unit 185 .

[0064] In addition, the control unit 170 can enable the image signal or voice signal from the external device, such as a camera or camcorder, input through the external device interface unit 135 to be output through the display unit 180 or the audio output unit 185 according to the external device image playback instruction received through the user input interface unit 150.

[0065] On the other hand, the control unit 170 can control the display unit 180 to display an image, and can control the display unit 180 to display, for example, a broadcast image input through the tuner 131, an external input image input through the external device interface unit 135, an image input through the network interface unit, or an image stored in the storage unit 140. In this case, the image displayed on the display unit 180 can be a still image or a video, and can be a 2D image or a 3D image.

[0066] In addition, the control unit 170 can control to play content stored in the display device 100, or received broadcast content, or external input content input from the outside. The content can be in various forms such as broadcast images, external input images, audio files, static images, visited web page screens, and text files.

[0067] The wireless communication unit 173 can communicate with the external device through wired or wireless communication. The wireless communication unit 173 can perform short range communication with the external device. To this end, the wireless communication unit 173 can use Bluetooth. TM), BLE (Bluetooth Low Energy), Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Near Field Communication (NFC), Wireless-Fidelity (Wi-Fi), Wi-Fi Direct, and Wireless Universal Serial Bus (Wireless USB) technologies to support short-range communication. The wireless communication unit 173 can support wireless communication between the display device 100 and a wireless communication system, between the display device 100 and other display devices 100, or between the display device 100 and a network in which the display device 100 (or an external server) is located through a short-range wireless communication network (Wireless Area Networks). The short-range wireless communication network can be a short-range wireless personal communication network (Wireless Personal Area Networks).

[0068] Here, the other display devices 100 may be (or may be linked to) mobile terminals that can exchange data with the display device 100 of the present invention, such as wearable devices (for example, smartwatches, smart glasses, head mounted displays (HMD)), and smart phones. The wireless communication unit 173 can detect (or identify) wearable devices that can communicate around the display device 100. Furthermore, in a case where the detected wearable device is authenticated as a device that communicates with the display device 100 of the present invention, the control unit 170 may send at least a portion of the data processed in the display device 100 to the wearable device through the wireless communication unit 173. Therefore, the user of the wearable device can utilize the data processed in the display device 100 through the wearable device.

[0069] The display portion 180 may generate a driving signal by converting an image signal, a data signal, an OSD signal processed by the control portion 170 or an image signal, a data signal received from the external device interface portion 135 into R, G, and B signals, respectively.

[0070] on the other hand, Figure 1The display device 100 shown is only one embodiment of the present invention, and some of the components shown in the figure may be integrated, added or omitted according to the specifications of the display device 100 actually implemented.

[0071] That is, as needed, two or more components can be combined into one component, or one component can be subdivided into two or more components. In addition, the functions performed in each module are only used to illustrate the embodiments of the present invention, and the specific actions or devices thereof do not limit the scope of the present invention.

[0072] According to another embodiment of the present invention, the display device 100 and Figure 1 Unlike the example shown, the tuner 131 and the demodulator 132 are not provided, and images can be received and played back through the network interface 133 or the external device interface 135 .

[0073] For example, the display device 100 may be separated into an image processing device such as a set-top box for receiving broadcast signals or contents of various network services, and a content playing device for playing contents input from the image processing device.

[0074] In this case, the operation method of the display device according to the embodiment of the present invention described below can be obtained by referring to Figure 1 The above-mentioned display device 100 may be used to perform the above-mentioned operations, but may also be used to perform the above-mentioned operations by a separate image processing device such as a set-top box or a content playback device having a display unit 180 and an audio output unit 185 .

[0075] Next, refer to Figures 2 to 3 , a remote control device according to an embodiment of the present invention is described.

[0076] Figure 2 is a block diagram of a remote control device according to an embodiment of the present invention, Figure 3 An example of the actual configuration of a remote control device according to an embodiment of the present invention is shown.

[0077] First, refer to Figure 2 The remote control device 200 may include a fingerprint recognition unit 210 , a wireless communication unit 220 , a user input unit 230 , a sensor unit 240 , an output unit 250 , a power supply unit 260 , a storage unit 270 , a control unit 280 , and a voice acquisition unit 290 .

[0078] Reference Figure 2 The wireless communication unit 220 sends and receives signals with any one of the display devices according to the above-mentioned embodiments of the present invention.

[0079] The remote control device 200 may include an RF module 221 capable of transmitting and receiving signals with the display device 100 according to the RF communication standard, and an IR module 223 capable of transmitting and receiving signals with the display device 100 according to the IR communication standard. Furthermore, the remote control device 200 may include a Bluetooth module 225 capable of transmitting and receiving signals with the display device 100 according to the Bluetooth communication standard. Furthermore, the remote control device 200 may include an NFC module 227 capable of transmitting and receiving signals with the display device 100 according to the Near Field Communication (NFC) communication standard, and a Wireless Local Area Network (WLAN) module 229 capable of transmitting and receiving signals with the display device 100 according to the WLAN communication standard.

[0080] Furthermore, the remote control device 200 transmits a signal including information related to the movement of the remote control device 200 and the like to the display device 100 via the wireless communication unit 220 .

[0081] On the other hand, the remote control device 200 can receive signals transmitted by the display device 100 through the RF module 221, and can transmit instructions on power on / off, channel change, volume adjustment, etc. to the display device 100 through the IR module 223 as needed.

[0082] The user input unit 230 may be composed of a key group, a button, a touch panel, or a touch screen. The user can input instructions related to the display device 100 to the remote control device 200 by operating the user input unit 230. In the case where the user input unit 230 has a hard button, the user can input instructions related to the display device 100 to the remote control device 200 by pressing the hard button. Figure 3 Provide explanation.

[0083] Reference Figure 3 The remote control device 200 may include a plurality of buttons. The plurality of buttons may include a fingerprint recognition button 212, a power button 231, a home button 232, a live button 233, an external input button 234, a volume adjustment button 235, a voice recognition button 236, a channel change button 237, a confirmation button 238, and a return button 239.

[0084] The fingerprint recognition key 212 can be a key for recognizing the user's fingerprint. As one embodiment, since the fingerprint recognition key 212 can be pressed, it can also receive a pressing action and a fingerprint recognition action. The power key 231 can be a key for turning on / off the power of the display device 100. The home key 232 can be a key for moving to the home screen of the display device 100. The live broadcast key 233 is a key for displaying real-time broadcast programs. The external input key 234 can be a key for receiving external input connected to the display device 100. The volume adjustment key 235 can be a key for adjusting the volume output by the display device 100. The voice recognition key 236 can be a key for receiving the user's voice and recognizing the received voice. The channel change key 237 can be a key for receiving a broadcast signal from a specific broadcast channel. The confirmation key 238 can be a key for selecting a specific function, and the return key 239 can be a key for returning to the previous screen.

[0085] Again, yes Figure 2 Provide explanation.

[0086] When the user input unit 230 is provided with a touch screen, the user can input instructions related to the display device 100 to the remote control device 200 by touching the soft keys of the touch screen. In addition, the user input unit 230 can have various input devices that can be operated by the user, such as a scroll key or a joystick, and this embodiment does not limit the scope of the rights of the present invention.

[0087] The sensor part 240 may have a gyro sensor 241 or an acceleration sensor 243 , and the gyro sensor 241 may sense information about the movement of the remote control apparatus 200 .

[0088] For example, the gyro sensor 241 can sense information about the movement of the remote control device 200 based on the x, y, and z axes, and the acceleration sensor 243 can sense information about the movement speed of the remote control device 200. On the other hand, since the remote control device 200 can also include a distance measurement sensor, the distance between the remote control device 200 and the display unit 180 of the display device 100 can be sensed.

[0089] The output unit 250 may output an image or voice signal corresponding to an operation of the user input unit 230 or a signal transmitted from the display device 100. Through the output unit 250, the user can recognize whether the user input unit 230 is operated or whether the display device 100 is controlled.

[0090] For example, the output unit 250 may have an LED module 251 that lights up when the user input unit 230 is operated or transmits and receives signals with the display device 100 through the wireless communication unit 220, or a vibration module 253 that generates vibration, or a sound output module 255 that outputs sound, or a display module 257 that outputs an image.

[0091] Furthermore, the power supply unit 260 supplies power to the remote control device 200 and stops supplying power if the remote control device 200 does not move within a specified time, thereby reducing power waste. The power supply unit 260 resumes supplying power when a specified button on the remote control device 200 is pressed.

[0092] The storage unit 270 can store various programs and application data required for controlling or operating the remote control device 200. When the remote control device 200 wirelessly transmits and receives signals with the display device 100 via the RF module 221, the remote control device 200 and the display device 100 transmit and receive signals in a predetermined frequency band.

[0093] The control unit 280 of the remote control device 200 may store information on a frequency band in which signals can be wirelessly transmitted and received with the display device 100 paired with the remote control device 200 in the storage unit 270 and refer to the information.

[0094] The control unit 280 controls various matters related to the control of the remote control device 200. The control unit 280 can transmit a signal corresponding to a predetermined key operation of the user input unit 230 or a signal corresponding to the movement of the remote control device 200 sensed by the sensor unit 240 to the display device 100 via the wireless communication unit 220.

[0095] In addition, the voice acquisition unit 290 of the remote control device 200 can acquire voice.

[0096] The voice acquisition unit 290 may include at least one microphone 291 and may acquire voice through the microphone 291 .

[0097] Next, Figure 4 Provide explanation.

[0098] Figure 4 An example of using a remote control device according to an embodiment of the present invention is shown.

[0099] Figure 4 (a) in FIG. 1 illustrates that a cursor 205 corresponding to the remote control device 200 is displayed on the display unit 180 .

[0100] The user can move the remote control device 200 up and down, left and right, or rotate it. A cursor 205 displayed on the display unit 180 of the display device 100 corresponds to the movement of the remote control device 200. As shown in the figure, since the cursor 205 is displayed and moves correspondingly as the remote control device 200 moves in the 3D space, this device can be called a spatial remote control.

[0101] Figure 4 (b) in the figure illustrates that when the user moves the remote control device 200 to the left, the cursor 205 displayed on the display unit 180 of the display device 100 also moves to the left accordingly.

[0102] Information about the movement of the remote control device 200 detected by the sensor of the remote control device 200 is transmitted to the display device 100. The display device 100 can calculate the coordinates of the cursor 205 from the information about the movement of the remote control device 200. The display device 100 can display the cursor 205 corresponding to the calculated coordinates.

[0103] Figure 4 (c) illustrates a case where, while a specific button in remote control device 200 is pressed, the user moves remote control device 200 away from display unit 180. As a result, the selected area in display unit 180 corresponding to cursor 205 can be zoomed in and displayed in an enlarged manner.

[0104] In contrast, when the user moves the remote control device 200 closer to the display unit 180 , the selection area of the display unit 180 corresponding to the cursor 205 may be zoomed out and displayed in a reduced size.

[0105] On the other hand, when the remote control device 200 is away from the display unit 180 , the selection area may be expanded, and when the remote control device 200 is close to the display unit 180 , the selection area may be expanded.

[0106] Furthermore, when a specific button on the remote control device 200 is pressed, recognition of up, down, left, and right movements may be excluded. Specifically, when the remote control device 200 is moved closer to or further away from the display unit 180, up, down, left, and right movements may not be recognized, and only forward and backward movement may be recognized. When a specific button on the remote control device 200 is not pressed, only the cursor 205 moves in response to the up, down, left, and right movements of the remote control device 200.

[0107] On the other hand, the movement speed or movement direction of the cursor 205 may correspond to the movement speed or movement direction of the remote control apparatus 200 .

[0108] Meanwhile, the term "cursor" in this specification refers to an object displayed on the display unit 180 in response to an action of the remote control device 200. Therefore, the cursor 205 can be an object of various shapes other than the arrow shape shown in the drawings. For example, it can include concepts such as a point, a pointer, a prompt, and a thick outline. Furthermore, the cursor 205 can be displayed not only corresponding to a single point on the horizontal or vertical axis of the display unit 180, but also corresponding to multiple points such as a line or a surface.

[0109] The display device 100 of the embodiment may include a control part 170 that performs tone mapping to adjust the brightness of input image data.

[0110] The display device 100 may include a display portion 180 that displays an image of output image data whose brightness is adjusted based on the tone mapping performed in the control portion 170 .

[0111] The control unit 170 can generate a base mapping curve for the entire region from the input image data. The control unit 170 can extract information about each local region of the entire region and generate a local mapping curve reflecting the information about each local region for each local region. The control unit 170 can perform tone mapping for each local region using the local mapping curve reflecting the information about each local region. An image based on the output image data subjected to the local mapping performed by the control unit 170 for each local region can be displayed on the display unit 180.

[0112] The basic mapping curve may include a dynamic contract curve of the entire region, and the local mapping curve may include a local contract curve of the corresponding local region.

[0113] The control unit 170 may extract information about each local region of the entire area and generate adaptive local contrast curves for each local region from the mapping curve for the entire area based on the extracted information about each local region. The control unit 170 may perform tone mapping for each local region using the adaptive local contrast curves generated for each local region.

[0114] The display portion 180 may be provided as a flat display portion, a curved display portion having a curvature, or a bendable display portion whose curvature can be changed.

[0115] When the display portion 180 is provided as a flexible display portion, as an example, a curvature change request signal for the display portion 180 may be input from the user. As an example, the curvature change request signal input from the user may be input through the user input interface portion 150. The user input interface portion 150 may transmit the input curvature change request signal for the display portion 180 to the control portion 170. The control portion 170 may change the curvature of the display portion 180 according to the curvature change request signal from the user.

[0116] As an example, the user may transmit a curvature change request signal of the display unit 180 through the remote control device 200 .

[0117] The display unit 180 can be provided to enable continuous curvature change, allowing selection of an arbitrary curvature and change of the curvature. Alternatively, the display unit 180 can select a curvature stage from a predetermined plurality of curvature stages and change the curvature. For example, the curvature of the display unit 180 can be implemented as a flat surface, a first curvature of 1000R, a second curvature of 800R, and the like.

[0118] According to the display device 100 of the embodiment, as an example, if the user presses a specific button of the remote control device 200, the curvature of the display portion 180 may be sequentially changed to a flat surface, a first curvature, a second curvature, and so on.

[0119] Furthermore, according to the display device 100 of the embodiment, the curvature of the display unit 180 may be continuously changed while the user is pressing a specific button of the remote control device 200, or the curvature of the display unit 180 may be selected at the time when the user stops pressing the specific button.

[0120] Below, refer to Figure 5 , further describing the process of performing tone mapping in the display device of the embodiment.

[0121] Figure 5 is a flowchart showing a method for driving a display device according to an embodiment of the present invention.

[0122] The control unit 170 may generate a base mapping curve for the entire area ( S501 ).

[0123] According to the driving method of the display device 100 of the embodiment, the control part 170 may generate a basic mapping curve for the entire area from input image data.

[0124] As an example, the basic mapping curve may be generated by performing HDR (High Dynamic Range) processing on input image data.

[0125] The basic mapping curve generated by the control unit 170 may include a DTM (Dynamic Tone Mapping) curve and a DC (Dynamic Contrast) curve.

[0126] The control unit 170 may receive metadata along with the image data. The input image data may be an HDR image. The metadata may include information about the input image data. The metadata may include at least one of brightness information of the HDR image, maximum brightness information of each scene, and information for identifying the image as an HDR image.

[0127] In addition, the control unit 170 may calculate a brightness distribution histogram for the entire region of the input image data. The brightness distribution histogram of the HDR image may refer to distribution information of the brightness value of each of the plurality of pixels in the HDR image. The control unit 170 may calculate a brightness distribution histogram as a distribution diagram of the signal level (e.g., 0 to 1023) of each of the plurality of pixels in the HDR image.

[0128] The control unit 170 can generate a DTM (Dynamic Tone Mapping) curve based on the brightness distribution histogram information of the input image data. The DTM curve can refer to a mapping curve in the RGB domain. In the RGB domain, the input image data can be mapped to the output image data based on the generated DTM curve.

[0129] The control unit 170 may generate a DC (Dynamic Contrast) curve. The DC curve may refer to a mapping curve in the luminance Y domain. In the luminance Y domain, input image data may be mapped to output image data according to the generated DC curve.

[0130] On the other hand, since it is well known that the control unit 170 generates a DTM curve and a DC curve in order to generate output image data suitable for the characteristics of the display unit 180 from input image data, detailed description thereof is omitted here.

[0131] Next, the control unit 170 may analyze information of each local area ( S503 ).

[0132] According to the driving method of the display device 100 of the embodiment, the control unit 170 can extract information of each partial region of the entire area. The control unit 170 can extract information of each partial region constituting the entire area.

[0133] As an example, information about each local area can be extracted from a detail map generated based on pixel information of the entire area. The detail map can include APL (Average Picture Level) information and sharpness information of each local area.

[0134] Reference Figure 6 and Figure 7 , describing detailed diagrams of an embodiment.

[0135] Figure 6 1 is a diagram showing an example of an HDR-processed image according to an embodiment of the present invention. Figure 7 According to an embodiment of the present invention, Figure 6 Detail view of the image shown.

[0136] Figure 6 FIG. 1 is a diagram showing an image processed by HDR according to an embodiment. The entire area of the image can be divided into a plurality of local areas.

[0137] Taking into account image processing speed and performance, the plurality of local regions can be divided into any number within the entire region. For example, if the entire image region has a resolution of 1920*1080, the plurality of local regions can be provided in the number of 96*54. This is merely an example, and the size and number of the plurality of local regions can be varied as needed.

[0138] Figure 7 Yes Figure 6 A graph showing the level of detail for the entire area of the image shown. The level of detail can be the amount of change in the brightness values of adjacent pixels within a certain distance. The level of detail for each pixel can refer to the step (or rate of change) of change between that pixel and its neighbors. Pixels with larger color or brightness changes compared to surrounding pixels may indicate pixels with a high level of detail.

[0139] As an example, the detail map can be extracted by applying a Laplacian filter to the entire area. The detail map can also be extracted using various other tools or other methods.

[0140] The information of each local area can be extracted from a detail map generated based on the pixel information of the entire area. The detail map can include APL (Average Picture Level) information and sharpness information of each local area.

[0141] The APL information of each local area may refer to the brightness of each local area, and the sharpness information of each local area may refer to the detail level of each local area.

[0142] As an example, Figure 6 and Figure 7 The first region R1 shown can be determined as a region with no change in detail level. Figure 6 and Figure 7 The first region R1 shown can be determined as a region without sharpness.

[0143] in addition, Figure 6 and Figure 7 The second region R2 and the third region R3 shown can be determined as regions having variations in the degree of detail. Figure 6 and Figure 7 The second region R2 and the third region R3 shown can be judged as regions having sharpness.

[0144] It can be seen that Figure 6 and Figure 7 The APL levels of the illustrated first region R1 , second region R2 , and third region R3 are different from each other.

[0145] As described above, the control unit 170 can extract the APL information and sharpness information of each local area from the detail map.

[0146] Next, the control unit 170 may apply a local mapping curve to each local area ( S505 ).

[0147] The following describes a process in which the control unit 170 of the display device 100 according to the embodiment generates a local mapping curve for each local area and applies the generated local mapping curve to tone mapping of the corresponding local area.

[0148] The display device 100 of the embodiment of the present invention does not use the same mapping curve to perform tone mapping in the entire area. The display device 100 of the embodiment of the present invention can generate local mapping curves for each local area reflecting information of each local area, and perform tone mapping using the corresponding local mapping curve for each local area.

[0149] The display device 100 of the embodiment may generate an adaptive local contrast curve for each local area based on the extracted information of each local area, and perform tone mapping for each local area.

[0150] The control unit 170 may generate a local mapping curve reflecting information of each local area for each local area. The control unit 170 may generate a local mapping curve for each local area using APL information and sharpness information of each local area.

[0151] For example, the APL and sharpness levels of the first, second, and third regions R1, R2, and R3 differ from one another. Specifically, the APL values and sharpness values of each region differ from one another. Consequently, different local mapping curves can be generated for each region.

[0152] The control unit 170 may generate a first local mapping curve for the first region R1 based on the APL information and sharpness information of the first region. The control unit 170 may generate a second local mapping curve for the second region R2 based on the APL information and sharpness information of the second region. The control unit 170 may generate a third local mapping curve for the third region R3 based on the APL information and sharpness information of the third region.

[0153] For example, the first region R1 may be determined to be a region with no change in detail level. That is, the first region R1 may be determined to be a region with no sharpness. The control unit 170 may perform local tone mapping using the base mapping curve generated for the entire region, without generating a separate first local mapping curve for the first region R1.

[0154] The second region R2 may be determined to have a varying degree of detail. That is, the second region R2 may be determined to have sharpness. The control unit 170 may generate a separate second local mapping curve for the second region R2 and perform local tone mapping for the second region R2 using the generated second local mapping curve.

[0155] As an example, the control unit 170 may generate a second local mapping curve that can enhance the brightness and contrast of the second region R2. If the sharpness of the local region is high, the control unit 170 may enhance the brightness and contrast of the corresponding local region. The control unit 170 may apply the second local mapping curve instead of the DC curve in the base mapping curve generated for the entire region to perform local tone mapping for the second region R2.

[0156] The third region R3 may be determined to have a varying degree of detail. That is, the third region R3 may be determined to have sharpness. The control unit 170 may generate a separate third local mapping curve for the third region R3 and perform local tone mapping for the third region R3 using the generated third local mapping curve.

[0157] As an example, the control unit 170 may generate a third local mapping curve that can enhance the brightness and contrast of the third region R3. If the sharpness of the local region is high, the control unit 170 may enhance the brightness and contrast of the corresponding local region. The control unit 170 may apply the third local mapping curve instead of the DC curve in the base mapping curve generated for the entire region to perform local tone mapping for the third region R3.

[0158] The control unit 170 may perform tone mapping for each local area using a local mapping curve reflecting information of each corresponding local area.

[0159] According to an embodiment, the local mapping curve may be generated by further adjusting a contrast curve reflecting information of each corresponding local area in a basic mapping curve generated for the entire area.

[0160] As an example, the local mapping curve of the first region R1 may include a DTM curve (applied to the RGB domain) of the base mapping curve generated for the entire region and a DC curve (applied to the luminance domain) of the base mapping curve generated for the entire region.

[0161] The local mapping curve of the second region R2 may include a DTM curve of a base mapping curve generated for the entire region (applied to the RGB domain) and a second local mapping curve generated for the second region (applied to the luminance domain).

[0162] The local mapping curve of the third region R3 may include a DTM curve of a basic mapping curve generated for the entire region (applied to the RGB domain) and a third local mapping curve generated for the third region (applied to the luminance domain).

[0163] As described above, according to the display device and the driving method thereof according to the embodiments of the present invention, information of a local area of an image can be used to improve the brightness ratio in the local area, thereby enhancing the stereoscopic effect of the image and the user's sense of immersion.

[0164] According to the display device and driving method thereof according to the embodiment of the present invention, the brightness and sharpness information of each local area of the image can be used to improve the brightness-darkness ratio in the local area, thereby improving the stereoscopic effect of the image and the user's immersion.

[0165] According to the display device and the driving method thereof according to the embodiments of the present invention, an adaptive local contrast curve can be generated for each corresponding local area based on the information extracted from each local area and tone mapping can be performed, thereby improving the light-dark ratio in the local area and improving the three-dimensional effect of the image and the user's immersion.

[0166] Figure 8 FIG. 1 is a diagram illustrating application of gain adjustment to a local area based on detail map information according to an embodiment of the present invention.

[0167] The control unit 170 of the display device 100 of the embodiment may extract sharpness information and APL information of each local area from the detail map, and determine the brightness enhancement degree and contrast enhancement degree of the corresponding local area.

[0168] The control unit 170 may use at least one of the sharpness information and the APL information of each local area to determine the brightness enhancement degree, the contrast enhancement degree, etc. of the corresponding local area.

[0169] As an example, Figure 8 As shown, the control unit 170 can extract the brightness enhancement degree and / or contrast enhancement degree in the corresponding local area by calculating the gain based on the sharpness degree and the gain based on the APL degree in each local area. Through this process, the control unit 170 can detect the local area of interest for which brightness enhancement and / or contrast enhancement is to be processed.

[0170] In addition, the control unit 170 may adjust the output gain with respect to the input by calculating a gain for a bright level and a gain for a dark level for the gain of the detected local region of interest.

[0171] Through this process, the control section 170 can determine the brightness enhancement degree and the contrast enhancement degree of the corresponding local area based on the sharpness information and the APL information in each local area.

[0172] On the other hand, the display device 100 in the above-mentioned embodiment has been described based on the case where the display portion 180 has a flat shape.

[0173] According to an embodiment, the display portion 180 may be provided as a curved display portion having a curvature. In addition, the display portion 180 may be provided as a bendable display portion having a variable curvature.

[0174] The control unit 180 of the embodiment generates a local mapping curve reflecting at least one of the APL information, sharpness information, position information, and curvature information of the corresponding local area for each local area when the display unit 180 is provided as a curved display unit or a flexible display unit, and performs tone mapping for each local area.

[0175] Figure 9 FIG. 1 is a diagram illustrating a change in curvature of a display device according to an embodiment of the present invention.

[0176] According to an embodiment, the local mapping curve may be generated by further adjusting a contrast curve reflecting at least one of APL information, sharpness information, position information, and curvature information of the corresponding local area in the basic mapping curve generated for the entire area.

[0177] As an example, the display device 100 can be provided as a flexible display device with variable curvature, in the form of a flat display portion 901, a display portion 903 with a first curvature 1000R, and a display portion 905 with a second curvature 800R, and the curvature of the display portion can be changed according to user request.

[0178] In the case where the display device 100 includes a flat display portion 901, the local mapping curves in the first region R1, the second region R2, and the third region R3 can be as shown in FIG. Figures 5 to 8 Determine as described.

[0179] When the display device 100 includes the display portion 903 having a first curvature, the control portion 170 can further enhance the brightness of the first region R1 lacking sharpness, compared to a case where the flat display portion 901 is provided. The control portion 170 can further enhance the brightness and contrast of the second region R2 having sharpness, compared to a case where the flat display portion 901 is provided. The control portion 170 can further enhance the brightness and contrast of the third region R3 having sharpness, compared to a case where the flat display portion 901 is provided.

[0180] When the display portion 905 having the second curvature is provided, the control portion 170 can further enhance the brightness of the first region R1 without sharpness, compared to the case where the display portion 903 having the first curvature is provided. For the second region R2 having sharpness, the control portion 170 can further enhance the brightness and contrast, compared to the case where the display portion 903 having the first curvature is provided. For the third region R3 having sharpness, the control portion 170 can further enhance the brightness and contrast, compared to the case where the display portion 903 having the first curvature is provided.

[0181] If the curvature of the corresponding local area is large, the control unit 170 of the display device 100 of the embodiment may enhance the brightness and contrast of the corresponding local area.

[0182] If the corresponding local area has a curvature and is located at the center of the entire area, the control unit 170 of the display device 100 of the embodiment can enhance the brightness. If the corresponding local area has a curvature and is located at the side of the entire area, the control unit 170 can enhance the contrast.

[0183] The control unit 170 may generate a first local mapping curve for the first region R1 based on the APL information, sharpness information, position information, and curvature information of the first region. For the second region R2, the control unit 170 may generate a second local mapping curve based on the APL information, sharpness information, position information, and curvature information of the second region. For the third region R3, the control unit 170 may generate a third local mapping curve based on the APL information, sharpness information, position information, and curvature information of the third region.

[0184] The control unit 170 may perform tone mapping for each local area using a local mapping curve reflecting information of each corresponding local area.

[0185] As described above, the flexible display device and driving method thereof according to the embodiments of the present invention can utilize information of a local area of an image to improve the brightness-darkness ratio in the local area, thereby enhancing the stereoscopic effect of the image and the user's sense of immersion.

[0186] According to the flexible display device and driving method thereof according to the embodiments of the present invention, the brightness and sharpness information of each local area of the image can be used to improve the brightness-to-darkness ratio in the local area, thereby improving the stereoscopic effect of the image and the user's immersion.

[0187] According to the flexible display device and driving method thereof according to the embodiments of the present invention, the brightness, sharpness information, and position information of each local area of the image can be used to improve the light-to-dark ratio in the local area, and enhance the stereoscopic effect of the image and the user's immersion.

[0188] According to the flexible display device and driving method thereof according to the embodiments of the present invention, the brightness, sharpness information, position information, and curvature information of each local area of the image can be used to further improve the stereoscopic effect of the image and the user's immersion.

[0189] According to the flexible display device and the driving method thereof according to the embodiments of the present invention, an adaptive local contrast curve is generated for each corresponding local area based on the information extracted from each local area and tone mapping is performed, thereby improving the light-dark ratio in the local area and enhancing the stereoscopic effect of the image and the user's immersion.

[0190] If the display unit 180 is provided as a flexible display unit, as an example, a curvature change request signal for the display unit 180 may be input from the user. As an example, the curvature change request signal input from the user may be input through the user input interface unit 150. The user input interface unit 150 may transmit the input curvature change request signal for the display unit 180 to the control unit 170. The control unit 170 may change the curvature of the display unit 180 according to the curvature change request signal from the user. The control unit 170 may adjust the local mapping curve of each local area according to the curvature of the display unit 180.

[0191] Figure 10 FIG. 1 is a diagram illustrating an example of a local area contrast curve applied to a local area according to an embodiment of the present invention.

[0192] The control unit 170 of the display device 100 of the embodiment may extract information of each local area of the entire area, and generate a local area contrast curve reflecting the information of each corresponding local area for each local area.

[0193] The local area contrast curve may refer to the aforementioned first local mapping curve, second local mapping curve, and third local mapping curve.

[0194] Figure 10 The first curve 1001 shown may be a DC curve of a base mapping curve for the entire region. The second curve 1003 is an example of a contrast curve for a local region.

[0195] The second curve 1003 can be compared with the reference Figures 5 to 8 The first local mapping curve, the second local mapping curve, and the third local mapping curve correspond to each other. In this case, the second curve 1003 can adjust the contrast according to the gain determined by the APL information and sharpness information of the local area. The first local mapping curve, the second local mapping curve, and the third local mapping curve can be generated based on the information of each local area and can be generated as local contrast curves with different shapes.

[0196] In addition, the second curve 1003 can be compared with the reference Figure 9 The first local mapping curve, the second local mapping curve, and the third local mapping curve correspond to each other. In this case, the second curve 1003 can adjust the contrast according to the gain determined by the APL information, sharpness information, position information, and curvature information of the local area. The first local mapping curve, the second local mapping curve, and the third local mapping curve can be generated based on the information of each local area and can be generated as local contrast curves with different shapes.

[0197] On the other hand, the control unit 170 of the display device 100 of the embodiment can set the maximum value of the second curve 1003. This is because when the contrast is excessively enhanced, problems such as low grayscale burial and high grayscale saturation may occur. Figure 10 When the second curve 1003 shown is a curve representing the maximum value set in the control unit 170, the first local mapping curve, the second local mapping curve, and the third local mapping curve generated for each local area can be generated to have values between the first curve 1001 and the second curve 1003.

[0198] Figure 11 FIG. 1 is a diagram illustrating an example of a local contrast curve applied to each APL according to an embodiment of the present invention.

[0199] The control unit 170 of the display device 100 of the embodiment may generate different adaptive local contrast curves according to APL information of local areas.

[0200] For example, when the APL level of a local area has a relatively low value, the control section 170 may generate a local contrast curve as a first curve 1101. When the APL level of a local area has a relatively intermediate value, the control section 170 may generate a local contrast curve as a second curve 1103. When the APL level of a local area has a relatively high value, the control section 170 may generate a local contrast curve as a third curve 1105.

[0201] As an example, when the APL level of a local area has a relatively intermediate value, the second curve 1103 may be set so that the APL value at a point where the ratio of the input value to the output value is 1 is larger than that of the first curve 1101. When the APL level of a local area has a relatively intermediate value, the second curve 1103 may be set so that the APL value at a point where the ratio of the input value to the output value is 1 is smaller than that of the third curve 1105.

[0202] As described above, the control unit 170 can adjust the shape of the local contrast curve for the corresponding local area based on the APL level of the local area. By measuring the degree of change in the local contrast curve for the corresponding local area based on the change in the APL level of the local area in the display device 100, it can be determined whether the concept of the adaptive local mapping curve proposed in the present invention has been applied. If the local contrast curve tends to change based on the change in the APL level of the local area, it can be determined that the concept of the adaptive local mapping curve proposed in the present invention has been applied.

[0203] Figure 12 FIG. 1 is a diagram illustrating a case where different sharpness levels are applied to the same APL according to an embodiment of the present invention.

[0204] The control unit 170 of the display device 100 of the embodiment may generate adaptive local contrast curves that are different from each other according to sharpness information of local areas.

[0205] For example, the control unit 170 may increase the local contrast in a plurality of local areas with the same APL level as the sharpness of the local area increases. The control unit 170 may generate a local contrast curve for each local area so that the bright area becomes brighter and the dark area becomes darker as the sharpness of the local area increases.

[0206] The control unit 170 may generate local contrast curves having different contrast tendencies for different sharpnesses at the same APL.

[0207] The control unit 170 may generate a local contrast curve having a relatively high contrast tendency for the first sharpness pattern 1201 having a relatively high sharpness value. The control unit 170 may generate a local contrast curve having a relatively intermediate contrast tendency for the second sharpness pattern 1203 having a relatively intermediate sharpness value. The control unit 170 may generate a local contrast curve having a relatively low contrast tendency for the third sharpness pattern 1205 having a relatively low sharpness value.

[0208] As described above, the control unit 170 can adjust the shape of the adaptive local contrast curve based on the sharpness of the local area. In the display device 100, the degree of change in local contrast is detected by luminance measurement in response to changes in the sharpness of the local area, thereby determining whether the adaptive local mapping curve concept proposed in the present invention has been applied. If the luminance measurement indicates a tendency for local contrast to change in response to changes in the sharpness of the local area, it can be determined that the adaptive local mapping curve concept proposed in the present invention has been applied.

[0209] Figure 13 FIG. 1 is a diagram illustrating a local area contrast curve at each APL according to a change in curvature and a case where different sharpness levels are applied at the same APL according to an embodiment of the present invention.

[0210] The control unit 170 of the embodiment can generate different adaptive local contrast curves based on the curvature information of the local area. The control unit 170 can generate different adaptive local contrast curves based on the curvature information and APL information of the local area. The control unit 170 can generate different adaptive local contrast curves based on the curvature information and sharpness information of the local area. The control unit 170 can generate different adaptive local contrast curves based on the curvature information, APL information, and sharpness information of the local area.

[0211] As described above, the control unit 170 can adjust the shape of the local contrast curve for the corresponding local area based on the curvature information of the local area. The display device 100 measures the degree of change in the local contrast curve for the corresponding area based on the change in the degree of curvature of the local area, thereby determining whether the adaptive local mapping curve concept proposed in the present invention has been applied. If the local contrast curve shows a tendency to change based on the change in the degree of curvature of the local area, it can be determined that the adaptive local mapping curve concept proposed in the present invention has been applied.

[0212] The control unit 170 may perform tone mapping for each local area using an adaptive local mapping curve reflecting information of each corresponding local area.

[0213] An image based on the output image data on which adaptive local tone mapping is performed for each local area in the control unit 170 may be displayed on the display unit 180 .

[0214] According to the display device and the driving method thereof according to the embodiment of the present invention, the information of the local area of the image can be used to improve the brightness and darkness ratio in the local area, thereby improving the stereoscopic effect of the image and the user's immersion.

[0215] According to the display device and driving method thereof according to the embodiment of the present invention, the APL information, sharpness information, and position information of a local area of an image can be used to improve the brightness and darkness ratio in the local area, thereby improving the stereoscopic effect of the image and the user's immersion.

[0216] According to the display device and the driving method thereof according to the embodiments of the present invention, the APL information, sharpness information, position information, and curvature information of a local area of an image can be used to further improve the stereoscopic effect of the image and the user's sense of immersion.

[0217] According to the display device and the driving method thereof according to the embodiment of the present invention, adaptive local tone mapping can be performed according to the corresponding local area based on the information of each local area of the image, thereby further improving the stereoscopic effect of the image and the user's immersion.

[0218] Figure 14 This is a flowchart for explaining an operating method of a display device according to another embodiment of the present invention.

[0219] The control unit 170 of the display device 100 may detect a region of interest from the HDR image ( S1401 ).

[0220] The control section 170 may perform tone mapping based on a base mapping curve on the input HDR image, and detect a region of interest from the tone-mapped image.

[0221] use Figure 5 The description of the underlying mapping curve is replaced by the description of the

[0222] The control unit 170 may detect one or more regions of interest from the HDR image. Each region of interest may be a region including an object.

[0223] In one embodiment, the control unit 170 may detect the object using an object recognition model based on an artificial neural network trained through deep learning or machine learning. The artificial neural network may use at least one of a deep neural network (DNN) and a convolutional neural network (CNN).

[0224] The object can represent one of humans, animals, and microorganisms.

[0225] The object recognition model may be a single-stage YOLO (You Only Look Once) model or a two-stage Faster R-CNN (Regions with Convolution Neural Networks) model.

[0226] The YOLO (You Only Look Once) model is a model that can predict the objects present in an image and the locations of the corresponding objects at one time.

[0227] The YOLO (You Only Look Once) model divides the original image into grids of equal size. For each grid, it predicts the number of bounding boxes centered in the grid and specified in a predefined pattern, and calculates reliability based on this.

[0228] Then, it is possible to determine whether the image includes an object or only has the background, select a position with high object reliability, and grasp the object category.

[0229] The Faster R-CNN (Regions with Convolution Neural Networks) model is explained. First, a CNN (Convolution Neural Network) model extracts feature maps from an image. Based on these feature maps, multiple regions of interest (RoIs) are extracted. RoI pooling is performed on each RoI.

[0230] RoI pooling is a process of setting a grid so that the feature map projected onto the region of interest matches a preset H×W size, and extracting the maximum value from each cell included in each grid, thereby extracting a feature map with an H×W size.

[0231] A feature vector may be extracted from the feature map having an H×W size, and identification information of the object may be acquired from the feature vector.

[0232] The object recognition model may be stored in the storage unit 140 .

[0233] The control unit 170 may recognize an object using an object recognition model and classify the object into a plurality of elements. If the object represents a person, the plurality of elements may include a head and a body.

[0234] The control unit 170 may acquire one or more sub-images from the detected region of interest ( S1403 ).

[0235] Below, a map can contain the RGB information of an image. The map can be generated by the brightness value based on the RGB data.

[0236] In one embodiment, when the detected region of interest includes an object representing a person, the one or more sub-maps may include a head map corresponding to the head of the person and a body map corresponding to the body of the person.

[0237] The head image may be an image generated based on pixel information of the area where the person's head is located. The pixel information may include one or more of APL information and sharpness information.

[0238] The body map may be a map generated based on pixel information of a region where the body of a person is located.

[0239] One or more sub-images may further include a depth map, which may be a map representing the overall perspective.

[0240] The depth map may be a map that applies contrast differently depending on the location of an object, and may more clearly represent the contrast difference between a nearby object and a distant object.

[0241] The control unit 170 extracts a detail map from the HDR image ( S1405 ).

[0242] The detail map may be a map generated based on pixel information of the entire area of the HDR image.

[0243] In one embodiment, the detail map can be extracted by applying a Laplacian filter to the entire area. The detail map can also be extracted using various other tools or other methods.

[0244] use Figure 7 The description replaces the detailed description of the detail drawing.

[0245] The control unit 170 extracts image gain by combining the plurality of sub-images and the detail image ( S1407 ).

[0246] The control unit 170 may generate a merged image by merging a plurality of sub-images and a detail image, and may calculate an image gain from the merged image.

[0247] Figure 15 FIG. 1 is a diagram illustrating a process of extracting a plurality of sub-images for an object detected from an HDR image and obtaining a gain of a merged image from the extracted plurality of sub-images according to an embodiment of the present invention.

[0248] Reference Figure 15 , an HDR image 1500 is shown.

[0249] The control unit 170 may detect the object 1501 from the HDR image 1500 using an object recognition model.

[0250] The control unit 170 can extract the head and body from the detected object 1501 .

[0251] The control unit 170 may generate a head map 1510 based on the extracted pixel information of the region where the head is located.

[0252] The control unit 170 may generate a body map 1530 based on the pixel information of the extracted region where the body is located.

[0253] The control unit 170 may generate a detail map 1550 from the HDR image 1500 .

[0254] The control unit 170 may generate a merged image 1570 by merging the head image 1510 , the body image 1530 , and the detail image 1550 .

[0255] The control unit 170 may calculate a graph gain from the merged graph 1570 .

[0256] The graph gain may have a range of 1 to 2.

[0257] The image gain can be extracted per pixel, with the gain approaching 2 as the pixel approaches white, and approaching 1 as the pixel approaches black.

[0258] Again Figure 14 .

[0259] The control unit 170 may generate a mixed curve by calculating the image gain and the local area contrast curve ( S1409 ).

[0260] The control unit 170 may generate a mixed curve by multiplying the image gain of each image data level and the local area contrast curve.

[0261] The control unit 170 of the display device 100 may extract information of each local area of the entire area, and generate a local area contrast curve reflecting the information of each corresponding local area for each local area.

[0262] The local area contrast curve may refer to the aforementioned first local mapping curve, second local mapping curve, and third local mapping curve.

[0263] The control unit 170 may divide the merged image into a plurality of local image regions, and generate a blending curve by multiplying the image gain of each local image region and the local region contrast curve.

[0264] The control unit 170 may generate a number of mixed curves corresponding to the number of local area contrast curves.

[0265] For example, assume that the entire area of the HDR image is divided into three local areas.

[0266] The control part 170 may generate the first mixing curve by multiplying the first local mapping curve corresponding to the first local area and the first map gain corresponding to the first local area.

[0267] The control part 170 may generate a second mixing curve by multiplying the second local mapping curve corresponding to the second local area and the second map gain corresponding to the second local area.

[0268] The control part 170 may generate a third mixing curve by multiplying a third local mapping curve corresponding to the third local area and a third map gain corresponding to the third local area.

[0269] Figure 16 FIG. 4 is a diagram illustrating a process of generating a blending curve by multiplying a map gain by a local area contrast curve according to an embodiment of the present invention.

[0270] Reference Figure 16 , shows a local area contrast curve 1610 generated by reflecting local area information from the basic mapping curve 1601.

[0271] The local area contrast curve 1610 may be based on Figure 5 An embodiment of the present invention is based on a curve obtained based on information of local areas constituting the entire area.

[0272] The control unit 170 may generate a blending curve 1630 by multiplying the gain of the merged map 1570 generated based on the head map, the body map, the depth map, and the detail map and the local area contrast curve.

[0273] According to the blending curve 1630 , the brightness / darkness ratio of the region of interest of the HDR image is maximized, thereby significantly improving the sense of presence and stereoscopic effect.

[0274] Again Figure 14 .

[0275] The control unit 170 performs tone mapping of the HDR image based on the blending curve ( S1411 ).

[0276] The control unit 170 may output image data by tone mapping the input HDR image data in accordance with the blending curve.

[0277] Figure 17a and Figure 17b FIG. 1 is a diagram illustrating a method of identifying an object and applying a feature map of elements of the object to verify whether to generate a tone mapping curve according to an embodiment of the present invention.

[0278] Reference Figure 17a , the display device 100 displays a first HDR image 1710 including a face image 1711 of a person to be recognized as an object on the display unit 180 .

[0279] Figure 17b In the embodiment, the display device 100 displays the second HDR image 1730 including the background image 1731 which is not recognized as an object on the display unit 180 .

[0280] The first HDR image 1710 and the second HDR image 1730 are identical except for the face image 1711 and the background image 1731 .

[0281] The positions of the face image 1711 and the background image 1731 are identical to each other.

[0282] According to an embodiment of the present invention, if an object is recognized from an HDR image, a blending curve is generated using a graph of elements constituting the recognized object, and tone mapping is performed based on the generated blending curve, the brightness of the face image 1711 and the brightness of the background image 1731 are different from each other.

[0283] This is because the brightness measured by the mixed curve when the object, the face image 1711, is recognized and the brightness measured by the local area contrast curve when the object is not recognized are different from each other.

[0284] In contrast, if the embodiment of the present invention is not applied and the object recognition process is not performed, and tone mapping is performed according to the local area contrast curve, the brightness of the face image 1711 and the brightness of the background image 1731 are the same as each other.

[0285] This is because tone mapping is performed on both the face image 1711 and the background image 1731 based on a local area contrast curve to which the object map (merged map) based on object recognition is not applied.

[0286] Figure 18 is a diagram illustrating a process of generating an output image through tone mapping based on object recognition of an input HDR image according to an embodiment of the present invention.

[0287] Figure 18 The process may be executed by a control unit including one or more processors.

[0288] The control part 170 may perform global tone mapping on the entire area of the input HDR image.

[0289] The control unit 170 may perform global tone mapping on the input HDR image using the base mapping curve.

[0290] The base mapping curve can include Figure 5The dynamic tone mapping curve (DTM curve) and dynamic contrast curve (DC curve) described in.

[0291] The control unit 170 may receive the HDR image and metadata of the HDR image together. The metadata of the HDR image may include at least one of brightness information of the HDR image, maximum brightness information of each scene, and information for identifying that the image is an HDR image.

[0292] The control unit 170 may extract an object using an object recognition model after performing global tone mapping, and separate the extracted object into a head region and a body region.

[0293] The control unit 170 may generate a head map for the head region, a body map for the body region, a depth map and a detail map for the entire region, and generate a merged map by merging the generated head map, body map, depth map and detail map.

[0294] The control unit 170 may calculate a graph gain for a merged graph.

[0295] On the other hand, the control unit 170 may generate a local tone mapping curve to be calculated with the map gain.

[0296] The control unit 170 may divide the HDR image into a plurality of local areas.

[0297] When the entire area of the HDR image has a resolution of 1920*1080, the plurality of local areas may be provided in the number of 96*54. The control unit 170 may calculate a brightness distribution histogram indicating the brightness distribution of each local area.

[0298] The brightness distribution histogram may refer to distribution information of the brightness value of each of a plurality of pixels constituting each local area.

[0299] The control unit 170 may calculate a brightness distribution histogram, which is a distribution graph of signal levels (for example, 0 to 1023) of each of a plurality of pixels in each local area of the HDR image.

[0300] The control unit 170 may calculate the local APL of the local area using the brightness distribution histogram.

[0301] The control unit 170 may generate a local tone mapping curve using the calculated local APL.

[0302] The control unit 170 may generate a mixed tone mapping curve (mixing curve) by calculating the map gain of the merged map generated based on the object recognition model and the local tone mapping curve.

[0303] The graph gain can have a value of 1 to 2.

[0304] The control unit 170 may generate a mixed tone mapping curve (mixed curve) by multiplying the map gain and the local tone mapping curve.

[0305] The control unit 170 may map the input HDR image data to the output image data in accordance with the generated hybrid tone mapping curve, and may output the output image reflecting the mapping result to the display unit 180 .

[0306] As described above, according to the embodiments of the present invention, the image quality of the region of interest to the viewer of the HDR image can be improved by performing tone mapping on the region of interest.

[0307] This can maximize the sense of presence and three-dimensionality of HDR images, thereby significantly improving the competitiveness of image quality.

[0308] The above description is merely an exemplary description of the technical concept of the present invention. A person skilled in the art in the art may make various modifications and variations without departing from the essential characteristics of the present invention.

[0309] Therefore, the embodiments disclosed in the present invention are not intended to limit the technical concept of the present invention, but are intended to illustrate. These embodiments do not limit the scope of the technical concept of the present invention.

[0310] The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the right scope of the present invention.

Claims

1. A display device, wherein: Display unit; A region of interest and a detail map are extracted from the entire area of an input image, one or more sub-maps are extracted from the extracted region of interest, the detail map and the one or more sub-maps are merged to generate a merged image, a mixing curve is generated based on an image gain of the generated merged image and a local mapping curve generated based on local area information of each of a plurality of local areas constituting the entire area, tone mapping of the input image is performed based on the generated mixing curve, and an output image on which the tone mapping has been performed is displayed on the display unit.

2. The display device according to claim 1, wherein The control unit generates the mixing curve by multiplying the map gain and the local mapping curve.

3. The display device according to claim 2, wherein: The map gain has a range of 1 to 2.

4. The display device according to claim 1, wherein The system further includes a storage unit that stores an object recognition model based on an artificial neural network that is trained by deep learning and recognizes an object included in the region of interest.

5. The display device according to claim 4, wherein In a case where the recognized object represents a person, the one or more sub-graphs include a head graph corresponding to the head of the person and a body graph corresponding to the body of the person; The control unit generates the merged image using the head image, the body image, and the detail image, wherein the detail image is generated based on pixel information of the entire area of the input image; Each of the head map, the body map, and the detail map includes one or more of APL information and sharpness information of a corresponding region. The display device according to claim 1 , wherein: The control unit generates the local curve for each local area from a base mapping curve for tone mapping the entire area of the input image based on the local area information.

7. The display device according to claim 6, wherein: The local mapping curve is generated by further adjusting the basic mapping curve to form a contrast curve reflecting information of each local area.

8. The display device according to claim 1, wherein When tone mapping is performed by applying the blending curve to the region of interest, the control unit measures a first luminance of the region of interest; When tone mapping is performed on the region of interest by applying only the local mapping curve, the control unit measures a second luminance on the region of interest; The first brightness and the second brightness are different from each other.

9. A method for operating a display device, wherein: include: The step of extracting the region of interest and the detail map from the entire region of the input image; A step of extracting one or more sub-images from the extracted region of interest; a step of merging the detail image and the one or more sub-images to generate a merged image; generating a blending curve based on the graph gain of the generated merged graph and a local mapping curve generated based on local region information of each of the plurality of local regions constituting the entire region; and The step of performing tone mapping of the input image based on the generated blending curve.

10. The method for operating a display device according to claim 9, wherein: The step of generating the mixing curve includes the step of generating the mixing curve by multiplying the map gain and the local mapping curve.

11. The method for operating a display device according to claim 10, wherein: The map gain has a range of 1 to 2.

12. The method for operating a display device according to claim 8, wherein: The method further includes the step of identifying an object contained in the region of interest using an object recognition model based on an artificial neural network trained by deep learning.

13. The method for operating a display device according to claim 12, wherein: In a case where the recognized object represents a person, the one or more sub-graphs include a head graph corresponding to the head of the person and a body graph corresponding to the body of the person; The step of generating the merged map includes the step of generating the merged map using the head map, the body map, and the detail map, wherein the detail map is generated based on pixel information of the entire area of the input image; Each of the head map, the body map, and the detail map includes one or more of APL information and sharpness information of a corresponding region.

14. The method for operating a display device according to claim 8, wherein: The method further includes generating the local curve for each local area from a basic mapping curve used for tone mapping the entire area of the input image based on the local area information.

15. The method for operating a display device according to claim 14, wherein: The local mapping curve is generated by further adjusting the basic mapping curve to form a contrast curve reflecting information of each local area.

Images