Image display device

Abstract

The present application relates to an image display device. An image display device according to an embodiment of the present application includes a display and a signal processing device that signal-processes an input image and outputs an output image; the signal processing device performs, in parallel, a first super-resolution process and a second super-resolution process based on the input image, and outputs, as the output image, a synthesized image based on result data of the first super-resolution process and result data of the second super-resolution process. Thus, a plurality of super-resolution processes can be efficiently performed.

Description

Technical Field

[0001] The present invention relates to an image display apparatus, and more specifically, to an image display apparatus capable of efficiently performing a plurality of super-resolution processes. Background Technology

[0002] An image display device is a device for displaying images.

[0003] Image display devices can display images stored internally or images received from the outside.

[0004] On the other hand, when a low-resolution image is input into an image display device, it is preferable to perform super-resolution processing in order to display a high-resolution image.

[0005] Korean Patent Publication No. 10-2019-0131205, which is the first existing document, relates to a super-resolution network processing method and system. It discloses a method for upsampling a received image based on a convolutional neural network (CNN) and restoring the upsampled image to output a high-resolution image (HR).

[0006] However, due to the serial structure of super-resolution processing, the subsequent network depends on the result data of the previous step, which has the disadvantage of limited learning or tuning.

[0007] Korean Patent No. 10-2467092, which is existing document 2, relates to a super-resolution image processing method and system that is robust to coding noise using a plurality of neural networks. It discloses that, based on the level of coding noise generated during the compression of low-resolution images, a plurality of neural networks capable of generating super-resolution images are used to perform super-resolution image processing that is robust to coding noise.

[0008] However, existing literature 2 has the drawback of using multiple neural networks just to reduce the coding noise of the image. Summary of the Invention

[0009] The problem that the invention aims to solve

[0010] The technical problem to be solved by the present invention is to provide an image display device capable of efficiently performing multiple super-resolution processes.

[0011] Another technical problem to be solved by the present invention is to provide an image display device that can efficiently improve the clarity of the edges and textures of an image.

[0012] Technical solutions to the problem

[0013] An image display apparatus according to an embodiment of the present invention for solving the above-mentioned problems includes: a display; and a signal processing apparatus for processing an input image and outputting an output image; the signal processing apparatus performs a first super-resolution processing and a second super-resolution processing in parallel based on the input image, and outputs a synthesized image as an output image based on the result data of the first super-resolution processing and the result data of the second super-resolution processing.

[0014] On the other hand, the signal processing device can increase the sharpness or resolution of the edges and textures of objects within the input image based on the first super-resolution processing and the second super-resolution processing performed in parallel.

[0015] On the other hand, during the first super-resolution processing, the signal processing device can perform learning based on the first learning database, and during the second super-resolution processing, it can perform learning based on a second learning database that is different from the first learning database.

[0016] On the other hand, the signal processing apparatus may include a neural network processor for learning processing; the neural network processor may perform learning based on a first learning database during the first super-resolution processing, and may perform learning based on a second learning database different from the first learning database during the second super-resolution processing.

[0017] On the other hand, during the first super-resolution processing, the signal processing device can perform learning based on the first resolution image and the second resolution image; during the second super-resolution processing, it can perform learning based on the third resolution image, which has a lower resolution than the first resolution image, and the already processed image.

[0018] On the other hand, the signal processing device can extract a first feature map based on the result data of the first super-resolution processing, and can extract a second feature map based on the result data of the second super-resolution processing; the signal processing device can calculate the synthesis ratio of the result data of the first super-resolution processing and the result data of the second super-resolution processing based on the first feature map and the second feature map; the signal processing device can output the synthesized image as the output image based on the calculated synthesis ratio.

[0019] On the other hand, the signal processing device can change the synthesis ratio on a pixel-by-pixel basis.

[0020] On the other hand, the signal processing device can change the synthesis ratio based on the type or resolution of the content.

[0021] On the other hand, the signal processing apparatus may include: a first super-resolution processing unit that performs first super-resolution processing based on an input image; and a second super-resolution processing unit that performs second super-resolution processing in parallel with the first super-resolution processing unit based on the input image.

[0022] On the other hand, the signal processing apparatus may further include: a first feature map extraction unit that extracts a first feature map based on the result data of the first super-resolution processing; a second feature map extraction unit that extracts a second feature map based on the result data of the second super-resolution processing; a feature map analysis unit that calculates the synthesis ratio of the result data of the first super-resolution processing and the result data of the second super-resolution processing based on the first feature map and the second feature map; and an image synthesis unit that outputs the synthesized image as an output image based on the synthesis ratio calculated in the feature map analysis unit.

[0023] On the other hand, the signal processing device can perform a first super-resolution process and a second super-resolution process in parallel based on the input image, according to the super-resolution mode, and output the synthesized image as the output image based on the result data of the first super-resolution process and the result data of the second super-resolution process.

[0024] On the other hand, when the signal processing device is set to sports mode, it can perform first super-resolution processing and second super-resolution processing in parallel based on the input image, and output the synthesized image as the output image based on the result data of the first super-resolution processing and the result data of the second super-resolution processing.

[0025] On the other hand, when the input image is a dynamic image, the signal processing device can perform a first super-resolution processing and a second super-resolution processing in parallel based on the input dynamic image, and output the synthesized dynamic image as the output dynamic image based on the result data of the first super-resolution processing and the result data of the second super-resolution processing.

[0026] On the other hand, the signal processing device can also perform a third super-resolution process in parallel based on the input image, and output the synthesized image as the output image based on the result data of the first super-resolution process, the result data of the second super-resolution process, and the result data of the first super-resolution process.

[0027] On the other hand, the signal processing device can increase the sharpness or resolution of the edges and textures of objects in the input image based on the first and second super-resolution processes executed in parallel; the signal processing device can increase the sharpness or resolution of flat areas within objects in the input image based on the third super-resolution process executed in parallel.

[0028] On the other hand, the signal processing device can extract a first feature map based on the result data of the first super-resolution processing, extract a second feature map based on the result data of the second super-resolution processing, extract a third feature map based on the result data of the third super-resolution processing, calculate the synthesis ratio of the result data of the first super-resolution processing, the result data of the second super-resolution processing, and the result data of the third super-resolution processing based on the first feature map, the second feature map, and the third feature map, and output the synthesized image as the output image based on the calculated synthesis ratio.

[0029] On the other hand, the signal processing device can change the synthesis ratio on a pixel-by-pixel basis.

[0030] On the other hand, the signal processing apparatus may include: a first super-resolution processing unit that performs a first super-resolution processing based on an input image; a second super-resolution processing unit that performs a second super-resolution processing in parallel with the first super-resolution processing unit based on the input image; and a third super-resolution processing unit that performs a third super-resolution processing in parallel with the first super-resolution processing unit and the first super-resolution processing unit based on the input image.

[0031] On the other hand, the signal processing apparatus may further include: a first feature map extraction unit that extracts a first feature map based on the result data of the first super-resolution processing; a second feature map extraction unit that extracts a second feature map based on the result data of the second super-resolution processing; a third feature map extraction unit that extracts a third feature map based on the result data of the third super-resolution processing; a feature map analysis unit that calculates the synthesis ratio of the result data of the first super-resolution processing, the result data of the second super-resolution processing, and the result data of the third super-resolution processing based on the first feature map, the second feature map, and the third feature map; and an image synthesis unit that outputs the synthesized image as an output image based on the synthesis ratio calculated in the feature map analysis unit.

[0032] Another embodiment of the image display device of the present invention includes: a display; and a signal processing device, which performs signal processing on an input image and outputs an output image; the signal processing device performs a first super-resolution processing, a second super-resolution processing and a third super-resolution processing in parallel based on the input image, and outputs a synthesized image as an output image based on the result data of the first super-resolution processing, the result data of the second super-resolution processing and the result data of the third super-resolution processing.

[0033] Invention Effects

[0034] An image display apparatus according to an embodiment of the present invention includes: a display; and a signal processing device that performs signal processing on an input image and outputs an output image; the signal processing device performs a first super-resolution process and a second super-resolution process in parallel based on the input image, and outputs a synthesized image as an output image based on the result data of the first super-resolution process and the result data of the second super-resolution process. Thus, multiple super-resolution processes can be performed efficiently. Furthermore, the sharpness of image edges and textures can be effectively improved.

[0035] On the other hand, the signal processing device can increase the sharpness or resolution of the edges and textures of objects within the input image based on the parallel execution of the first and second super-resolution processes. This allows for efficient improvement of the sharpness or resolution of the image's edges and textures.

[0036] On the other hand, the signal processing device can perform learning based on a first learning database during the first super-resolution processing, and can perform learning based on a second learning database different from the first learning database during the second super-resolution processing. Therefore, it is possible to efficiently improve the sharpness of image edges and textures while efficiently performing multiple super-resolution processes.

[0037] On the other hand, the signal processing apparatus may include a neural network processor for learning processing; during the first super-resolution processing, the neural network processor may perform learning based on a first learning database, and during the second super-resolution processing, it may perform learning based on a second learning database different from the first learning database. Thus, it is possible to efficiently improve the sharpness of image edges and textures while efficiently performing multiple super-resolution processes.

[0038] On the other hand, during the first super-resolution processing, the signal processing device can perform learning based on the first-resolution image and the second-resolution image; during the second super-resolution processing, it can perform learning based on the third-resolution image (which has a lower resolution than the first-resolution image) and the already processed image. Thus, it is possible to efficiently improve the sharpness of image edges and textures while efficiently performing multiple super-resolution processes.

[0039] On the other hand, the signal processing device can extract a first feature map based on the result data of the first super-resolution processing, and can extract a second feature map based on the result data of the second super-resolution processing; the signal processing device can calculate the synthesis ratio of the result data of the first super-resolution processing and the result data of the second super-resolution processing based on the first feature map and the second feature map; the signal processing device can output the synthesized image as the output image based on the calculated synthesis ratio. Thus, while efficiently performing multiple super-resolution processes, it is possible to efficiently improve the edge and texture clarity of the image.

[0040] On the other hand, the signal processing device can change the synthesis ratio on a pixel-by-pixel basis. This allows for efficient performance of multiple super-resolution processes while simultaneously improving the sharpness of image edges and textures.

[0041] On the other hand, the signal processing device can change the synthesis ratio based on the type or resolution of the content. This allows for efficient performance of multiple super-resolution processes while simultaneously improving the sharpness of image edges and textures.

[0042] On the other hand, the signal processing apparatus may include: a first super-resolution processing unit that performs a first super-resolution process based on the input image; and a second super-resolution processing unit that performs a second super-resolution process in parallel with the first super-resolution processing unit based on the input image. Thus, it is possible to efficiently improve the sharpness of image edges and textures while efficiently performing a plurality of super-resolution processes.

[0043] On the other hand, the signal processing apparatus may further include: a first feature map extraction unit that extracts a first feature map based on the result data of the first super-resolution processing; a second feature map extraction unit that extracts a second feature map based on the result data of the second super-resolution processing; a feature map analysis unit that calculates the synthesis ratio of the result data of the first super-resolution processing and the result data of the second super-resolution processing based on the first and second feature maps; and an image synthesis unit that outputs the synthesized image as an output image based on the synthesis ratio calculated in the feature map analysis unit. Thus, while efficiently performing multiple super-resolution processes, it is possible to efficiently improve the edge and texture clarity of the image.

[0044] On the other hand, the signal processing device, based on the super-resolution mode, can perform a first super-resolution process and a second super-resolution process in parallel based on the input image, and output the synthesized image as the output image based on the result data of the first and second super-resolution processes. Thus, it is possible to efficiently improve the sharpness of image edges and textures while efficiently performing multiple super-resolution processes.

[0045] On the other hand, when the signal processing device is set to sports mode, it can perform a first super-resolution process and a second super-resolution process in parallel based on the input image, and output the synthesized image as the output image based on the result data of the first super-resolution process and the result data of the second super-resolution process. Thus, it is possible to efficiently improve the sharpness of image edges and textures while efficiently performing multiple super-resolution processes.

[0046] On the other hand, when the input image is a moving image, the signal processing device can perform a first super-resolution process and a second super-resolution process in parallel based on the input moving image, and output the synthesized moving image as the output moving image based on the result data of the first super-resolution process and the result data of the second super-resolution process. Thus, it is possible to efficiently improve the sharpness of the image's edges and textures while efficiently performing multiple super-resolution processes.

[0047] On the other hand, the signal processing device can also perform a third super-resolution process in parallel based on the input image, and output the synthesized image as the output image based on the result data of the first super-resolution process, the result data of the second super-resolution process, and the result data of the first super-resolution process. Thus, it is possible to efficiently improve the edge and texture clarity of the image while efficiently performing multiple super-resolution processes.

[0048] On the other hand, the signal processing device can increase the sharpness or resolution of the edges and textures of objects within the input image based on the first and second super-resolution processes executed in parallel; the signal processing device can also increase the sharpness or resolution of flat regions within objects within the input image based on the third super-resolution process executed in parallel. Thus, it is possible to efficiently improve the sharpness of image edges and textures while efficiently performing multiple super-resolution processes.

[0049] On the other hand, the signal processing device can extract a first feature map based on the result data of the first super-resolution processing, extract a second feature map based on the result data of the second super-resolution processing, and extract a third feature map based on the result data of the third super-resolution processing. It can also calculate the synthesis ratio of the result data of the first, second, and third super-resolution processing based on the first, second, and third feature maps, and output the synthesized image as the output image based on the calculated synthesis ratio. Thus, it can efficiently improve the edge and texture clarity of the image while efficiently performing multiple super-resolution processes.

[0050] On the other hand, the signal processing device can change the synthesis ratio on a pixel-by-pixel basis. This allows for efficient performance of multiple super-resolution processes while simultaneously improving the sharpness of image edges and textures.

[0051] On the other hand, the signal processing apparatus may include: a first super-resolution processing unit that performs a first super-resolution process based on an input image; a second super-resolution processing unit that performs a second super-resolution process in parallel with the first super-resolution processing unit based on the input image; and a third super-resolution processing unit that performs a third super-resolution process in parallel with the first super-resolution processing unit and the first super-resolution processing unit based on the input image. Thus, it is possible to efficiently perform a plurality of super-resolution processes while efficiently improving the sharpness of image edges and textures.

[0052] On the other hand, the signal processing apparatus may further include: a first feature map extraction unit that extracts a first feature map based on the result data of the first super-resolution processing; a second feature map extraction unit that extracts a second feature map based on the result data of the second super-resolution processing; a third feature map extraction unit that extracts a third feature map based on the result data of the third super-resolution processing; a feature map analysis unit that calculates the synthesis ratio of the result data of the first super-resolution processing, the result data of the second super-resolution processing, and the result data of the third super-resolution processing based on the first feature map, the second feature map, and the third feature map; and an image synthesis unit that outputs the synthesized image as an output image based on the synthesis ratio calculated in the feature map analysis unit. Thus, while efficiently performing multiple super-resolution processes, it is possible to efficiently improve the edge and texture clarity of the image.

[0053] Another embodiment of the image display apparatus of the present invention includes: a display; and a signal processing device that performs signal processing on an input image and outputs an output image; the signal processing device performs a first super-resolution processing, a second super-resolution processing, and a third super-resolution processing in parallel based on the input image, and outputs a synthesized image as an output image based on the result data of the first super-resolution processing, the result data of the second super-resolution processing, and the result data of the third super-resolution processing. Thus, multiple super-resolution processing operations can be performed efficiently. Furthermore, the sharpness of image edges and textures can be effectively improved. Attached Figure Description

[0054] Figure 1 This is a diagram illustrating an image display device according to an embodiment of the present invention.

[0055] Figure 2 yes Figure 1 An example of an internal block diagram of an image display device.

[0056] Figure 3 yes Figure 2 An example of an internal block diagram of a signal processing device.

[0057] Figure 4a It is shown Figure 2 A diagram illustrating the control method of a remote control device.

[0058] Figure 4b yes Figure 2 Internal block diagram of the remote control device.

[0059] Figure 5 This is a flowchart illustrating the operation of an image display device according to an embodiment of the present invention.

[0060] Figure 6 This is an example of an internal block diagram of a signal processing device according to an embodiment of the present invention.

[0061] Figures 7 to 10c It is used for explanation Figure 5 or Figure 6 The image.

[0062] Figure 11 This is a flowchart illustrating the operation of an image display device according to another embodiment of the present invention.

[0063] Figure 12 This is an example of an internal block diagram of a signal processing apparatus according to another embodiment of the present invention. Detailed Implementation

[0064] The present invention will now be described in further detail with reference to the accompanying drawings.

[0065] The suffixes “module” and “section” used in the following description of the constituent elements are assigned solely for the convenience of writing this specification and do not inherently possess any particularly important meaning or function. Therefore, the terms “module” and “section” may be used interchangeably.

[0066] Figure 1 This is a diagram illustrating an image display device according to an embodiment of the present invention.

[0067] Referring to the accompanying drawings, the image display device 100 may include a display 180.

[0068] On the other hand, the display 180 can be implemented using any type of panel. For example, the display 180 can be any type of liquid crystal display panel (LCD panel), organic light-emitting panel (OLED panel), inorganic light-emitting panel (LED panel), etc.

[0069] In addition to the panel used for image display, LCD panels may also require additional backlighting.

[0070] On the other hand, organic or inorganic light-emitting panels do not require additional backlighting to display images.

[0071] The image display device 100 can be controlled by an internal tuning unit ( Figure 2 (e.g., 105) to receive broadcast signals.

[0072] Alternatively, the image display device 100 can be connected to a set-top box (STB) and receive broadcast or image signals through the set-top box (STB).

[0073] On the other hand, the image display device 100 can receive image signals from the outside via wired or wireless networks.

[0074] On the other hand, the image display device 100 can receive image signals from the outside via wired or wireless networks.

[0075] On the other hand, the image display device 100 can receive image signals from a connected external device.

[0076] The external device at this time can be a set-top box (STB), a mobile terminal, a USB-type storage device or tablet, or a set-top box or laptop connected by HDMI, etc.

[0077] on the other hand, Figure 1 The image display device 100 can be a TV, a display screen, a tablet PC, a mobile terminal, etc.

[0078] Figure 2 yes Figure 1 An example of an internal block diagram of an image display device.

[0079] Reference Figure 2 An image display device 100 according to an embodiment of the present invention may include an image receiving unit 105, an external device interface unit 130, a storage unit 140, a user input interface unit 150, a sensor unit (not shown), a signal processing device 170, a display 180, and an audio output unit 185.

[0080] An image display device 100 according to an embodiment of the present invention may further include a power supply unit 190 and a microcomputer 173.

[0081] The image receiving unit 105 may include a tuning unit 110, a demodulation unit 120, a network interface unit 130, and an external device interface unit 130.

[0082] On the other hand, unlike the accompanying drawings, the image receiving unit 105 may also include only the tuning unit 110, the demodulation unit 120, and the external device interface unit 130. That is, the network interface unit 130 may not be included.

[0083] The tuning unit 110 selects an RF broadcast signal corresponding to a channel selected by the user or all pre-stored channels from the RF (Radio Frequency) broadcast signals received via an antenna (not shown). Additionally, it converts the selected RF broadcast signal into an intermediate frequency signal or a baseband image or voice signal.

[0084] For example, if the selected RF broadcast signal is a digital broadcast signal, it is converted into a digital IF signal (DIF); if it is an analog broadcast signal, it is converted into an analog baseband image or voice signal (CVBS / SIF). That is, the tuning unit 110 can process digital or analog broadcast signals. The analog baseband image or voice signal (CVBS / SIF) output from the tuning unit 110 can be directly input to the signal processing device 170.

[0085] On the other hand, in order to receive broadcast signals from multiple channels, the tuning unit 110 may have multiple tuners. Alternatively, it may be a single tuner that simultaneously receives broadcast signals from multiple channels.

[0086] The demodulation unit 120 receives the digital IF signal (DIF) converted in the tuning unit 110 and performs demodulation.

[0087] The demodulation unit 120 can output a streaming signal (TS) after performing demodulation and channel decoding. At this time, the streaming signal can be a signal multiplexed from an image signal, a voice signal, or a data signal.

[0088] The streaming signal output from the demodulation unit 120 can be input to the signal processing device 170. After performing demultiplexing, image / speech signal processing, etc., the signal processing device 170 outputs an image on the display 180 and outputs speech to the audio output unit 185.

[0089] The external device interface section 130 can transmit and receive data with a connected external device (not shown), such as a set-top box 50. For this purpose, the external device interface section 130 may include an A / V input / output section (not shown).

[0090] The external device interface unit 130 can be connected to external devices such as DVD (Digital Versatile Disk), Blu-ray, game devices, cameras, camcorders, computers (laptops), set-top boxes, etc., in wired / wireless manner, and can also perform input / output operations with external devices.

[0091] The A / V input / output section can receive image and voice signals from external devices. On the other hand, the wireless communication section (not shown) can perform short-range wireless communication with other electronic devices.

[0092] Through this wireless communication unit (not shown), the external device interface unit 130 can exchange data with the adjacent mobile terminal 600. In particular, in mirror mode, the external device interface unit 130 can receive device information, running application information, application images, etc. from the mobile terminal 600.

[0093] The network interface unit 135 provides an interface for connecting the image display device 100 to a wired / wireless network, including the Internet. For example, the network interface unit 135 can receive content or data provided by the Internet or a content provider or network operator via a network.

[0094] On the other hand, the network interface unit 135 may include a wireless communication unit (not shown).

[0095] The storage unit 140 can store programs for processing and controlling various signals within the signal processing device 170, and can also store processed image, voice, or data signals.

[0096] In addition, the storage unit 140 can also perform the function of temporarily storing image, voice, or data signals input through the external device interface unit 130. Furthermore, the storage unit 140 can store information related to a designated broadcast channel through channel memory functions such as channel mapping.

[0097] Figure 2 An embodiment is shown where the storage unit 140 and the signal processing device 170 are separately arranged, but the scope of the invention is not limited thereto. The storage unit 140 may be included within the signal processing device 170.

[0098] The user input interface 150 transmits signals input by the user to the signal processing device 170, or transmits signals from the signal processing device 170 to the user.

[0099] For example, user input signals such as power on / off, channel selection, and screen settings can be sent / received from the remote control device 200, or user input signals input by local keys (not shown) such as power button, channel button, volume button, and setting button can be transmitted to the signal processing device 170, or user input signals input by a sensor unit (not shown) that senses the user's gestures can be transmitted to the signal processing device 170, or signals from the signal processing device 170 can be sent to the sensor unit (not shown).

[0100] The signal processing device 170 can demultiplex the stream input through the tuning unit 110, demodulation unit 120, network interface unit 135, or external device interface unit 130, or generate and output signals for image or voice output by processing the demultiplexed signals.

[0101] For example, the signal processing device 170 can receive broadcast signals or HDMI signals received from the image receiving unit 105, perform signal processing based on the received broadcast signals or HDMI signals, and output the processed image signals.

[0102] The image signal processed in the signal processing device 170 can be input to the display 180 and displayed as an image corresponding to the corresponding image signal. In addition, the image signal processed in the signal processing device 170 can be output to an external output device through the external device interface unit 130.

[0103] The voice signal processed in the signal processing device 170 can be output as sound to the audio output unit 185. In addition, the voice signal processed in the signal processing device 170 can be input to an external output device through the external device interface unit 130.

[0104] Although not in Figure 2 As shown, the signal processing apparatus 170 may include a demultiplexing unit, an image processing unit, etc. That is, the signal processing apparatus 170 can perform various signal processing operations, and thus can be implemented as a system-on-chip (SOC). This will be discussed later. Figure 3 Please provide an explanation.

[0105] In addition, the signal processing device 170 can control the overall operation within the image display device 100. For example, the signal processing device 170 can control the tuning unit 110 to select RF broadcasts corresponding to a user-selected channel or a pre-stored channel.

[0106] In addition, the signal processing device 170 can control the image display device 100 according to user instructions or internal programs input through the user input interface 150.

[0107] On the other hand, the signal processing device 170 can control the display 180 to display an image. At this time, the image displayed on the display 180 can be a static image or a dynamic image, and can be a 2D image or a 3D image.

[0108] On the other hand, the signal processing device 170 can display a specified object within the image displayed on the display 180. For example, the object can be at least one of the following: a linked webpage (newspaper, magazine, etc.), an EPG (Electronic Program Guide), various menus, controls, icons, still images, moving images, and text.

[0109] On the other hand, the signal processing device 170 can identify the user's position based on the image captured by the imaging unit (not shown). For example, it can determine the distance (z-axis coordinate) between the user and the image display device 100. In addition, it can determine the x-axis and y-axis coordinates within the display 180 corresponding to the user's position.

[0110] The display 180 generates drive signals by converting image signals, data signals, OSD signals, control signals processed in the signal processing device 170 or image signals, data signals, control signals received from the external device interface unit 130.

[0111] On the other hand, the display 180 is configured as a touch screen, which can be used as an input device in addition to being an output device.

[0112] The audio output unit 185 receives the signal that has been processed by the voice in the signal processing device 170 and outputs it as voice.

[0113] A camera unit (not shown) captures images of the user. The camera unit (not shown) can be implemented using a single camera, but is not limited to this; it can also be implemented using multiple cameras. The image information captured by the camera unit (not shown) can be input to the signal processing device 170.

[0114] The signal processing device 170 can detect the user's gesture based on each or a combination of images captured by the imaging unit (not shown) or signals detected by the sensor unit (not shown).

[0115] The power supply unit 190 can supply power to the entire image display device 100. In particular, the power supply unit 190 can supply power to the signal processing device 170, which can be implemented as a system on chip (SOC), the display 180 for image display, and the audio output unit 185 for audio output.

[0116] Specifically, the power supply unit 190 may include a function that converts AC voltage to DC voltage. DC / DC converters and DC / DC converters that convert the level of DC voltage.

[0117] The remote control device 200 can send user input to the user input interface unit 150. For this purpose, the remote control device 200 can use Bluetooth, RF (Radio Frequency) communication, infrared (IR) communication, UWB (Ultra-Wideband), ZigBee, etc. Furthermore, the remote control device 200 receives image, voice, or data signals output from the user input interface unit 150 and displays or outputs them via voice on the remote control device 200.

[0118] On the other hand, the image display device 100 described above can be a digital broadcast receiver capable of receiving fixed or mobile digital broadcasts.

[0119] on the other hand, Figure 2The block diagram of the image display device 100 shown is a block diagram for an embodiment of the present invention. The various components of the block diagram can be integrated, added to, or omitted according to the specifications of the actually implemented image display device 100. That is, two or more components can be combined into one component, or one component can be split into two or more components, as needed. Furthermore, the functions performed in each block are used to illustrate embodiments of the present invention, and their specific actions or devices do not limit the scope of the present invention.

[0120] Figure 3 yes Figure 2 An example of an internal block diagram of a signal processing device.

[0121] Referring to the accompanying drawings, a signal processing apparatus 170 according to an embodiment of the present invention may include a demultiplexing unit 310, an image processing unit 320, a processor 330, and an audio processing unit 370. In addition, it may also include a data processing unit (not shown).

[0122] The demultiplexing unit 310 demultiplexes the input stream. For example, in the case of an input MPEG-2 TS, it can be demultiplexed to separate the image, voice, and data signals. Here, the stream signal input to the demultiplexing unit 310 can be a stream signal output from the tuning unit 110, the demodulation unit 120, or the external device interface unit 130.

[0123] The image processing unit 320 can perform signal processing on the input image. For example, the image processing unit 320 can perform image processing on the image signal demultiplexed by the demultiplexing unit 310.

[0124] Therefore, the image processing unit 320 may include an image decoder 325, a scaler 335, an image quality processing unit 635, an image encoder (not shown), a graphics processing unit 340, a frame rate conversion unit 350, and a formatter 360, etc.

[0125] The image decoder 325 decodes the demultiplexed image signal, and the scaler 335 scales the resolution of the decoded image signal so that it can be output on the display.

[0126] The image decoder 325 can have decoders of various specifications. For example, MPEG-2 and H.264 decoders, 3D image decoders for color and depth images, and decoders for multi-view images.

[0127] The scaler 335 can scale the input image signal that has been decoded in the image decoder 325, etc.

[0128] For example, when the size or resolution of the input image signal is small, the scaler 335 can perform upsampling, and when the size or resolution of the input image signal is large, it can perform downsampling.

[0129] The image quality processing unit 635 can perform image quality processing on the input image signal that has been decoded in the image decoder 325, etc.

[0130] For example, the image quality processing unit 635 can perform noise removal processing on the input image signal, or expand the grayscale resolution of the input image signal, or perform image resolution enhancement, or perform signal processing based on high dynamic range (HDR), or change the frame rate, or perform image quality processing corresponding to panel characteristics, especially the panel.

[0131] The graphics processing unit 340 generates OSD signals based on user input or autonomously. For example, it can generate signals for displaying various information as graphics or text on the screen of the display 180 based on user input signals. The generated OSD signals can include various data such as user interface screens, menu screens, controls, and icons of the image display device 100. In addition, the generated OSD signals can include 2D objects or 3D objects.

[0132] Furthermore, the graphics processing unit 340 can generate a cursor that can be displayed on the screen based on the cursor signal input from the remote control device 200. In particular, such a cursor can be generated in a cursor signal processing unit, and the graphics processing unit 240 may include such a cursor signal processing unit (not shown). Of course, the cursor signal processing unit (not shown) may also be provided separately and not within the graphics processing unit 240.

[0133] The frame rate converter (FRC) 350 can convert the frame rate of the input image. On the other hand, the frame rate converter 350 can also output directly without the need for separate frame rate conversion.

[0134] On the other hand, Formatter 360 can convert the format of the input image signal into an image signal that can be displayed on the monitor and then output it.

[0135] In particular, Formatter 360 can change the format of image signals to correspond to the display panel.

[0136] On the other hand, Formatter360 can also change the format of image signals.

[0137] The processor 330 can control the overall operation within the image display device 100 or the signal processing device 170.

[0138] For example, the processor 330 can control the tuning of the tuner 110 to select an RF broadcast corresponding to a user-selected channel or a pre-stored channel.

[0139] In addition, the processor 330 can control the image display device 100 according to user instructions or internal programs input through the user input interface 150.

[0140] In addition, the processor 330 can perform data transmission control with the network interface unit 135 or the external device interface unit 130.

[0141] In addition, the processor 330 can control the operation of the demultiplexing unit 310, the image processing unit 320, etc. in the signal processing device 170.

[0142] On the other hand, the audio processing unit 370 within the signal processing apparatus 170 can perform speech processing on the demultiplexed speech signal. Therefore, the audio processing unit 370 can have various decoders.

[0143] In addition, the audio processing unit 370 within the signal processing device 170 can process bass, treble, volume adjustment, etc.

[0144] The data processing unit (not shown) within the signal processing apparatus 170 can perform data processing on the demultiplexed data signal. For example, if the demultiplexed data signal is an encoded data signal, it can be decoded. The encoded data signal may be electronic program guide information containing broadcast information such as the start and end times of broadcast programs broadcast on various channels.

[0145] on the other hand, Figure 3 The block diagram of the signal processing apparatus 170 shown is a block diagram for one embodiment of the present invention. The various components of the block diagram may be integrated, added, or omitted according to the specifications of the actual implemented signal processing apparatus 170.

[0146] In particular, the frame rate conversion unit 350 and the formatter 360 can also be set independently of the image processing unit 320.

[0147] On the other hand, the signal processing apparatus 170 of one embodiment of the present invention may further include a neural network processor 333 for learning processing and the like.

[0148] Figure 4a It is shown Figure 2 A diagram illustrating the control method of a remote control device.

[0149] like Figure 4aAs shown in (a), a cursor 205 corresponding to the remote control device 200 is displayed on the display 180.

[0150] The user can adjust the remote control device up and down, left and right (200 degrees). Figure 4a (b) of, before and after ( Figure 4a (c) Movement or rotation. The cursor 205 displayed on the display 180 of the image display device corresponds to the movement of the remote control device 200. As shown, since such a remote control device 200 moves and displays the cursor 205 according to the movement in 3D space, it can be called a space remote control or a 3D cursor device.

[0151] Figure 4a Example (b) illustrates that if the user moves the remote control device 200 to the left, the cursor 205 displayed on the display 180 of the image display device also moves to the left accordingly.

[0152] Information related to the movement of the remote control device 200, detected by the sensors of the remote control device 200, is transmitted to the image display device. The image display device can calculate the coordinates of the cursor 205 from the information related to the movement of the remote control device 200. The image display device can then display the cursor 205 in a manner corresponding to the calculated coordinates.

[0153] Figure 4a Example (c) illustrates a case where, with a specific button within the remote control device 200 pressed, the user moves the remote control device 200 away from the display 180. As a result, the selection area on the display 180 corresponding to the cursor 205 can be zoomed in and enlarged. Conversely, when the user moves the remote control device 200 closer to the display 180, the selection area on the display 180 corresponding to the cursor 205 can be zoomed out and reduced in size. Alternatively, the selection area can be zoomed out when the remote control device 200 is away from the display 180, and zoomed in when the remote control device 200 is close to the display 180.

[0154] On the other hand, when a specific button within the remote control device 200 is pressed, the recognition of up / down and left / right movement can be excluded. That is, when the remote control device 200 moves away from or closer to the display 180, only forward / backward movement can be recognized, while up / down, left / right movement is not recognized. When the specific button within the remote control device 200 is not pressed, only the cursor 205 moves according to the up / down, left / right movement of the remote control device 200.

[0155] On the other hand, the moving speed or moving direction of the cursor 205 can correspond to the moving speed or moving direction of the remote control device 200.

[0156] Figure 4b yes Figure 2 Internal block diagram of the remote control device.

[0157] Referring to the accompanying drawings, the remote control device 200 may include a wireless communication unit 425, a user input unit 435, a sensor unit 440, an output unit 450, a power supply unit 460, a storage unit 470, and a control unit 480.

[0158] The wireless communication unit 425 transmits and receives signals with any of the image display devices in the aforementioned embodiments of the present invention. In the image display devices of the present invention embodiments, one image display device 100 will be described as an example.

[0159] In this embodiment, the remote control device 200 may have an RF module 421, which can transmit and receive signals with the image display device 100 according to the RF communication standard. Additionally, the remote control device 200 may have an IR module 423, which can transmit and receive signals with the image display device 100 according to the IR communication standard.

[0160] In this embodiment, the remote control device 200 transmits a signal including information related to the movement of the remote control device 200 to the image display device 100 via the RF module 421.

[0161] Additionally, the remote control device 200 can receive signals transmitted from the image display device 100 via the RF module 421. Furthermore, the remote control device 200 can transmit commands related to power on / off, channel change, volume change, etc., to the image display device 100 via the IR module 423 as needed.

[0162] The user input unit 435 may be composed of a keyboard, buttons, a touchpad, or a touch screen. The user can operate the user input unit 435 to input commands related to the image display device 100 into the remote control device 200. If the user input unit 435 has physical buttons, the user can input commands related to the image display device 100 into the remote control device 200 by pressing the physical buttons. If the user input unit 435 has a touch screen, the user can input commands related to the image display device 100 into the remote control device 200 by touching the soft keys on the touch screen. Furthermore, the user input unit 435 may have various types of input methods that the user can operate, such as scroll keys or rotary keys; this embodiment does not limit the scope of the invention.

[0163] The sensor unit 440 may include a gyroscope sensor 441 or an accelerometer sensor 443. The gyroscope sensor 441 can sense information related to the movement of the remote control device 200.

[0164] As an example, the gyroscope sensor 441 can sense information related to the operation of the remote control device 200 with reference to the x, y, and z axes. The accelerometer sensor 443 can sense information related to the moving speed of the remote control device 200. On the other hand, a distance measurement sensor may also be included, thereby allowing the sensing of the distance between the device and the display 180.

[0165] The output unit 450 can output image or voice signals corresponding to the operation of the user input unit 435 or to the signals transmitted from the image display device 100. Through the output unit 450, the user can identify whether the user input unit 435 is operated or whether the image display device 100 is controlled.

[0166] As an example, the output unit 450 may include an LED module 451 that lights up when operating the user input unit 435 or transmitting and receiving signals with the image display device 100 via the wireless communication unit 425, a vibration module 453 that generates vibration, a sound output module 455 that outputs sound, or a display module 457 that outputs images.

[0167] The power supply unit 460 supplies power to the remote control device 200. When the remote control device 200 remains stationary for a specified period, the power supply unit 460 interrupts the power supply, thereby reducing power waste. The power supply unit 460 can restore power when a designated key on the remote control device 200 is activated.

[0168] The storage unit 470 can store various types of programs, application data, etc., required for the control or operation of the remote control device 200. If the remote control device 200 wirelessly transmits and receives signals with the image display device 100 via the RF module 421, the remote control device 200 and the image display device 100 transmit and receive signals via a specified frequency band. The control unit 480 of the remote control device 200 can store information such as the frequency band of the image display device 100 that can wirelessly transmit and receive signals with the remote control device 200 in the storage unit 470 for reference.

[0169] The control unit 480 controls all matters related to the control of the remote control device 200. The control unit 480 can transmit signals corresponding to the key operations of the user input unit 435 or signals corresponding to the movement of the remote control device 200 sensed by the sensor unit 440 to the image display device 100 via the wireless communication unit 425.

[0170] The user input interface 150 of the image display device 100 may include: a wireless communication unit 151, which is capable of wirelessly transmitting and receiving signals with the remote control device 200; and a coordinate value calculation unit 415, which is capable of calculating the coordinate value of the cursor corresponding to the action of the remote control device 200.

[0171] The user input interface unit 150 can wirelessly transmit and receive signals with the remote control device 200 via the RF module 412. Additionally, via the IR module 413, it can receive signals transmitted by the remote control device 200 according to the IR communication standard.

[0172] The coordinate calculation unit 415 corrects for hand tremors or errors in the signal corresponding to the action of the remote control device 200 received by the wireless communication unit 151, thereby being able to calculate the coordinate values ​​(x, y) of the cursor 205 to be displayed on the display 170.

[0173] The transmission signal from the remote control device 200 input to the image display device 100 via the user input interface 150 is transmitted to the signal processing device 170 of the image display device 100. The signal processing device 170 determines information related to the operation and key operation of the remote control device 200 from the signal transmitted by the remote control device 200, and can control the image display device 100 accordingly.

[0174] As another example, the remote control device 200 can calculate the cursor coordinates corresponding to its actions and output them to the user input interface 150 of the image display device 100. In this case, the user input interface 150 of the image display device 100 can transmit information related to the received cursor coordinates to the signal processing device 170 without the need for an additional process to correct for hand tremors or errors.

[0175] Alternatively, as another example, unlike the attached figure, the coordinate value calculation unit 415 may also be located inside the signal processing device 170 instead of the user input interface unit 150.

[0176] Figure 5 This is a flowchart illustrating the operation of an image display device according to an embodiment of the present invention.

[0177] Referring to the accompanying drawings, in an embodiment of the present invention, the signal processing device 170 within the image display device 100 determines whether it is necessary to perform a super-resolution mode on the received input image (S5310).

[0178] For example, if the signal processing device 170 in the image display device 100 receives an input image with a first resolution and the display 180 has a second resolution that is higher than the first resolution, it can determine that a super-resolution mode needs to be performed on the input image.

[0179] As another example, when the resolution of the received input image is less than the reference resolution, the signal processing device 170 in the image display device 100 can determine that a super-resolution mode needs to be performed on the input image, regardless of the resolution of the display 180.

[0180] As another example, when the signal processing device 170 in the image display device 100 receives an input image whose first resolution is less than the reference resolution, and the display 180 has a second resolution that is higher than the first resolution, it can determine that a super-resolution mode needs to be performed on the input image.

[0181] Next, the signal processing apparatus 170 of one embodiment of the present invention enters the super-resolution mode when it is necessary to execute the super-resolution mode.

[0182] Next, according to the super-resolution mode, the signal processing apparatus 170 of one embodiment of the present invention can perform a first super-resolution processing and a second super-resolution processing in parallel based on the input image (S5320), and output the synthesized image as an output image based on the result data of the first super-resolution processing and the result data of the second super-resolution processing (S5330).

[0183] On the other hand, the signal processing device 170 can perform a first super-resolution process and a second super-resolution process in parallel based on the input image according to the super-resolution mode, and output the synthesized image as the output image based on the result data of the first super-resolution process and the result data of the second super-resolution process.

[0184] This enables the efficient execution of multiple super-resolution processes. Furthermore, it allows for the efficient improvement of image edge and texture sharpness.

[0185] On the other hand, the signal processing device 170 can increase the sharpness or resolution of the edges and textures of objects within the input image based on the first super-resolution processing and the second super-resolution processing executed in parallel.

[0186] On the other hand, during the first super-resolution processing, the signal processing device 170 can perform learning based on the first learning database, and during the parallel second super-resolution processing, it can perform learning based on a second learning database different from the first learning database. Therefore, it is possible to efficiently improve the sharpness of image edges and textures while efficiently performing multiple super-resolution processes.

[0187] For example, the signal processing device 170 may perform a first super-resolution process centered on the edge of an object in the input image, and perform a second super-resolution process in parallel centered on the texture of the object in the input image.

[0188] Therefore, it is possible to efficiently perform multiple super-resolution processes while effectively improving the sharpness of image edges and textures.

[0189] On the other hand, the signal processing apparatus 170 includes a neural network processor 333 for learning processing. During the first super-resolution processing, the neural network processor 333 can perform learning based on a first learning database, and during the second super-resolution processing, it can perform learning based on a second learning database different from the first learning database. Therefore, it is possible to efficiently improve the sharpness of image edges and textures while efficiently performing multiple super-resolution processes.

[0190] Figure 6 This is an example of an internal block diagram of a signal processing device according to an embodiment of the present invention.

[0191] Referring to the accompanying drawings, a signal processing apparatus 170 according to an embodiment of the present invention may include: a first super-resolution processing unit 610 that performs a first super-resolution processing based on an input image IMa; and a second super-resolution processing unit 620 that performs a second super-resolution processing in parallel with the first super-resolution processing unit 610 based on the input image IMa.

[0192] For example, in super-resolution mode, the signal processing device 170 can control the first super-resolution processing unit 610 and the second super-resolution processing unit 620 to operate in parallel.

[0193] On the other hand, during the first super-resolution processing, the first super-resolution processing unit 610 can perform learning based on the first learning database.

[0194] For example, during the first super-resolution processing, the first super-resolution processing unit 610 may utilize the neural network processor 333 to perform learning based on the first learning database.

[0195] Specifically, the first super-resolution processing unit 610 within the signal processing device 170 can perform learning based on the first resolution image and the second resolution image during the first super-resolution processing.

[0196] On the other hand, during the second super-resolution processing, the second super-resolution processing unit 620 can perform learning based on the second learning database.

[0197] For example, during the second super-resolution processing, the second super-resolution processing unit 620 can utilize the neural network processor 333 to perform learning based on a second learning database that is different from the first learning database.

[0198] On the other hand, the second super-resolution processing unit 620 within the signal processing device 170 can perform learning based on the second super-resolution processing, based on a third resolution image with a resolution lower than that of the first resolution image, and the processed image.

[0199] On the other hand, the first super-resolution processing unit 610 and the second super-resolution processing unit 620 can perform learning or adjustment learning independently, respectively.

[0200] On the other hand, the first super-resolution processing unit 610 mainly processes edges within the input image, while the second super-resolution processing unit 620 can process regions within the input image other than edges, such as textures.

[0201] Therefore, it is possible to efficiently perform multiple super-resolution processes while effectively improving the sharpness of image edges and textures.

[0202] On the other hand, the signal processing device 170 can extract a first feature map MPa based on the result data IMb of the first super-resolution processing, and can extract a second feature map MPb based on the result data IMc of the second super-resolution processing.

[0203] At this point, the result data IMb of the first super-resolution processing can be the result image of the first super-resolution processing, and the result data IMc of the second super-resolution processing can be the result image of the second super-resolution processing.

[0204] On the other hand, the signal processing device 170 can calculate the synthesis ratio RTa of the result data IMb of the first super-resolution processing and the result data IMc of the second super-resolution processing based on the first feature map MPa and the second feature map MPb, and output the synthesized image as the output image IMm based on the calculated synthesis ratio RTa.

[0205] For this purpose, the signal processing apparatus 170 may further include a first feature map extraction unit 612 for extracting a first feature map MPa based on the result data IMb of the first super-resolution processing and a second feature map extraction unit 622 for extracting a second feature map MPb based on the result data IMc of the second super-resolution processing.

[0206] On the other hand, the first feature map extraction unit 612 and the second feature map extraction unit 622 can extract the degree of improvement in resolution and sharpness as feature values ​​or feature maps MPa and MPb, respectively, based on the result data IMb and IMc of the super-resolution processing, in pixel units.

[0207] For example, the feature values ​​or feature maps MPa and MPb output from the first feature map extraction unit 612 and the second feature map extraction unit 622 may include the level or energy of the mid- and high-frequency components of the spatial frequency of the result data IMb and IMc of each super-resolution processing.

[0208] On the other hand, the first feature map extraction unit 612 and the second feature map extraction unit 622 may also use different feature values ​​or feature maps as feature values ​​or feature maps MPa, MPb of the degree of improvement in resolution and sharpness.

[0209] On the other hand, the signal processing apparatus 170 may further include: a feature map analysis unit 630 that calculates the synthesis ratio RTA of the result data IMb of the first super-resolution processing and the result data IMc of the second super-resolution processing based on the first feature map MPa and the second feature map MPb; and an image synthesis unit 640 that outputs the synthesized image as an output image IMm based on the synthesis ratio RTA calculated in the feature map analysis unit 630.

[0210] On the other hand, the image synthesis unit 640 can determine the synthesis ratio RTa of the result data IMb of the first super-resolution processing and the result data IMc of the second super-resolution processing, based on each feature value or feature map MPa, MPb, in pixel units.

[0211] For example, the image synthesis unit 640 can determine the optimal synthesis ratio RTa for improving resolution and sharpness based on various feature values ​​or feature maps MPa, MPb.

[0212] As another example, the image compositing unit 640 can change the compositing ratio RTa according to the individual image components for which resolution and sharpness are to be enhanced.

[0213] On the other hand, the feature map analysis unit 630 or the image synthesis unit 640 within the signal processing device 170 can change the synthesis ratio RTa in pixel units.

[0214] For example, the feature map analysis unit 630 or the image synthesis unit 640 within the signal processing apparatus 170 can be controlled to increase the ratio of the result data IMb of the first super-resolution processing near the edge region within the input image IMa.

[0215] As another example, the feature map analysis unit 630 or the image synthesis unit 640 within the signal processing device 170 can be controlled to increase the ratio of the result data IMc of the second super-resolution processing in the vicinity of the texture region within the input image IMa.

[0216] Therefore, it is possible to efficiently perform multiple super-resolution processes while effectively improving the sharpness of image edges and textures.

[0217] On the other hand, the feature map analysis unit 630 or the image synthesis unit 640 within the signal processing device 170 can change the synthesis ratio RTa based on the type or resolution of the content.

[0218] For example, the feature map analysis unit 630 or the image synthesis unit 640 can be controlled to increase the ratio of the result data IMc of the second super-resolution processing in the result data IMb of the first super-resolution processing and the result data IMc of the second super-resolution processing as the resolution increases.

[0219] Therefore, it is possible to efficiently improve the sharpness of image edges and textures in accordance with the image resolution.

[0220] As another example, the feature map analysis unit 630 or the image synthesis unit 640 can be controlled to increase the ratio of the first super-resolution processing result data IMb in the first super-resolution processing result data IMb and the second super-resolution processing result data IMc as the content moves or the movement vector increases.

[0221] Therefore, it is possible to efficiently improve the clarity of image edges and textures in response to the movement of content.

[0222] Thus, according to Figure 6 The first super-resolution processing unit 610 and the second super-resolution processing unit 620 within the signal processing device 170 perform super-resolution processing in parallel, thereby enabling learning or adjustment to be matched with each objective.

[0223] In addition, according to Figure 6 The first super-resolution processing unit 610 and the second super-resolution processing unit 620 within the signal processing device 170 are independent of each other during super-resolution processing and can be implemented relatively uncomplicatedly.

[0224] on the other hand, Figure 6 The feature map analysis unit 630 within the signal processing device 170 can finely adjust the feature map analysis to prevent a decrease in the resolution or clarity of the result data IMb and IMc of each super-resolution processing.

[0225] On the other hand, when the input image is a dynamic image, the signal processing device 170 can perform a first super-resolution processing and a second super-resolution processing in parallel based on the input dynamic image, and output the synthesized dynamic image as an output dynamic image based on the result data IMb of the first super-resolution processing and the result data IMc of the second super-resolution processing.

[0226] Therefore, it is possible to efficiently perform multiple super-resolution processes while effectively improving the sharpness of edges and textures in dynamic images.

[0227] Figures 7 to 10c It is used for explanation Figure 5 or Figure 6 The image.

[0228] Figure 7 This is a diagram illustrating super-resolution processing of the first input image.

[0229] Referring to the attached diagram, Figure 7 (a) illustrates the first input image 710.

[0230] In the case where the first input image 710 includes branches and leaves, the region Arx including a portion of the leaves is not clear, as is the region Arxa in the magnified image 712.

[0231] Figure 7 (b) illustrates the example based on Figure 6 The first super-resolution processing unit 610 and the second super-resolution processing unit 620 process the first input image and output the image 720.

[0232] The first super-resolution processing unit 610 performs edge region processing, while the second super-resolution processing unit 620 performs texture region processing in parallel, including making a portion of the leaf region Ara, such as region Araa in the magnified image 722, clear. In addition, the edge regions of the branches also become clear.

[0233] Therefore, it is possible to efficiently perform multiple super-resolution processes while effectively improving the sharpness of image edges and textures.

[0234] Figure 8 This is a diagram illustrating super-resolution processing of the second input image.

[0235] Referring to the attached diagram, Figure 8 (a) illustrates the second input image 810.

[0236] In the case that the second input image 810 includes various fruits, the edge regions RGx of the fruits are clear, but the regions OBx that include part of the texture of the fruits are not clear, such as the region TXx in the magnified image 812.

[0237] Figure 8 (b) illustrates the example based on Figure 6 The first super-resolution processing unit 610 and the second super-resolution processing unit 620 process the second input image to output the image 820.

[0238] The first super-resolution processing unit 610 performs edge region processing, while the second super-resolution processing unit 620 performs texture region processing in parallel, including making a portion of the fruit's region OBa, such as region Txa within the magnified image 822, clear. Additionally, the fruit's edge region EGa also becomes clear.

[0239] Therefore, it is possible to efficiently perform multiple super-resolution processes while effectively improving the sharpness of image edges and textures.

[0240] Figure 9 This is a diagram illustrating super-resolution processing of a third input image.

[0241] Referring to the attached diagram, Figure 9 (a) illustrates the third input image 910.

[0242] In the case of the third input image 910, which includes the rock wall and the sky, the area OBy, which includes a portion of the rock wall, is not clear, just like the area Txy in the magnified image 912.

[0243] Figure 9 (b) illustrates the example based on Figure 6 The first super-resolution processing unit 610 and the second super-resolution processing unit 620 process the third input image and output the image 920.

[0244] On the other hand, the first super-resolution processing unit 610 performs edge region processing, and the second super-resolution processing unit 620 performs texture region processing in parallel, including making a portion of the rock wall region OBb, such as region Txb in the magnified image 922, clear. In addition, the edge regions of the rock wall also become clear.

[0245] Therefore, it is possible to efficiently perform multiple super-resolution processes while effectively improving the sharpness of image edges and textures.

[0246] On the other hand, when the signal processing device 170 is set to sports mode or super-resolution mode, it can perform a first super-resolution process and a second super-resolution process in parallel based on the input image, and output the synthesized image as the output image IMm based on the result data IMb of the first super-resolution process and the result data IMc of the second super-resolution process. For this, see [reference needed]. Figure 10a This will be discussed later.

[0247] Figures 10a to 10c These are illustrations of various settings screens used for parallel processing of multiple super-resolution processes.

[0248] Figure 10a This example shows a setup screen.

[0249] Referring to the accompanying drawings, the signal processing device 170 can control the display setting screen 1010 based on the input signal.

[0250] The settings screen 1010 at this time may include a sports mode item 1012 for setting the sports mode and a normal mode item 1014 for setting the normal mode.

[0251] On the other hand, when the signal processing device 170 selects the sports mode item 1012 based on the input signal, such as Figure 10b As shown, the settings screen 1010b can be controlled to display the enabled item 1022 and disabled item 1024 of the super-resolution mode 1020.

[0252] On the other hand, when the signal processing device 170 selects the enabled item 1022 of the super-resolution mode 1020 based on the input signal, such as Figure 10c As shown, the settings screen 1010c can be controlled to display an edge and texture item 1032 and an edge, texture and flat area item 1034.

[0253] For example, when the edge and texture item 1032 is selected, the signal processing device 170, such as Figure 6 As shown, it can be controlled so that two super-resolution processing units operate in parallel, with the first super-resolution processing unit 610 performing processing on the edge regions of the image and the second super-resolution processing unit 620 performing processing on the texture regions of the image.

[0254] Therefore, it is possible to efficiently perform multiple super-resolution processes while effectively improving the sharpness of image edges and textures.

[0255] As another example, when the signal processing device 170 selects edge, texture, and flat area items 1034, such as Figure 12 As shown, it can be controlled so that three super-resolution processing units operate in parallel: the first super-resolution processing unit 610 performs processing on the edge regions of the image, the second super-resolution processing unit 620 performs processing on the texture regions of the image, and the third super-resolution processing unit 625 performs processing on the flat regions of the image.

[0256] That is, when the edge, texture, and flat area items 1034 are selected, the signal processing device 170 can interact with... Figure 6 The first and second super-resolution processes are performed together, and a third super-resolution process is also executed in parallel. Based on the result data IMb of the first super-resolution process, the result data IMc of the second super-resolution process, and the result data IMb of the first super-resolution process, the synthesized image is output as the output image IMm. Therefore, it is possible to efficiently improve the edge and texture sharpness of the image while efficiently performing multiple super-resolution processes.

[0257] Therefore, it is possible to efficiently improve the sharpness of image edges, textures, and flat areas while performing multiple super-resolution processes efficiently.

[0258] On the other hand, when edge, texture, and flat region items 1034 are selected, the signal processing device 170 can increase the sharpness or resolution of the edges and textures of objects within the input image based on the first and second super-resolution processes executed in parallel, and increase the sharpness or resolution of flat regions within objects within the input image based on the third super-resolution process executed in parallel. Thus, it is possible to efficiently improve the sharpness of image edges and textures while efficiently performing multiple super-resolution processes.

[0259] Figure 11 This is a flowchart illustrating the operation of an image display device according to another embodiment of the present invention.

[0260] Referring to the accompanying drawings, in another embodiment of the image display device 100, the signal processing device 170b determines whether it is necessary to perform a super-resolution mode on the received input image (S1110).

[0261] For example, if the signal processing device 170b in the image display device 100 receives an input image with a first resolution and the display 180 has a second resolution that is higher than the first resolution, it can determine that a super-resolution mode needs to be performed on the input image.

[0262] As another example, when the resolution of the received input image is less than the reference resolution, the signal processing device 170b in the image display device 100 can determine that a super-resolution mode needs to be performed on the input image.

[0263] Next, the signal processing apparatus 170b of another embodiment of the present invention enters the super-resolution mode when it is necessary to execute the super-resolution mode.

[0264] Next, according to another embodiment of the present invention, the signal processing apparatus 170b performs a first super-resolution process, a second super-resolution process, and a third super-resolution process in parallel based on the input image according to the super-resolution mode (S1120), and outputs the synthesized image as an output image based on the result data of the first super-resolution process, the result data of the second super-resolution process, and the result data of the third super-resolution process (S1130).

[0265] On the other hand, the signal processing device 170b, based on the super-resolution mode, can perform a first super-resolution process, a second super-resolution process, and a third super-resolution process in parallel based on the input image, and output the synthesized image as the output image based on the result data of the first super-resolution process, the result data of the second super-resolution process, and the result data of the third super-resolution process. Thus, multiple super-resolution processes can be performed efficiently.

[0266] On the other hand, the signal processing device 170b can increase the sharpness or resolution of the edges, textures and flat areas of objects in the input image based on the first super-resolution processing, the second super-resolution processing, the third super-resolution processing, which are executed in parallel.

[0267] On the other hand, during the first super-resolution processing, the signal processing apparatus 170b can perform learning based on a first learning database; during the parallel second super-resolution processing, it can perform learning based on a second learning database different from the first learning database; and during the parallel third super-resolution processing, it can perform learning based on a third learning database different from the second learning database. Thus, multiple super-resolution processes can be performed efficiently.

[0268] For example, the signal processing device 170b can perform a first super-resolution process centered on the edge of an object in the input image, can perform a second super-resolution process in parallel centered on the texture of an object in the input image, and can perform a third super-resolution process in parallel centered on a flat region of an object in the input image.

[0269] Therefore, it is possible to efficiently improve the sharpness of image edges, textures, and flat areas while performing multiple super-resolution processes efficiently.

[0270] On the other hand, the signal processing apparatus 170b includes a neural network processor 333 for learning processing. During the first super-resolution processing, the neural network processor 333 can perform learning based on a first learning database; during the second super-resolution processing, it can perform learning based on a second learning database different from the first learning database; and during the third super-resolution processing, it can perform learning based on a third learning database different from the second learning database. Thus, it is possible to efficiently improve the sharpness of image edges, textures, and flat areas while efficiently performing multiple super-resolution processes.

[0271] On the other hand, the signal processing apparatus 170b of another embodiment of the present invention can perform a first super-resolution processing, a second super-resolution processing, and a third super-resolution processing in parallel based on an input image, and output the synthesized image as an output image IMm based on the result data IMb of the first super-resolution processing, the result data IMc of the second super-resolution processing, and the result data IMd of the third super-resolution processing. For this purpose, see [reference needed]. Figure 12 Describe it.

[0272] Figure 12 This is an example of an internal block diagram of a signal processing apparatus according to another embodiment of the present invention.

[0273] Referring to the accompanying drawings, a signal processing apparatus 170b according to another embodiment of the present invention includes a first super-resolution processing unit 610 that performs a first super-resolution processing based on an input image IMa, a second super-resolution processing unit 620 that performs a second super-resolution processing in parallel with the first super-resolution processing unit 610 based on the input image IMa, and a third super-resolution processing unit 625 that performs a third super-resolution processing in parallel with the first super-resolution processing unit 610 and the second super-resolution processing unit 620 based on the input image IMa.

[0274] For example, in super-resolution mode, the signal processing device 170b can control the first super-resolution processing unit 610, the second super-resolution processing unit 620, and the third super-resolution processing unit 625 to operate in parallel.

[0275] On the other hand, during the first super-resolution processing, the first super-resolution processing unit 610 can perform learning based on the first learning database.

[0276] For example, during the first super-resolution processing, the first super-resolution processing unit 610 may utilize the neural network processor 333 to perform learning based on the first learning database.

[0277] Specifically, during the first super-resolution processing, the first super-resolution processing unit 610 within the signal processing apparatus 170b can perform learning based on the first resolution image and the second resolution image.

[0278] On the other hand, during the second super-resolution processing, the second super-resolution processing unit 620 can perform learning based on the second learning database.

[0279] For example, during the second super-resolution processing, the second super-resolution processing unit 620 can utilize the neural network processor 333 to perform learning based on a second learning database that is different from the first learning database.

[0280] On the other hand, the second super-resolution processing unit 620 within the signal processing device 170b can perform learning based on the second super-resolution processing, based on a third resolution image with a resolution lower than that of the first resolution image, and the processed image.

[0281] For example, during the third super-resolution processing, the third super-resolution processing unit 625 can utilize the neural network processor 333 to perform learning based on a third learning database that is different from the second learning database.

[0282] On the other hand, the first super-resolution processing unit 610, the second super-resolution processing unit 620, and the third super-resolution processing unit 625 can each perform learning or adjust learning independently.

[0283] On the other hand, the first super-resolution processing unit 610 can mainly process the edges in the input image, the second super-resolution processing unit 620 can process the regions in the input image other than the edges, such as texture, and the third super-resolution processing unit 620 can process the flat regions in the input image.

[0284] Therefore, it is possible to efficiently perform multiple super-resolution processes while effectively improving the sharpness of image edges, textures, and flat areas.

[0285] On the other hand, the signal processing device 170b can extract a first feature map MPa based on the result data IMb of the first super-resolution processing, extract a second feature map MPb based on the result data IMc of the second super-resolution processing, and extract a third feature map MPc based on the result data IMd of the third super-resolution processing.

[0286] On the other hand, the signal processing device 170b can calculate the synthesis ratio RTa of the result data IMb of the first super-resolution processing, the result data IMc of the second super-resolution processing, and the result data IMd of the third super-resolution processing based on the first feature map MPa, the second feature map MPb, and the third feature map MPc, and output the synthesized image as the output image IMm based on the calculated synthesis ratio RTb.

[0287] For this purpose, the signal processing apparatus 170b may further include a first feature map extraction unit 612 for extracting a first feature map MPa based on the result data IMb of the first super-resolution processing, a second feature map extraction unit 622 for extracting a second feature map MPb based on the result data IMc of the second super-resolution processing, and a third feature map extraction unit 627 for extracting a third feature map MPc based on the result data IMd of the third super-resolution processing.

[0288] On the other hand, the first feature map extraction unit 612, the second feature map extraction unit 622, and the third feature map extraction unit 627 can extract the degree of improvement in resolution and sharpness as feature values ​​or feature maps MPa, MPb, and MPc, respectively, based on the result data IMb, IMc, and IMd of the super-resolution processing, in pixel units.

[0289] For example, the feature values ​​or feature maps MPa, MPb, and MPc output by the first feature map extraction unit 612, the second feature map extraction unit 622, and the third feature map extraction unit 627 may include the levels or energies of the low, medium, and high frequency components of the spatial frequency of the result data IMb, IMc, and IMd of each super-resolution processing.

[0290] On the other hand, the first feature map extraction unit 612, the second feature map extraction unit 622, and the third feature map extraction unit 627 can also use different feature values ​​or feature maps as feature values ​​or feature maps MPa, MPb, and MPc to indicate the degree of improvement in resolution and sharpness.

[0291] On the other hand, the signal processing apparatus 170b may further include: a feature map analysis unit 630, which calculates the synthesis ratio RTb of the result data IMb of the first super-resolution processing, the result data IMc of the second super-resolution processing, and the result data IMd of the third super-resolution processing based on the first feature map MPa, the second feature map MPb, and the third feature map MPc; and an image synthesis unit 640, which outputs the synthesized image as an output image IMm based on the synthesis ratio RTb calculated in the feature map analysis unit 630.

[0292] On the other hand, the image synthesis unit 640 can determine the synthesis ratio RTb of the result data IMb of the first super-resolution processing, the result data IMc of the second super-resolution processing, and the result data IMd of the third super-resolution processing, based on each feature value or feature map MPa, MPb, and MPc, in pixel units.

[0293] For example, the image synthesis unit 640 can determine the optimal synthesis ratio RTb for improving resolution and sharpness based on various feature values ​​or feature maps MPa, MPb, and MPc.

[0294] As another example, the image compositing unit 640 can change the compositing ratio RTb according to the individual image components for which resolution and sharpness are to be enhanced.

[0295] On the other hand, the feature map analysis unit 630 or the image synthesis unit 640 within the signal processing device 170b can change the synthesis ratio RTb in pixel units.

[0296] For example, the feature map analysis unit 630 or the image synthesis unit 640 in the signal processing apparatus 170b can be controlled to increase the ratio of the result data IMb of the first super-resolution processing near the edge region in the input image IMa.

[0297] As another example, the feature map analysis unit 630 or the image synthesis unit 640 in the signal processing device 170b can be controlled to increase the ratio of the result data IMc of the second super-resolution processing in the vicinity of the texture region in the input image IMa.

[0298] As another example, the feature map analysis unit 630 or the image synthesis unit 640 in the signal processing device 170b can be controlled to increase the ratio of the result data IMd of the third super-resolution processing in the vicinity of a flat region in the input image IMa.

[0299] Therefore, it is possible to efficiently perform multiple super-resolution processes while effectively improving the sharpness of image edges, textures, and flat areas.

[0300] On the other hand, the feature map analysis unit 630 or the image synthesis unit 640 within the signal processing device 170b can change the synthesis ratio RTb based on the type or resolution of the content.

[0301] For example, the feature map analysis unit 630 or the image synthesis unit 640 can be controlled to increase the ratio of the result data IMc of the second super-resolution processing in the result data IMb of the first super-resolution processing, the result data IMc of the second super-resolution processing, and the result data IMd of the third super-resolution processing as the resolution increases.

[0302] Therefore, it can efficiently improve the clarity of image edges, textures, and flat areas, corresponding to the image resolution.

[0303] As another example, the feature map analysis unit 630 or the image synthesis unit 640 can be controlled to increase the ratio of the first super-resolution processing result data IMb in the first super-resolution processing result data IMb, the second super-resolution processing result data IMc, and the third super-resolution processing result data IMd as the content moves or the movement vector increases.

[0304] Therefore, it can efficiently improve the clarity of image edges, textures, and flat areas in response to content movement.

[0305] Thus, according to Figure 12 The first super-resolution processing unit 610, the second super-resolution processing unit 620, and the third super-resolution processing unit 625 within the signal processing device 170b perform super-resolution processing in parallel, thereby enabling them to learn or adjust in a way that matches their respective purposes.

[0306] In addition, according to Figure 12 The first super-resolution processing unit 610, the second super-resolution processing unit 620, and the third super-resolution processing unit 625 in the signal processing device 170b are independent of each other and can be implemented relatively uncomplicatedly.

[0307] on the other hand, Figure 12 The feature map analysis unit 630 within the signal processing device 170b can finely adjust the feature map analysis to prevent a decrease in the resolution or sharpness of the result data IMb, IMc, and IMd of each super-resolution processing.

[0308] On the other hand, when the input image is a dynamic image, the signal processing device 170b can perform a first super-resolution processing, a second super-resolution processing, and a third super-resolution processing in parallel based on the input dynamic image, and output the synthesized dynamic image as an output dynamic image based on the result data IMb of the first super-resolution processing, the result data IMc of the second super-resolution processing, and the result data IMd of the third super-resolution processing.

[0309] Therefore, it is possible to efficiently perform multiple super-resolution processes while effectively improving the clarity of edges, textures, and flat areas of dynamic images.

[0310] The preferred embodiments of the present invention have been described above. However, the present invention is not limited to the specific embodiments described above. It will be apparent to those skilled in the art that various modifications can be made without departing from the spirit of the invention as claimed in the claims. Such modifications should not be understood separately from the technical concept or prospect of the present invention.

Claims

1. An image display device, wherein, include: monitor; as well as A signal processing device that processes the input image and outputs the output image. The signal processing device performs a first super-resolution process and a second super-resolution process in parallel based on the input image, and outputs the synthesized image as the output image based on the result data of the first super-resolution process and the result data of the second super-resolution process.

2. The image display device according to claim 1, wherein, The signal processing device increases the sharpness or resolution of the edges and textures of objects within the input image based on the first super-resolution processing and the second super-resolution processing performed in parallel.

3. The image display device according to claim 1, wherein, During the first super-resolution processing, the signal processing device performs learning based on the first learning database. During the second super-resolution processing, the signal processing device performs learning based on a second learning database that is different from the first learning database.

4. The image display device according to claim 3, wherein, The signal processing device includes a neural network processor for learning processing; The neural network processor performs learning based on the first learning database during the first super-resolution processing; The neural network processor performs learning based on the second super-resolution processing and on a second learning database that is different from the first learning database.

5. The image display device according to claim 1, wherein, During the first super-resolution processing, the signal processing device performs learning based on the first resolution image and the second resolution image. The signal processing device performs learning based on the second super-resolution processing, based on a third resolution image with a resolution lower than that of the first resolution image, and the processed image.

6. The image display device according to claim 1, wherein, The signal processing device extracts a first feature map based on the result data of the first super-resolution processing and extracts a second feature map based on the result data of the second super-resolution processing. The signal processing device calculates the synthesis ratio of the result data of the first super-resolution processing and the result data of the second super-resolution processing based on the first feature map and the second feature map. The signal processing device outputs the synthesized image as the output image based on the calculated synthesis ratio.

7. The image display device according to claim 6, wherein, The signal processing device changes the synthesis ratio in pixels.

8. The image display device according to claim 6, wherein, The signal processing device changes the synthesis ratio based on the type or resolution of the content.

9. The image display device according to claim 1, wherein, The signal processing device includes: A first super-resolution processing unit performs first super-resolution processing based on the input image; and The second super-resolution processing unit performs the second super-resolution processing in parallel with the first super-resolution processing unit based on the input image.

10. The image display device according to claim 9, wherein, The signal processing device further includes: The first feature map extraction unit extracts a first feature map based on the result data of the first super-resolution processing; The second feature map extraction unit extracts a second feature map based on the result data of the second super-resolution processing; The feature map analysis unit calculates the synthesis ratio of the result data from the first super-resolution processing and the result data from the second super-resolution processing based on the first feature map and the second feature map; and The image synthesis unit outputs the synthesized image as the output image based on the synthesis ratio calculated in the feature map analysis unit.

11. The image display device according to claim 1, wherein, The signal processing device performs the first super-resolution processing and the second super-resolution processing in parallel based on the input image according to the super-resolution mode, and outputs the synthesized image as the output image based on the result data of the first super-resolution processing and the result data of the second super-resolution processing.

12. The image display device according to claim 1, wherein, When the signal processing device is set to sports mode, it performs the first super-resolution processing and the second super-resolution processing in parallel based on the input image, and outputs the synthesized image as the output image based on the result data of the first super-resolution processing and the result data of the second super-resolution processing.

13. The image display device according to claim 1, wherein, When the input image is a dynamic image, the signal processing device performs the first super-resolution processing and the second super-resolution processing in parallel based on the input dynamic image, and outputs the synthesized dynamic image as the output dynamic image based on the result data of the first super-resolution processing and the result data of the second super-resolution processing.

14. The image display device according to claim 1, wherein, The signal processing device, based on the input image, also performs a third super-resolution processing in parallel, and outputs the synthesized image as the output image based on the result data of the first super-resolution processing, the result data of the second super-resolution processing, and the result data of the first super-resolution processing.

15. The image display device according to claim 14, wherein, The signal processing device increases the sharpness or resolution of the edges and textures of objects within the input image based on the first super-resolution processing and the second super-resolution processing executed in parallel. The signal processing device increases the sharpness or resolution of flat regions within the object in the input image based on the parallel execution of the third super-resolution processing.

16. The image display device according to claim 13, wherein, The signal processing device extracts a first feature map based on the result data of the first super-resolution processing, extracts a second feature map based on the result data of the second super-resolution processing, and extracts a third feature map based on the result data of the third super-resolution processing. The signal processing device calculates the synthesis ratio of the result data of the first super-resolution processing, the result data of the second super-resolution processing, and the result data of the third super-resolution processing based on the first feature map, the second feature map, and the third feature map. The signal processing device outputs the synthesized image as the output image based on the calculated synthesis ratio.

17. The image display device according to claim 16, wherein, The signal processing device changes the synthesis ratio in pixels.

18. The image display device according to claim 13, wherein, The signal processing device includes: The first super-resolution processing unit performs first super-resolution processing based on the input image; The second super-resolution processing unit, based on the input image, performs the second super-resolution processing in parallel with the first super-resolution processing unit; and The third super-resolution processing unit performs the third super-resolution processing in parallel with the first super-resolution processing unit and the first super-resolution processing unit based on the input image.

19. The image display device according to claim 18, wherein, The signal processing device further includes: The first feature map extraction unit extracts a first feature map based on the result data of the first super-resolution processing; The second feature map extraction unit extracts a second feature map based on the result data of the second super-resolution processing; The third feature map extraction unit extracts the third feature map based on the result data of the third super-resolution processing; The feature map analysis unit calculates the synthesis ratio of the result data of the first super-resolution processing, the result data of the second super-resolution processing, and the result data of the third super-resolution processing based on the first feature map, the second feature map, and the third feature map; and The image synthesis unit outputs the synthesized image as the output image based on the synthesis ratio calculated in the feature map analysis unit.

20. An image display device, wherein, include: monitor; as well as A signal processing device processes the input image and outputs the output image. The signal processing device performs a first super-resolution process, a second super-resolution process, and a third super-resolution process in parallel based on the input image, and outputs the synthesized image as the output image based on the result data of the first super-resolution process, the result data of the second super-resolution process, and the result data of the third super-resolution process.

Images