Display device and operating method therefor

Abstract

A display device and an operating method thereof are disclosed. The operating method for the display device according to at least one of various embodiments of the present disclosure may include the following steps: receiving a video playback request signal; decoding video data; calculating an offset with respect to a video delay by receiving information about the decoded video data; sending the calculated offset with respect to the video delay to a graphics thread, and controlling a screen to be output by synchronizing video and graphics.

Description

Technical area

[0001] The present disclosure relates to a display device and an operating method thereof. background

[0002] Recently, the functions of terminal devices have diversified, such as data and voice communication, capturing images and videos using a camera, recording voice, playing music files through a speaker system, and outputting images or videos to a display.

[0003] Some devices have an additional electronic game function or perform a multimedia player function.

[0004] As the functions of such terminals have diversified, they are implemented in the form of multimedia players with complex functions such as taking pictures or videos, playing music or video files, playing games and receiving broadcasts.

[0005] Accordingly, the processing of multimedia data in which video and graphics are processed simultaneously on one terminal is increasing, but the processing time of high-demand videos is gradually increasing compared to that of graphics, so synchronization problems often occur, which causes inconvenience in video display from the user's perspective. Detailed description of the inventionTechnical problem

[0006] The present disclosure may provide a synchronization method according to the occurrence of a physical delay between video processing hardware and graphics processing hardware.

[0007] The present disclosure may provide a display device and an operating method thereof that handles the physical delay between the above-mentioned hardware configurations through software. Technical solution

[0008] The operating method of a display device according to at least one embodiment of various embodiments of the present disclosure may include: receiving a video playback request signal; decoding video data; calculating an offset with respect to a video delay by receiving information of the decoded video data; sending the calculated offset with respect to the video delay to a graphics thread; and controlling an output of a screen by synchronizing video and graphics.

[0009] According to the method of operation of the display device according to at least one embodiment of the various embodiments of the present disclosure, the method may further comprise storing the calculated offset with respect to the video delay.

[0010] According to the operating method of the display device according to at least one embodiment of the various embodiments of the present disclosure, the method may further comprise receiving an event signal and checking a property of the received event signal.

[0011] According to the operating method of the display device according to at least one embodiment of the various embodiments of the present disclosure, the method may further comprise, when the checked property of the received event signal is a first property, reading the stored offset and sending the offset to the graphics thread; and controlling an output of a screen according to the event signal by synchronizing video and graphics.

[0012] According to the operating method of the display device according to at least one embodiment of the various embodiments of the present disclosure, the first characteristic may represent an event signal that does not require any additional frames for video processing other than the number of frames used to calculate the offset for the calculated video delay.

[0013] According to a method of operating a display device according to at least one embodiment of the various embodiments of the present disclosure, the method may further comprise, when the verified property of the received event signal is a second property, recalculating an offset for the video delay; storing the recalculated offset for the video delay; and sending the recalculated offset to the graphics thread; and controlling an output of a screen according to the event signal by synchronizing video and graphics.

[0014] According to a method of operating a display device according to at least one embodiment of the various embodiments of the present disclosure, the second characteristic may represent an event signal that requires additional frames for video processing besides the number of frames used to calculate the offset for the calculated video delay.

[0015] According to at least one embodiment of the various embodiments of the present disclosure, the display device may comprise: a video decoder configured to decode video data when a video playback request signal is received; a video processor configured to receive information of the decoded video data and calculate an offset with respect to a video delay; a graphics processor configured to process graphics data for the decoded video data; and a controller configured to control the output of a screen on which video and graphics are synchronized by sending the calculated offset with respect to the video delay to a graphics thread of the graphics processor.

[0016] According to at least one embodiment of the various embodiments of the present disclosure, a display device may further comprise a memory configured to store the calculated offset with respect to the video delay.

[0017] According to a display device according to at least one embodiment of the various embodiments of the present disclosure, when an event signal is received, the controller may be configured to check a property of the received event signal.

[0018] According to a display device according to at least one embodiment of the various embodiments of the present disclosure, when the checked property of the received event signal is a first property, the controller may be configured to read the stored offset and send the offset to the graphics thread of the graphics processor and output a screen corresponding to the event signal by synchronizing video and graphics.

[0019] According to a display device according to at least one embodiment of the various embodiments of the present disclosure, the first characteristic may represent an event signal that does not require any additional frames for video processing other than the number of frames used to calculate the offset for the calculated video delay.

[0020] According to a display device according to at least one embodiment of the various embodiments of the present disclosure, when the checked property of the received event signal is a second property, the controller may be configured to control the video processor to recalculate an offset for the video delay, control the video processor to store the recalculated offset for the video delay in memory, send the recalculated offset to the graphics thread of the graphics processor, and output a screen according to the event signal by synchronizing video and graphics.

[0021] According to a display device according to at least one embodiment of the various embodiments of the present disclosure, the second characteristic may represent an event signal that requires additional frames for video processing besides the number of frames used to calculate the offset for the calculated video delay. Effect of the invention

[0022] According to at least one embodiment of the present disclosure, the screen can be output by synchronizing video and graphics despite the physical delay occurring between the hardware configurations, thereby increasing the user's satisfaction with the display device.

[0023] According to at least one of the various embodiments of the present disclosure, synchronization between video and graphics may be performed in a software manner without the need to add a separate hardware device, thereby avoiding additional costs and increasing system efficiency. Brief description of the drawings Fig. 1 is a block diagram illustrating a configuration of a display device according to an embodiment of the present disclosure. Fig. 2 is a block diagram of a remote control device according to an embodiment of the present disclosure. Fig. 3 shows an actual configuration example of a remote control device according to an embodiment of the present disclosure. Fig. 4 shows an example of using a remote control device according to an embodiment of the present disclosure. Fig. 5 is a diagram for explaining a horizontal mode and a vertical mode of a stand display device according to an embodiment of the present disclosure. Fig. 6 is a block diagram of a configuration of a display device according to another embodiment of the present disclosure. Fig. 7 and Fig. 8 are flowcharts illustrating an operating method of a display device according to an embodiment of the present disclosure. Fig. 9 to 11 are diagrams illustrating a method of synchronizing a video region and a graphics region in a video processing process of a display device according to an embodiment of the present disclosure. Fig. 12 and Fig. 13 are diagrams illustrating a process of video processing of a display device according to an embodiment of the present disclosure. Best embodiment

[0024] Embodiments of the present invention are explained in more detail below with reference to the drawings. The suffixes "module" and "part" used in the following description for components are used merely to simplify the description and have no meaning or function of their own.

[0025] The display device according to the embodiment of the present invention is, for example, a smart display device that adds a computer support function to the broadcast reception function. It can have a more user-friendly interface such as a manual input device, a touch screen, or a room remote control by adding an Internet function, etc., while retaining the broadcast reception function. It can also perform functions such as email, web browsing, banking, or gaming by connecting to the Internet and a computer supporting a wired or wireless Internet function. A standardized general-purpose operating system can be used for these various functions.

[0026] The display device described in the present invention can therefore perform various user-friendly functions, for example, because various applications can be freely added to or deleted from the universal operating system kernel. In particular, the display device can be a network TV, an HBB TV, a smart TV, an LED TV, an OLED TV, etc., and in some cases can also be used on a smartphone.

[0027] Fig. 1 is a block diagram illustrating the configuration of a display device 100 according to an embodiment of the present invention.

[0028] With reference to Fig. 1, the display device 100 may include a broadcast receiving unit 130, an external device interface 135, a memory 140, a user input interface 150, a controller 170, a wireless communication interface 173, a display 180, a speaker 185, and a power supply circuit 190.

[0029] The broadcast receiving unit 130 may include a tuner 131, a demodulator 132 and a network interface 133.

[0030] The tuner 131 can select a specific broadcast channel according to a channel selection command. The tuner 131 can receive a broadcast signal for the selected specific broadcast channel.

[0031] The demodulator 132 can separate the received broadcast signal into a video signal, an audio signal, and a data signal related to a broadcast program, and restore the separated video signal, audio signal, and data signal into an outputtable form.

[0032] The external device interface 135 can receive an application or application list in a neighboring external device and send it to the controller 170 or the memory 140.

[0033] The external device interface 135 can provide a connection between the display device 100 and the external device. The external device interface 135 can receive image and audio signals from an external device connected wirelessly or wired to the display device 100 and send them to the controller 170. The external device interface 135 can include a plurality of external input ports. The plurality of external input ports can include an RGB port, one or more High Definition Multimedia Interface (HDMI) ports, and a component port.

[0034] An image signal from an external device input via an external device interface 135 can be output via a display 180. An audio signal from an external device input via the external device interface 135 can be output via a speaker 185.

[0035] An external device that can be connected to the external device interface 135 can be a set-top box (STB), a Blu-ray player, a DVD player, a game console, a soundbar, a smartphone, a PC, a USB storage device, or a home theater, but this is only an example.

[0036] The network interface 133 may provide an interface for connecting the display device 100 to a wired / wireless network, including the Internet. The network interface 133 may send or receive data to or from other users or other electronic devices via the connected network or another network linked to the connected network.

[0037] In addition, some of the content data stored on the display device 100 may be sent to other users or other electronic devices pre-registered on the display device 100 to selected users or selected electronic devices.

[0038] The network interface 133 can access a given website via the connected network or another network connected to the connected network. That is, it can access a given website via the network and send or receive data to or from the corresponding server.

[0039] Furthermore, the network interface 133 can receive content or data provided by a content provider or a network operator. That is, the network interface 133 can receive content such as movies, advertisements, games, VOD, broadcast signals, etc., as well as related information provided by a content provider or a network operator over the network.

[0040] In addition, the network interface 133 can receive update information and update files of the firmware provided by the network operator and send data to the Internet or to a content provider or network operator.

[0041] The network interface 133 can select and receive a desired application from the applications publicly accessible over the network.

[0042] The memory 140 can store programs for each signal processing and control within the controller 170 and can store processed images, voices, or data signals.

[0043] In addition, the memory 140 can perform a function of temporarily storing images, voices, or data signals input from an external device interface 135 or a network interface 133, and can store information about a specific image through a channel memory function.

[0044] The memory 140 may store an application or an application list input from an external device interface 135 or a network interface 133.

[0045] The storage 140 can play back content files (video files, still image files, music files, document files, application files, etc.) stored in the storage 140 and provide them to the user.

[0046] The user input interface 150 can send signals input by the user to the controller 170 or send a signal from the controller 170 to the user. For example, the user input interface 150 can receive and process control signals such as power on / off, channel selection, and screen settings from the remote control device 200 via various communication methods such as Bluetooth, ultra-wideband (UWB), ZigBee, radio frequency (RF), or infrared (IR), or process control signals from the controller 170 for transmission to the remote control device 200.

[0047] In addition, the user input interface 150 may send a control signal input from a local key (not shown) such as a power key, a channel key, a volume key, a setting value, etc., to the controller 170.

[0048] An image signal processed by the controller 170 can be input to the display 180 and displayed as an image corresponding to the image signal. Furthermore, an image signal processed by the controller 170 can be input to an external output device via an external device interface 135.

[0049] An audio signal processed by the controller 170 may be output as audio to a speaker 185. Furthermore, an audio signal processed by the controller 170 may be input to an external output device via an external device interface 135.

[0050] In addition, the controller 170 can control the overall operation within the display device 100.

[0051] Furthermore, the controller 170 can control the display device 100 by a user command or internal program input via the user input interface 150 and connect to a network to allow the user to download a desired application or application list to the display device 100.

[0052] The controller 170 enables the output of the user-selected channel information, etc., via the display 180 or the speaker 185 along with the processed image or audio signal.

[0053] In addition, the controller 170 enables the output of an image signal or an audio signal from an external device, such as a camera or a camcorder, via the display 180 or the speaker 185 according to an image playback command of the external device received via the user input interface 150 via the external device interface 135.

[0054] Meanwhile, the controller 170 may control the display 180 to display an image, and may, for example, control a broadcast image input via the tuner 131, an image input via the external device interface 135, an image input via the network interface, or an image stored in the memory 140 to be displayed on the display 180. In this case, the image displayed on the display 180 may be a still image or a moving image, and may be a 2D image or a 3D image.

[0055] Furthermore, the controller 170 can control the playback of content stored on the display device 100, or of received broadcast content, or of externally input content. The content can be in various forms, such as broadcast images, externally input images, audio files, still images, connected web screens, document files, etc.

[0056] The wireless communication interface 173 can perform wired or wireless communication with an external device. The wireless communication interface 173 can perform short-range communication with an external device. To this end, the wireless communication interface 173 supports short-range communication using at least one of the following technologies: Bluetooth, radio frequency identification (RFID), Infrared Data Association (IrDA), UWB, ZigBee, near-field communication (NFC), wireless fidelity (Wi-Fi), Wi-Fi Direct, and wireless universal serial bus (wireless USB).The wireless communication interface 173 may support wireless communication between the display device 100 and a wireless communication system, between the display device 100 and another display device 100, or between the display device 100 and a network in which the display device (100 or an external server) is located, via a wireless network. The wireless network may be a wireless personal area network.

[0057] Here, the other display device 100 can be a wearable device (e.g., a smartwatch, data glasses, or a head-mounted display (HMD)) that can exchange data with the display device 100 according to the present invention, or a mobile terminal such as a smartphone. The wireless communication interface 173 can detect wearable devices that can communicate in the vicinity of the display device 100.

[0058] If the detected portable device is a device certified for communication with the display device 100 according to the present invention, the controller 170 may further send at least a portion of the data processed in the display device 100 to the portable device via the wireless communication interface 173. The user of the portable device can thus use the data processed in the display device 100 via the portable device.

[0059] The display 180 can generate a drive signal by converting the image signal, data signal, on-screen display (OSD) signal processed in the controller 170 or the image signal, data signal, etc. received from the external device interface 135 into R, G, or B signals, respectively.

[0060] However, the Fig. The display device 100 illustrated in Figure 1 is merely one embodiment of the present invention. Some of the illustrated components may be integrated, added, or omitted depending on the specifications of the actually implemented display device 100.

[0061] That is, two or more components may be combined into a single component, as needed, or a component may be divided into two or more components. The functions performed by each block are intended to illustrate one embodiment of the present invention, and the specific acts or devices do not limit the scope of the present invention.

[0062] According to another embodiment of the present invention, the display device 100 can receive and display an image via a network interface 133 or an external device interface 135 without having a tuner 131 and a demodulator 132, unlike in Fig. 1 shown.

[0063] For example, the display device 100 may be implemented by separating an image processing device, such as a set-top box for receiving broadcast signals or contents corresponding to various network services, and a content playback device for playing contents input from the image processing device.

[0064] In this case, the operation method of the display device according to the embodiment of the present invention, which will be described below, can be implemented not only by the method described with reference to Fig. 1, but also by any of the image processing devices, such as the separate set-top box or the content playback device with the display 180 and the audio output unit 185.

[0065] Next, a remote control device according to an embodiment of the present invention will be described with reference to Fig. 2 and Fig. 3 described.

[0066] Fig. 2 is a block diagram of a remote control device according to an embodiment of the present invention, and Fig. 3 shows an actual configuration example of a remote control device 200 according to an embodiment of the present invention.

[0067] With reference to Fig. 2, the remote control device 200 may include a fingerprint recognition device 210, a wireless communication circuit 220, a user input interface 230, a sensor 240, an output interface 250, a power supply circuit 260, a memory 270, a controller 280, and a microphone 290.

[0068] With reference to Fig. 2, the wireless communication circuit 220 sends and receives signals to and from each of the display devices 100 according to the above-described embodiments of the present invention.

[0069] The remote control device 200 may be equipped with an RF circuit 221 that can send and receive signals to and from the display device 100 according to an RF communication standard, and with an IR circuit 223 that can send and receive signals to and from the display device 100 according to an IR communication standard. Furthermore, the remote control device 200 may be equipped with a Bluetooth circuit 225 that can send and receive signals to and from the display device 100 according to the Bluetooth communication standard.Furthermore, the remote control device 200 may be equipped with an NFC circuit 227 that can send and receive signals to and from the display device 100 according to the NFC (Near Field Communication) communication standard, and a WLAN circuit 229 that can send and receive signals to and from the display device 100 according to the WLAN (Wireless LAN) communication standard.

[0070] Furthermore, the remote control device 200 sends a signal containing information about the movement of the remote control device 200 to the display device 100 via the wireless communication circuit 220.

[0071] Meanwhile, the remote control device 200 can receive a signal transmitted from the display device 100 via the RF circuit 221 and can send commands for powering on / off, changing channels, changing volume, etc. to the display device 100 via the IR circuit 223 when needed.

[0072] The user input interface 230 may consist of a keyboard, a button, a touchpad, or a touchscreen. The user can enter a command related to the display device 100 into the remote control device 200 via the user input interface 230. If the user input interface 230 is equipped with a button, the user can enter commands for the display device 100 into the remote control device 200 via the button. This is demonstrated by Fig. 3 explained.

[0073] With reference to Fig. 3, the remote control device 200 may include a plurality of buttons. The plurality of buttons may include a fingerprint recognition button 212, a power button 231, a home screen button 232, a live button 233, an external input button 234, a volume control button 235, a voice recognition button 236, a channel change button 237, a confirmation button 238, and a back button 239.

[0074] The fingerprint recognition button 212 may be a button for recognizing a user's fingerprint. In one embodiment, the fingerprint recognition button 212 may perform a press actuation and receive both a press actuation and a fingerprint recognition.

[0075] The power switch 231 may be a button for turning the display device 100 on / off.

[0076] The home screen button 232 may be a button for switching to the home screen of the display device 100.

[0077] The Live button 233 may be a button for displaying a live broadcast program.

[0078] The external input button 234 may be a button for receiving an external input connected to the display device 100.

[0079] The volume control button 235 may be a button for adjusting the magnitude of the volume output by the display device 100.

[0080] The voice recognition button 236 may be a button for receiving a user's voice and recognizing the received voice.

[0081] The channel change button 237 may be a button for receiving a broadcast signal of a specific broadcast channel.

[0082] The confirmation key 238 may be a key for selecting a specific function and the back key 239 may be a key for returning to the previous screen.

[0083] Fig. 2 is described again.

[0084] If the user input interface 23 has a touchscreen, the user can input commands for the display device 100 into the remote control device 200 by touching the softkey on the touchscreen. The user input interface 23 may also have various input options, such as a scroll button or a rotary knob button. This embodiment does not limit the scope of the present disclosure.

[0085] The sensor 240 may include a gyro sensor 241 or an acceleration sensor 243, and the gyro sensor 241 may detect information about the movement of the remote control device 200.

[0086] For example, the gyro sensor 241 can detect information about the operation of the remote control device 200 based on the x-, y-, and z-axes, and the acceleration sensor 243 can detect information about the movement speed of the remote control device 200. Meanwhile, the remote control device 200 can be further equipped with a distance measuring sensor to detect the distance to the display 180 of the display device 100.

[0087] The output interface 250 can output a video or audio signal corresponding to the operation of the user input interface 23 or the signal transmitted from the display device 100.

[0088] The user can recognize whether the user input interface 23 is being operated or whether the display device 100 is being controlled via the output interface 250.

[0089] For example, the output interface 250 may be equipped with an LED 251 that lights up when the user input interface 23 is operated or a signal is sent and received to the display device 100 via the wireless communication unit 225, a vibrator 253 that generates vibrations, a speaker 255 that outputs sound, or a display 257 that outputs an image.

[0090] In addition, the power supply circuit 260 supplies power to the remote control device 200 and can reduce energy waste by cutting off the power supply when the remote control device 200 does not move for a predetermined period of time.

[0091] The power supply circuit 260 can resume power supply when a predetermined button provided in the remote control device 200 is pressed.

[0092] The memory 270 can store various types of programs, application data, etc. required for the control or operation of the remote control device 200.

[0093] When the remote control device 200 wirelessly transmits and receives signals via the display device 100 and the RF circuit 221, the remote control device 200 and the display device 100 transmit and receive signals over a predetermined frequency band.

[0094] The controller 280 of the remote control device may store and reference in the memory 270 information about the frequency band over which signals may be wirelessly transmitted and received with the display device 100 coupled to the remote control device 200.

[0095] The controller 280 controls all matters related to the control of the remote control device 200. The controller 280 may send a signal corresponding to a predetermined key operation of the user input interface 230 or a signal corresponding to the movement of the remote control device 200 detected by the sensor 240 to the display device 100 via the wireless communication unit 225.

[0096] In addition, the microphone 290 of the remote control device 200 can detect speech.

[0097] A plurality of microphones 290 may be provided.

[0098] Next, Fig. 4 described.

[0099] Fig. 4 shows an example of using a remote control device 200 according to an embodiment of the present invention.

[0100] Fig. 4(a) shows that a pointer 205 corresponding to a remote control device 200 is displayed on a display 180.

[0101] A user can move the remote control device 200 up, down, left, and right, or rotate it. The pointer 205 displayed on the display 180 of the display device 100 corresponds to the movement of the remote control device 200. As shown in the drawing, this remote control device 200 can be called a "space remote control" because the pointer 205 moves and is displayed according to the movement in three-dimensional space.

[0102] Fig. 4(b) illustrates that when a user moves the remote control device 200 to the left, the pointer 205 displayed on the display 180 of the display device 100 also moves to the left in response.

[0103] Information about the movement of the remote control device 200, detected by the sensor of the remote control device 200, is sent to the display device 100. The display device 100 can calculate the coordinates of the pointer 205 from the information about the movement of the remote control device 200. The display device 100 can display the pointer 205 in response to the calculated coordinates.

[0104] Fig. 4(c) exemplifies a case where a user moves the remote control device 200 away from the display 180 while pressing a specific button on the remote control device 200. This allows the selection area in the display 180 corresponding to the pointer 205 to be zoomed in and displayed enlarged.

[0105] Conversely, when the user moves the remote control device 200 closer to the display 180, the selection area in the display 180 corresponding to the pointer 205 can be zoomed out and displayed in a reduced size.

[0106] When the remote control device 200 moves away from the display 180, the selection area can be zoomed out, and when the remote control device 200 approaches the display 180, the selection area can be zoomed in.

[0107] Furthermore, if a specific button on the remote control device 200 is pressed, detection of up, down, left, and right movements can be excluded. That is, if the remote control device 200 moves away from or toward the display 180, up, down, left, and right movements are not detected, only forward and backward movements. If no specific button on the remote control device 200 is pressed, only the pointer 205 moves according to the up, down, left, and right movements of the remote control device 200.

[0108] The movement speed or direction of the pointer 205 may correspond to the movement speed or direction of the remote control device 200.

[0109] However, the pointer in this description refers to an object displayed on the display 180 in response to an operation of the remote control device 200. The pointer 205 may include objects of various shapes, such as a dot, a cursor, a prompt, a bold outline, etc., in addition to the arrow shape shown in the drawing. Furthermore, the pointer 205 may be displayed to correspond to a point on the horizontal or vertical axis of the display 180, and may also be displayed to correspond to multiple points, such as a line or area.

[0110] Fig. 5(a) and Fig. 5(b) are drawings for explaining the horizontal mode and the vertical mode of a standing display device according to an embodiment of the present disclosure.

[0111] In the Fig. 5(a) and Fig. 5(b) shows a stand-type display device 100.

[0112] A shaft 103 and a stand 105 may be connected to the display device 100.

[0113] The shaft 103 can connect the display device 100 and the stand 105. The shaft 103 can extend vertically.

[0114] The lower end of the shaft 103 may be connected to an edge of the base 105.

[0115] The lower end of the shaft 103 can be rotatably connected to the circumference of the base 105.

[0116] The display device 100 and the shaft 103 can rotate about a vertical axis with respect to the base 105.

[0117] The upper part of the shaft 103 can be connected to the back of the display device 100.

[0118] The stand base 105 can be used to hold the display device 100.

[0119] The display device 100 may be configured to include the shaft 103 and the base 105.

[0120] The display device 100 can rotate about a point where the upper part of the shaft 103 and the back of the display 180 meet.

[0121] Fig. 5(a) may show that the display 180 operates in a horizontal mode in which the horizontal length is greater than the vertical length, and Fig. Figure 5(b) may show that the display 180 operates in a vertical mode in which the vertical length is greater than the horizontal length.

[0122] A user can hold and move a stand-up display. Unlike fixed devices, the 100-style stand-up display offers enhanced mobility, so the user is not limited by the location.

[0123] Embodiments of a video processing method in a display device 100 according to the present disclosure will be described below.

[0124] For ease of explanation, an application execution screen in which both a video area and a graphics area are present is provided as an example of an application execution request. However, the present disclosure is not limited to this and can be applied to processing all types of videos that have both a video area and a graphics area.

[0125] The display device 100 may include a hardware configuration for processing a video region or a hardware configuration for processing a graphics region in a video path.

[0126] However, unlike processing a graphics region, in the hardware configuration for processing a video region, video images must be saved, copied, etc., during the processing of the video region, which causes a physical delay.

[0127] For example, if a request is received to change a video region and a graphics region, a change may be requested at the application level at the same time, but at the hardware level, as described above, due to the physical delay present in the video region, the video may be delayed more than the graphics, causing problems in output, i.e. playback.

[0128] Therefore, due to the physical delay occurring at the hardware level, a compensation process, i.e. synchronization, is required to synchronize the video area and the graphics area.

[0129] Fig. 6 is a block diagram of a display device 100 according to another embodiment of the present disclosure.

[0130] The display device 100 may be configured to include a display 180, a remote control device 200, and a processor 600.

[0131] In the following description of the display device 100, overlaps with the above-mentioned Fig. 1 to 5, and corresponding descriptions are omitted. For example, the following description of the configuration and function of the remote control device 200 (e.g., remote control) refers to at least one of the contents described above in the Fig. 2 to 4 described descriptions so that there are no duplicate descriptions.

[0132] With reference to Fig. 6, the processor 600 may be configured to include a memory 610, a decoder 620, a first processor 630, a second processor 640, and a control unit 650.

[0133] The processor 600 may receive user input via the remote control device 200.

[0134] In addition, the processor 600 may communicate with an application processor (AP) (not shown) to exchange data.

[0135] The memory 610 can store various data, such as data received or processed by the processor 600. The memory 610 can have the same configuration as the memory 610 shown in Fig. 1 or another configuration.

[0136] The memory 610 may store video delay (or offset) information generated by the processor 600.

[0137] The processor 600 or the control unit 650 can perform operations such as processing, control, and storage for all data communication, the configuration and provision of the user interface, and control operations between the remote control device 200 (or other external input devices - not shown). The control unit 650 can, for example, have the same configuration as the one shown in Fig. 1 shown control 170.

[0138] The decoder 610 may decode video data corresponding to a video playback request signal.

[0139] The first processor 630 may receive video data information about video data decoded by the decoder 620. In this sense, the first processor 630 may correspond to a video processor.

[0140] The first processor 630 can calculate and generate a video delay based on the video data information received from the decoder 620. The control unit 650 can control the calculated video delay and store it in the memory 610 via the interface manager.

[0141] The second processor 640 may perform processing in the graphics area. In this sense, the second processor 640 may correspond to a graphics processor.

[0142] The second processor 640 may process and output graphics data to be synchronized with the delayed video data by referring to the video delay calculated according to the control of the control unit 650 (or the surface manager).

[0143] Fig. 7 and Fig. 8 are flowcharts illustrating an operating method of a display device 100 according to an embodiment of the present disclosure.

[0144] Fig. 7 illustrates a synchronization processing method for a video region and a graphics region according to an embodiment of the present disclosure.

[0145] With reference to Fig. 7, a video playback request signal can be received (S101).

[0146] When a request signal for video reproduction is received in step S101, video data for a video area corresponding to the request signal may be decoded (S103).

[0147] Based on the information about the video data decoded in step S103, a video delay (ie, an offset) can be calculated (S105).

[0148] The video delay calculated in step S105 from the video data may be sent to a graphics thread for a graphics region (S107).

[0149] When the video delay information is sent to the graphics thread by step S107, a screen in which the video area and the graphics area are synchronized can be output (S109).

[0150] Fig. 8 describes a synchronization processing method for a video region and a graphics region according to another embodiment of the present disclosure.

[0151] In particular, Fig. 8 may be a processing procedure when an event occurs during the output of a video.

[0152] The display device 100 may receive an event signal (S201).

[0153] The display device 100 may check the characteristics of the event signal received in step S201 (S203).

[0154] If the characteristics of the event signal checked in step S203 are the first characteristics, the display device 100 may read the previously stored video delay (offset) from the memory 610 (S205).

[0155] At this point, the first property may indicate a case where no additional frames are required for video processing, apart from the number of frames used in calculating the offset for the previously calculated video delay.

[0156] The display device 100 may transmit the video delay offset read in step S205 to the graphics thread for processing the graphics area (S207).

[0157] When the video delay offset is sent to the graphics thread through step S207, the display device 100 may control the screen according to the event signal received in step S201 to configure and output it by performing synchronization processing so that the video area and the graphics area are synchronized (S209).

[0158] When the characteristic of the event signal confirmed in step S203 is the second characteristic, the display device 100 may recalculate the video delay offset (new offset) for the video data related to the video area instead of reading the previously stored video delay offset from the memory 610 and using it as it is (S211).

[0159] As above, in contrast to the first feature described above, the second property may indicate a case where an additional frame is required for video processing.

[0160] The display device 100 may send the video delay offset recalculated in step S211 to the graphics thread for processing the graphics area (S213).

[0161] The display device 100 may control the screen according to the event signal received in step S201 to configure and output it by synchronizing the video region and the graphics region based on the video delay offset sent to the graphics thread in step S213 so that the video region and the graphics region are synchronized (S215).

[0162] With reference to the Fig. 9 to 13 provide a detailed description of the Fig. 7 to 8 illustrated embodiments of the present disclosure as follows.

[0163] Fig. 9 to 11 are drawings illustrating a method for synchronizing the video area and the graphics area in the video processing process of the display device 100 according to an embodiment of the present disclosure.

[0164] With reference to Fig. 9(a), an image can be set up as described above with the video area and the graphics area, wherein the video area and the graphics area must be set up and output so that they are synchronized with each other at the time of output.

[0165] However, if the display device 100 makes a request for application-level video playback at time t1, for example, a physical delay occurs until the data for the video region is decoded by the video decoder 620 and then finally processed and output by the video processor 630.

[0166] The video processor 630 may send a video output readiness completion fact (a type of event) to the application layer when the processing of the decoded video data is completed and output is possible for the video domain (ie, at time t1') or when the output preparation for the video domain is completed.

[0167] At the application level, the graphics processor 640 may be requested to process the graphics region when the video region output readiness completion fact (at time t1') is received from the video processor 630.

[0168] The graphics processor 640 may process and output the graphics data for the graphics area (at time t1') according to the processing request of the application layer.

[0169] Through the above process, the display device 100 can output an image, that is, an image in which the video area and the graphics area are synchronized and synchronous, as shown in Fig. 9(b).

[0170] In other words, after the display device 100 completes the processing of the video region including the physical delay, it performs the processing of the graphics region without the physical delay, so that a single image synchronized with the video region and the graphics region can be output. Taking the physical delay into account, the processing of the video region is requested first at time t1, and then the processing of the graphics region is requested at time t1'. The physical delay is handled by controlling the processing request time, so that a synchronous screen can be established at the output time.

[0171] With reference to Fig. 10 can, however, be used with the method from Fig. 9 Processing, such as the next setting, may be restricted at the application level before image processing is completed according to the control, such as a setting (e.g. operation).

[0172] As in Fig. 10(a) and Fig. 10(b), if at application level, for example, at time t1 (see Fig. 9) If image processing is requested according to a specific setting, the graphics area processing is executed after the video area processing is completed, including the physical delay to ensure synchronization. In this case, if the physical delay is long or the video area processing is not executed before a certain time (e.g., at or after t2), the image processing originally requested at time t2 according to a different setting cannot be executed or will inevitably be delayed.

[0173] For example, if the processing speed for an image is assumed to be 30 fps, the physical delay in the video domain may exceed 33.3 ms and the 30 fps may not be met. This assumption often occurs when processing high-resolution images. Therefore, in a situation such as Fig. 10 a supplement to the procedure Fig. 9 may be required.

[0174] In Fig. 11(a) and Fig. 11(b) is considered as a complementary measure to the procedure under Fig. 9 discloses a method for synchronizing the video region and the graphics region, in which the physical delay offset incurred in processing the video region is communicated in advance to the graphics thread for processing the graphics region, so that the processing of the graphics region is postponed (or delayed) and processed based on the offset.

[0175] With reference to Fig. 11(b), at the application level, at time t1, the physical delay offset that occurs in the video domain is also sent to the graphics domain, so that the processing of the video domain starts at time t1, but the processing of the graphics domain starts at time t1', so that the two domains can be processed synchronously.

[0176] However, according to another embodiment of the present disclosure, the processing speed of the image may be set differently depending on various circumstances such as the screen mode of the display device, the type or property of the content, etc., and the display device 100 may implement the method of Fig. 11 when the physical delay occurring during the video range processing process calculated for one shot is greater than the time between the instants for another shot, and may otherwise apply the method of Fig. 9 apply.

[0177] Fig. 12 and Fig. 13 are drawings illustrating the video processing operation of the display device 100 according to an embodiment of the present disclosure.

[0178] With reference to Fig. 12, a method of processing an image by synchronizing the video and the graphic on the application execution screen downloaded and installed from the display device 100 will be described.

[0179] For simplicity, the application is described as an example that is not a native application integrated into the display device 100 during manufacture, but this is not limited thereto.

[0180] The display device 100 can execute an application according to the user's request and display an execution screen. For example, assume the user has switched to the "Music" tab.

[0181] In the above case, the display of the display device 100 may display the video in a thumbnail state when the focus is on a specific music video thumbnail.

[0182] After the video has been decoded in the decoder driver, information about the video can be sent to the video driver, i.e. the video scaler. At this time, the information about the video can be, for example, specifications such as 1080p, 60 Hz in the case of Fig. 12 included.

[0183] The video scaler can calculate a physical delay offset. The video scaler can calculate the physical delay offset based on the video information transmitted by the decoder driver mentioned above. For example, if a video scaler requires 4 frames of video data for video processing and the input video synchronization is 60 Hz, it can calculate a delay offset of 4 x 16.67 = 66.66 ms for the 4 frames of video data, since 60 Hz corresponds to 16.67 ms per frame.

[0184] The physical delay offset of 66.66 ms calculated by the video scaler can be reported to the surface manager, stored, and sent to the graphics thread of the graphics driver.

[0185] Because the display device 100 passes the video's delay information, the image size can be easily enlarged and moved when the corresponding video is clicked. In other words, it can be expanded to full screen mode. In this case, the video and graphics must be synchronized. In the graphics thread, by applying the previously received delay offset (66.66 ms), the video is synchronized with the graphics whenever the graphics change. This allows the synchronized video to be output during application execution.

[0186] Fig. 13 describes a method for controlling the operation when, as in Fig. 12, the application installed on the display device 100 is not executed directly, but a web browser is executed and then, after switching to the page, a specific video is played on the page corresponding to the application.

[0187] After the video is decoded in the decoder driver, when the video information is transferred to the video scaler, the video scaler can use the transferred video information to calculate the physical delay for video processing compared to graphics.

[0188] As described above, the video scaler requires 4 frames of video data for video processing. Fig. 13 assumes that the input video synchronization is 30 Hz, but this is not limited to this. 30 Hz corresponds to 33.33 ms per frame. Since 4 frames of video are ultimately required, this results in a delay offset of 4 x 33.33 = 133.33 ms. The calculated delay offset (i.e., 133.33 ms) can be sent to the graphics thread.

[0189] If the display device 100 is a display whose display field is as shown in Fig. 5(a) and Fig.5(b), and the display panel is rotated 90 degrees according to the user's request, i.e., the image is first output horizontally and then vertically, the video scaler can calculate an additional delay corresponding to the image rotation. Here, if 1 frame of video data is required for image rotation, 1 frame at the aforementioned 30 Hz will cause a delay of 33.33 ms. The newly generated delay plus the existing delay can be calculated as the final operation delay offset (a total of 33.33 + 133.33 = 166.66 ms) and sent to the graphics thread.

[0190] In order for the screen in display device 100 to move smoothly when a video position movement is requested, for example, by scrolling from top to bottom, the video and graphics must be synchronized during the corresponding operation. To this end, the graphics thread can apply the pre-received delay offset of 166.66 ms to synchronize with the video whenever the graphics change.

[0191] Although not explicitly mentioned, the order of at least some of the operations disclosed in the present disclosure may be performed concurrently, may be performed in a different order than described, or some may be omitted / added.

[0192] According to one embodiment of the present invention, the method described above can be implemented as processor-readable code on a medium on which a program is stored. Examples of processor-readable media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc.

[0193] The display device described above is not limited to the configuration and method of the above-described embodiments, and the embodiments can be configured by selectively combining all or part of each embodiment, so that various modifications can be made. Industrial applicability

[0194] According to the display device according to the present disclosure, a video area in which a physical delay occurs and a corresponding graphic area can be easily synchronized by software without adding hardware, so that an image in which the video area and the graphic area are synchronized can be provided to the user, and thus there is industrial applicability.

Claims

[1] A method of operating a display device, comprising: Receiving a video playback request signal; Decoding video data; Calculating an offset related to a video delay by receiving information of the decoded video data; Sending the calculated offset with respect to the video delay to a graphics thread; and Controlling a screen's output by synchronizing video and graphics. [2] The method of claim 1, further comprising storing the calculated offset with respect to the video delay. [3] The method of claim 2, further comprising: Receiving an event signal and Checking a property of the received event signal. [4] The method of claim 3, further comprising: if the checked property of the received event signal is a first property, reading the stored offset and sending the offset to the graphics thread; and Controlling a screen's output according to the event signal by synchronizing video and graphics. [5] The method of claim 4, wherein the first characteristic represents an event signal that does not require any additional frames for video processing other than the number of frames used to calculate the offset for the calculated video delay. [6] The method of claim 3, further comprising: if the checked property of the received event signal is a second property, recalculating an offset for the video delay; Saving the recalculated offset for the video delay and sending the recalculated offset to the graphics thread and Controlling a screen's output according to the event signal by synchronizing video and graphics. [7] The method of claim 6, wherein the second characteristic represents an event signal that requires additional frames for video processing in addition to the number of frames used to calculate the offset for the calculated video delay. [8] Display device comprising: a video decoder configured to decode video data when a video playback request signal is received; a video processor configured to receive information of the decoded video data and calculate an offset related to a video delay; a graphics processor configured to process graphics data for the decoded video data; and a controller configured to control the output of a display on which video and graphics are synchronized by sending the calculated offset relative to the video delay to a graphics thread of the graphics processor. [9] A display device according to claim 8, further comprising a memory arranged to store the calculated offset with respect to the video delay. [10] A display device according to claim 9, wherein, when an event signal is received, the controller is arranged to check a property of the received event signal. [11] The display device of claim 10, wherein, when the checked property of the received event signal is a first property, the controller is configured to read the stored offset and send the offset to the graphics thread of the graphics processor and output a screen corresponding to the event signal by synchronizing video and graphics. [12] The display device of claim 11, wherein the first characteristic represents an event signal that does not require any additional frames for video processing other than the number of frames used to calculate the offset for the calculated video delay. [13] The display device of claim 10, wherein, when the verified property of the received event signal is a second property, the controller is configured to control the video processor to recalculate an offset for the video delay, control the video processor to store the recalculated offset for the video delay in memory, send the recalculated offset to the graphics thread of the graphics processor, and output a screen corresponding to the event signal by synchronizing video and graphics. [14] The display device of claim 13, wherein the second characteristic represents an event signal that requires additional frames for video processing in addition to the number of frames used to calculate the offset for the calculated video delay.

Images