Display driving apparatus and display apparatus

Abstract

A display driving apparatus and a display apparatus are provided. The display apparatus includes: a video processing unit that processes a video signal; a graphic processing unit that processes the graphic signal; a mixing unit that mixes a video corresponding to the processed video signal and a graph corresponding to the processed graph signal; the display unit is used for outputting the mixed video and graph; and a main processing unit configured to control the video processing unit, the graphics processing unit, the mixing unit, and the display unit.

Description

[0001] This application claims the benefit of Korean Patent Application No. 10-2024-0107272, filed on August 12, 2024, with the Korean Intellectual Property Office, the entire disclosure of which is incorporated herein by reference. Technical Field

[0002] This disclosure relates to display driver devices and display devices. Background Technology

[0003] Display devices equipped with display units (such as televisions (TVs)) can receive various content provided from external sources and output images to the display units based on said content. With the increasing popularity of various video services using network environments, content including additional images (such as graphics) can be provided. Display devices may include additional processing units that process video signals and graphic signals separately, and these two processed signals may be mixed to output an image. Because video signals and graphic signals are processed through different paths, video frames and graphic frames may mismatch. Therefore, noise may appear in the output image, which can cause inconvenience to the user when viewing the output image. Summary of the Invention

[0004] In general, this disclosure relates to a display device for synchronizing and outputting video frames and graphics frames.

[0005] According to some embodiments, this disclosure relates to a display device, the display device comprising: a video processing unit for processing video signals; a graphics processing unit for processing graphics signals; a mixing unit for mixing video corresponding to the processed video signals and graphics corresponding to the processed graphics signals; a display unit for outputting the mixed video and graphics; and a main processing unit configured to control the video processing unit, the graphics processing unit, the mixing unit, and the display unit, wherein the video signals comprise a plurality of video frames, and the graphics signals comprise a plurality of graphics frames, and the main processing unit is configured to: calculate a video delay time for each of the plurality of video frames using a video rendering request time for each of the plurality of video frames to be rendered and a video output time for each of the plurality of video frames to be output to the display unit; calculate a graphics delay time for each of the plurality of graphics frames using a graphics rendering request time for each of the plurality of graphics frames to be rendered and a graphics output time for each of the plurality of graphics frames to be output to the display unit; and synchronize the plurality of video frames and the plurality of graphics frames by compensating for at least one of the video delay time and the graphics delay time.

[0006] According to some embodiments, this disclosure relates to a display device, the display device comprising: a video processing unit for processing video signals comprising multiple video frames and including multiple video processing components; a graphics processing unit for processing graphics signals comprising multiple graphics frames and including multiple graphics processing components; a mixing unit for mixing video corresponding to the processed video signals and graphics corresponding to the processed graphics signals; a display unit for outputting the mixed video and graphics; and a main processing unit configured to control the video processing unit, the graphics processing unit, the mixing unit, and the display unit, wherein the number of the multiple video processing components is greater than the number of the multiple graphics processing components, and the main processing unit is configured to: shorten the video delay time from the time for rendering each of the multiple video frames to the time for outputting each of the multiple video frames to the display unit; and extend the graphics delay time from the time for rendering each of the multiple graphics frames to the time for outputting each of the multiple graphics frames to the display unit.

[0007] According to some embodiments, this disclosure relates to a display driving device, the display driving device comprising: a first semiconductor chip including a first video processing unit, a first graphics processing unit, a first main processing unit, and a first memory unit, wherein the first video processing unit processes a video signal including multiple video frames, the first graphics processing unit processes a graphics signal including multiple graphics frames, the first main processing unit is configured to control the first video processing unit and the first graphics processing unit, and the first memory unit includes a first video buffer for storing processed video signals and a first graphics buffer for storing processed graphics signals; and a second semiconductor chip including a second video processing unit, a second graphics processing unit, a second main processing unit, and a second memory unit, wherein the second video processing unit processes the processed video signals, the second graphics processing unit processes the processed graphics signals, and the second main processing unit is configured to control the second video processing unit and the first graphics processing unit. The second graphics processing unit and the second memory unit include a second video buffer for storing processed video signals and a second graphics buffer for storing processed graphics signals. The first main processing unit is configured to: calculate a video delay time, from the time required to render each of the plurality of video frames to the time required to output each of the plurality of video frames; and calculate a graphics delay time, from the time required to render each of the plurality of graphics frames to the time required to output each of the plurality of graphics frames. The first and second main processing units are configured to: shorten the video delay time by omitting at least a portion of a plurality of first video processing components included in the first video processing unit and a plurality of second video processing components included in the second video processing unit; or lengthen the graphics delay time by adding a graphics buffer to at least one of the first and second graphics buffers.

[0008] According to some embodiments, this disclosure relates to the processing of the delay time of a video signal by a video processing unit of a display device and the processing of the delay time of a graphics signal by a graphics processing unit, wherein the delay time of the video signal and the delay time of the graphics signal can be controlled to be the same, so that the video signal and the graphics signal can be synchronized and output. Therefore, noise appearing in the output image can be reduced. Attached Figure Description

[0009] The exemplary embodiments will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings.

[0010] Figure 1 This is a diagram illustrating an example of a display device according to some embodiments.

[0011] Figure 2 This is a diagram illustrating an example of the geometry of video and graphics in a display device according to some embodiments.

[0012] Figure 3 This is a block diagram illustrating an example configuration of a display device according to some implementation methods.

[0013] Figure 4 This is a block diagram schematically illustrating examples of software and hardware used for outputting images in a display device according to some embodiments.

[0014] Figure 5 This illustrates according to some embodiments. Figure 4 A block diagram illustrating an example of hardware configuration.

[0015] Figure 6 This is a flowchart illustrating an example of an operation in a display device for controlling the synchronization and output of video and graphics, according to some embodiments.

[0016] Figure 7 This is a flowchart illustrating an example of a process for calculating video latency and graphics latency according to some implementation methods.

[0017] Figure 8 This is a flowchart illustrating an example of a process for calculating video latency and graphics latency according to some implementation methods.

[0018] Figure 9 This is a flowchart illustrating an example of a process for compensating for video delay time according to some implementation methods.

[0019] Figure 10 This is a flowchart illustrating an example of a process for compensating for video delay time according to some implementation methods.

[0020] Figure 11 This is a flowchart illustrating an example process for compensating for graphics delay time according to some implementation methods.

[0021] Figures 12 to 14 This is a diagram illustrating examples of vertical synchronization signals, video signals, and graphics signals according to some implementation methods.

[0022] Figure 15 This is a block diagram schematically illustrating an example configuration of a display device according to some implementations. Detailed Implementation

[0023] In the following text, exemplary embodiments will be explained in detail with reference to the accompanying drawings.

[0024] Figure 1 This is a diagram illustrating an example of a display device according to some embodiments. Figure 2 This is a diagram illustrating an example, according to some embodiments, in which the geometry of video and graphics is simultaneously altered in a display device.

[0025] exist Figure 1 In this embodiment, the display device 1 can be implemented as a display device including a display unit 10. The display device 1 can receive data about content from an external signal source, and can process the received content data according to a preset processing method to output the data as an image on the display unit 10.

[0026] In some embodiments, the display device 1, implemented as a display device, may be implemented as a television (TV) capable of processing broadcast images based on at least one of broadcast signals, broadcast information, and broadcast data received from a transmitting device of a broadcasting station. In this case, the display device 1 may be provided with a tuner for tuning the broadcast signals for each channel.

[0027] Display device 1 can be an image processing device (such as a set-top box) that sends signals to an external display unit connected by a wired or wireless connection. Display device 1 can also be a terminal device (including a smartphone or smart board (such as a tablet computer)) equipped with a display unit. Furthermore, display device 1 can be a monitor for a personal computer (PC) (such as a desktop computer or a laptop computer).

[0028] When display device 1 is a television, display device 1 can receive broadcast content based on at least one of broadcast signals, broadcast information, and broadcast data received directly or via an auxiliary device from the transmitting equipment of a broadcasting station. The auxiliary device can be connected to display device 1 via a cable or the like. The auxiliary device can correspond to a set-top box (STB), an OC box (single-connect box), a media box, etc. Wired or wireless interfaces, such as cables, can be used as connection methods between display device 1 and the auxiliary device.

[0029] Display device 1 can wirelessly receive broadcast content as radio frequency (RF) signals transmitted from a broadcasting station. For this purpose, display device 1 may include an antenna capable of receiving broadcast signals. However, the signal supply source for display device 1 is not limited to a broadcasting station, and the broadcast content can be received via terrestrial waves, cables, or satellites.

[0030] The standard of the signal received by display device 1 can be configured in various ways in response to the implementation of the device. Display device 1 can receive signals corresponding to the following standards as video content via wired connection: High Definition Multimedia Interface (HDMI), HDMI-Consumer Electronics Control (HDMI-CFC), DisplayPort (DP), DVI, Composite Video, Component Video, Super Video, Digital Vision Interface (DVI), Thunderbolt (or Thunderbolt interface), RGB cable, Radio Receiver and Television Equipment Manufacturers Alliance (SCART), Universal Serial Bus (USB), etc.

[0031] In some implementations, the display device 1 can be implemented as a smart TV or Internet Protocol TV (IP TV). A smart TV can receive and output broadcast signals in real time and may have web browsing capabilities, thereby providing a user environment where various content can be searched and consumed via the Internet while simultaneously outputting live broadcast signals. Furthermore, a smart TV may include an open software platform to provide interactive services to users. Therefore, a smart TV can provide users with content from applications offering various services through an open software platform. Such applications are applications that provide various types of services, including applications that provide services such as social networking, finance, news, weather, maps, music, movies, games, and e-books.

[0032] Display device 1 can process signals based on signals / data stored in internal / external storage media to output moving images, still images, applications, OSD displays, and user interfaces (UIs) for various operation controls on the screen.

[0033] Display device 1 is a source for providing content and can receive content from various external devices, including servers and terminal devices, via wired or wireless network communication, and the type of communication is unrestricted.

[0034] Specifically, display device 1 can receive signals corresponding to standards (such as Wi-Fi, Wi-Fi Direct, Bluetooth™, Bluetooth™ Low Energy, Zigbee™, Ultra Wideband (UWB), and Near Field Communication (NFC)) as video content via wireless network communication, in response to an implementation of interface unit 120 described below. Furthermore, display device 1 can receive content signals via wired network communication (such as Ethernet).

[0035] In some implementations, the external device may be configured as a content provider (i.e., a content server) that sends content to various devices, including display device 1, via a wired or wireless network. For example, the external device may provide media files (such as video on demand (VOD) or web content) in a live streaming manner.

[0036] In some implementations, multiple external devices may be included. In this case, the display device 1 may be connected to each of the multiple external devices and may be configured to receive various content from each of the connected external devices.

[0037] Display device 1 may receive video content (such as VOD or media content) from a web server (such as YouTube) or an over-the-top (OTT) service (such as Netflix).

[0038] Display device 1 can execute an application for content playback (e.g., a VOD application), and can receive content from an external device provided for content delivery, process the received content, and output an image corresponding to the content through display unit 10. Here, display device 1 can receive content from an external device using a user account corresponding to the executed application.

[0039] In some embodiments, the display device 1 may be as follows Figure 1 The diagram shows that video 22 and graphics 24 are output together on display unit 10. Specifically, display device 1 can receive a video signal corresponding to video 22 and a graphics signal corresponding to graphics 24, and can process the received video signal and the received graphics signal separately. Display device 1 can mix the video signal and graphics signal processed through separate paths, so that video 22 and graphics 24 can be output together on display unit 10.

[0040] Graphical signals may include signals used for outputting information (such as sub-pictures, captions, teletext, on-screen displays (OSDs) or user interfaces (UIs) output to convey information to a user (such as channel numbers and program titles) or for various operational controls). However, this disclosure is not limited thereto.

[0041] The graphic signal may be included in content provided from an external device (such as a server), or it may be provided from the external device as a separate signal from the content. Here, the external device providing the content and the external device providing the graphic signal may be the same or different from each other. Furthermore, the graphic signal may be built into an additional device (such as a display device 1 or a set-top box). In some embodiments, the graphic signal may be formed from multiple layers.

[0042] In some embodiments, the display device 1 may output interactive graphics (IG) or presentation graphics (PG) generated by processing graphics signals to the display unit 10 as graphics output.

[0043] In some implementations, video 22 may be overlaid on graphics 24 and may be output to display unit 10. Video 22 may be output in a separate area from the area where graphics 24 are output.

[0044] exist Figure 2 In the case where video 22 and graphics 24 are simultaneously output to display unit 10, the area of ​​video 22 can be gradually expanded. At this time, the time required for video 22 to be output to display unit 10 may be longer than the time required for graphics 24 to be output to display unit 10.

[0045] A typical display device 1 can be controlled by extending the time required for the graphic 24 to be output to the display unit 10. However, due to the resolution, size, position, etc. of the video 22, the video 22 and the graphic 24 may be mismatched, which can lead to noise.

[0046] Display device 1 can control the time required for video 22 and graphics 24 to be output to display unit 10 to be the same. Therefore, video 22 and graphics 24 can be synchronized and output to display unit 10, which reduces noise.

[0047] Figure 3 This is a block diagram illustrating an example configuration of a display device according to some implementation methods. Figure 3 In the display device 100, there may be a display unit 110, an interface unit 120, a user input unit 130, a memory unit 140, a video processing unit 150, a graphics processing unit 160, a mixing unit 170, and a main processing unit 180.

[0048] The display unit 110 can display images. The display unit 110 can be implemented in the form of liquid crystal, plasma, light-emitting diode, organic light-emitting diode, surface conduction electron emitter, carbon nanotube, nanocrystal, etc.

[0049] In some implementations, display unit 110 may output images of content received from an external device (such as a server). In example embodiments, display unit 110 may output video based on video signals and graphics based on graphics signals together. In some implementations, video may be output on display unit 110 while simultaneously overlapping graphics. Video may be output in a separate area separate from the area where graphics are output.

[0050] Interface unit 120 allows display device 100 to communicate with various external devices, including servers. Interface unit 120 may include wired interface unit 122 and wireless interface unit 124.

[0051] The wired interface unit 122 may include a connection unit for transmitting / receiving signals / data according to standards such as HDMI, HDMI-CFC, USB, component video, DisplayPort (DP), DVI, Thunderbolt, and RGB cable. The wired interface unit 122 may include at least one connector, at least one terminal, or at least one port corresponding to each of these standards.

[0052] The wired interface unit 122 is implemented in the form of an input port that receives signals from a source, and in some cases it can be configured to allow bidirectional signal transmission and reception by further including an output port.

[0053] The wired interface unit 122 can be connected to an antenna that can receive broadcast signals according to broadcast standards such as terrestrial / satellite broadcasting. In some embodiments, the wired interface unit 122 may include connectors or ports according to video and / or audio transmission standards such as HDMI, DisplayPort, DVI, Thunderbolt, composite video, component video, super video, SCART, etc., allowing cables to be connected to the wired interface unit 122.

[0054] When display device 1 receives broadcast signals through interface unit 120, display device 1 may further include a tuner that tunes the received broadcast signals by channel. The tuner may include a demodulator that demodulates the broadcast signal of the specific channel being tuned and outputs the broadcast signal as a transport stream (TS) signal. In other words, the tuner and demodulator may be designed as a single integrated chip, or they may be implemented as two separate chips.

[0055] Wired interface unit 122 may include a connector or port according to a common data transmission standard (such as a USB port). Wired interface unit 122 may include a connector or port according to an optical transmission standard (such as a connector or port to which an optical cable can be connected). Wired interface unit 122 may include a connector or port to which an external microphone or an external audio device equipped with a microphone is connected, and may receive or input audio signals from or to the microphone or audio device.

[0056] Wired interface unit 122 may include a connector or port to which an audio device (such as headphones, earphones, or an external speaker) can be connected, and which connector or port can transmit or output audio signals to the audio device. Wired interface unit 122 may include a connector or port according to a network transmission standard (such as Ethernet). For example, wired interface unit 122 may be implemented as a LAN card or the like for wired connection to a router or gateway.

[0057] The wired interface unit 122 can be wired to external devices (such as set-top boxes, optical media players or external display devices, speakers, servers, etc.) in a 1:1 or 1:N (where N is a natural number) manner to receive or send video / audio signals from or to external devices. The wired interface unit 122 may include connectors or ports for transmitting video / audio signals separately.

[0058] The wired interface unit 122 can be implemented as a communication circuit system including wireless communication modules (S / W modules, chips, etc.) corresponding to various types of communication protocols. In some embodiments, the wired interface unit 122 can be built into the display device 100, but it can also be implemented as an adapter (dongle, or converter) or module, and can be detachable from the connector of the display device 100.

[0059] The wireless interface unit 124 can be implemented in various ways corresponding to the implementation of the display device 100. For example, the wireless interface unit 124 can use wireless communication (such as radio frequency (RF), Zigbee, Bluetooth, Wi-Fi, ultra-wideband (UWB) and near field communication (NFC)) as the communication method.

[0060] The wireless interface unit 124 can be implemented as a communication circuit system including wireless communication modules (S / W modules, chips, etc.) corresponding to various types of communication protocols. In some embodiments, the wireless interface unit 124 may include a wireless LAN unit. The wireless LAN unit can be wirelessly connected to external devices via an access point (AP) under the control of the main processing unit 180. The wireless LAN unit includes a Wi-Fi module.

[0061] In some embodiments, the wireless interface unit 124 includes a wireless communication module that supports one-to-one direct communication between the display device 100 and an external device without an access point. The wireless communication module may be implemented to support communication methods such as Wi-Fi Direct, Bluetooth, and Bluetooth Low Energy. When the display device 100 performs direct communication with an external device, the memory unit 140 may store identification information (such as MAC address and IP address) of the external device used as the communication target device.

[0062] In the display device 100, the wireless interface unit 124 is configured to perform wireless communication with an external device via at least one of a wireless LAN unit and a wireless communication module, depending on performance. In some embodiments, the wireless interface unit 124 may also include a communication module that communicates via various communication methods, such as mobile communication (e.g., LTE), EM communication including magnetic fields, and visible light communication.

[0063] The wireless interface unit 124 can communicate wirelessly with external devices (such as servers on a network) to send data packets to or receive data packets from external devices.

[0064] The wireless interface unit 124 may include an IR transmitter and / or IR receiver capable of transmitting and / or receiving infrared (IR) signals according to infrared communication standards. The wireless interface unit 124 can receive or input remote control signals from a remote control or another external device via the IR transmitter and / or IR receiver, or can transmit or output remote control signals to another external device. As another example, the display device 100 can transmit and receive remote control signals with a remote control or other external device via the wireless interface unit 124 having another method (such as Wi-Fi or Bluetooth).

[0065] In some implementations, the wireless interface unit 124 may transmit predetermined data, which is information of a user's voice received via a voice input unit (such as a microphone), to an external device (such as a server). Here, the form / type of the transmitted data is not limited, and for example, the data may include an audio signal corresponding to the voice spoken by the user or voice features extracted from the audio signal.

[0066] Furthermore, the wireless interface unit 124 can receive data showing the processing results of the user's voice from an external device (such as a server). The display device 100 can output sound corresponding to the voice processing results via an internal or external speaker based on the received data. In some embodiments, the user's voice can be processed within the display device 100 itself without being sent to a server. That is, in some embodiments, the display device 100 can be implemented as a speech-to-text (STT) server.

[0067] The display device 100 can communicate with an input device (such as a remote control) via a wireless interface unit 124, thereby receiving a sound signal corresponding to the user's voice from the input device.

[0068] In some implementations, the communication module that communicates with external devices (such as servers) and the communication module that communicates with the remote controller may be different from each other. For example, the display device 100 may communicate with external devices via an Ethernet modem or a Wi-Fi module, and may communicate with the remote controller via a Bluetooth module.

[0069] In some implementations, the communication module that communicates with external devices (such as servers) and the communication module that communicates with remote controllers may be the same as each other. For example, the display device 100 may perform communication with external devices and remote controllers via a Bluetooth module.

[0070] In some implementations, the wireless interface unit 124 may be built into the display device 100, but it may also be implemented as an adapter or module and may be detachable from the connector of the display device 100.

[0071] In some embodiments, the display device 100 may receive broadcast signals via the interface unit 120. The display device 100 may extract or generate video signals corresponding to video and graphic signals corresponding to graphics based on data included in the broadcast signals.

[0072] In some implementations, the display device 100 may receive content signals in real-time streaming mode from an external device (such as a server) via the interface unit 120. The display device 100 may extract or generate video signals corresponding to video and graphic signals corresponding to graphics based on the content signals.

[0073] The user input unit 130 can send various preset control commands or unrestricted information to the main processing unit 180 through user input. The user input unit 130 includes various input tools capable of receiving user input. In some embodiments, the user input unit 130 may include a keyboard with buttons (such as power keys, numeric keys, and menu keys) disposed on the display device 100.

[0074] In some embodiments, the user input unit 130 may include an input device for generating preset commands / data / information / signals to enable remote control of the display device 100 and sending the commands / data / information / signals to the display device 100. The input device may include, for example, a remote control, a game controller, a keyboard, a mouse, etc., and may be detachable from the display device 100 and capable of receiving user input.

[0075] The remote control may be equipped with at least one button capable of receiving user input. In some embodiments, the remote control may be equipped with a touch detection unit for receiving user touch input and / or a motion detection unit for detecting user movement of the remote control itself. In some embodiments, the input device may include a terminal device (such as a smartphone with a remote control application installed), and in this case, user touch input via a touchscreen may be received.

[0076] An input device is an external device capable of wirelessly communicating with the main body of the display device 100, and the wireless communication includes Bluetooth, infrared communication, RF communication, wireless LAN, and Wi-Fi Direct.

[0077] In some implementations, the user input unit 130 may include a voice input unit that receives speech / sound spoken by the user. The voice input unit may be implemented as a microphone capable of receiving the user's voice, and the microphone may be located in the display device 100, separately from the display device 100, or in another device (such as a remote control) separate from the display device 100.

[0078] In some embodiments, the user input unit 130 may include a motion detection unit that detects the movement of the user's hand (i.e., gestures). The motion detection unit of the display device 100 may detect the distance the hand moves, the speed of movement, the area of ​​the movement region, etc., and may output data.

[0079] Memory unit 140 can be configured to store various data of display device 100. Memory unit 140 can retain data even when the power supply to display device 100 is interrupted, and may be provided with writable non-volatile memory (writable ROM) so that changes can be reflected therein. That is, memory unit 140 may be provided with one of flash memory, EPROM, and EEPROM. Memory unit 140 may also include volatile memory (such as DRAM or SRAM), which has a faster read or write speed for display device 100 compared to non-volatile memory.

[0080] The data stored in memory unit 140 includes, for example, an operating system for driving display device 100, and also includes various programs, applications 142, image data, and additional data executable on the operating system.

[0081] Specifically, memory unit 140 may store input / output signals or data in response to the operation of each component, as controlled by main processing unit 180. Memory unit 140 may store control programs for controlling display device 100, UI related to applications provided by the manufacturer or downloaded from external sources, graphics or images used to provide the UI, user information, documents, databases, or related data.

[0082] In some implementations, memory unit 140 may store a TV application or TV client as a program that allows the display device 100 to operate as a television, and a VOD application as a program that allows content received from an external device (such as a server) to be played.

[0083] In some embodiments, the video and graphics output from the display device 100 may originate from data stored in a non-volatile memory unit 140 (such as flash memory, hard disk, etc.). The memory unit 140 may be located inside or outside the display device 100, and when the memory unit 140 is located outside, the memory unit 140 may be connected to the display device 100 via a wired interface unit 122.

[0084] In some embodiments, the memory unit 140, the ROM or RAM in the main processing unit 180, or a memory card that can be installed in the display device 100 may be included.

[0085] The video processing unit 150 can process video signals so that the video corresponding to the video signals can be output to the display unit 110. The display device 100 can process video signals via the video processing unit 150 through a video path.

[0086] The graphics processing unit 160 can process graphics signals so that the graphics corresponding to the graphics signals can be output to the display unit 110. The display device 100 can process graphics signals via the graphics processing unit 160 through a graphics path. The graphics path may be different from the video path.

[0087] The mixing unit 170 can combine video signals and graphic signals, and output the video signals and graphic signals to the display unit 110. Therefore, the video corresponding to the video signal processed by the video processing unit 150 and the graphic corresponding to the graphic signal processed by the graphic processing unit 160 can be output together to the display unit 110.

[0088] In some implementations, the mixing unit 170 may be implemented as a hardware configuration (e.g., a chip) and may ultimately output synchronized video and graphics frames to the display unit 110.

[0089] In some implementations, the blending unit 170 may perform alpha blending for compositing video and graphics based on transparency information that indicates the degree of transparency of the graphics. The transparency information may correspond to an alpha value. For example, the alpha value may be 8-bit data capable of distinguishing between levels of transparency from 0 to 255.

[0090] The mixing unit 170 can synthesize video and graphics signals by referencing an alpha value. The image, where alpha mixing has been performed in the mixing unit 170 based on the alpha value, can be output through the display unit 110. In this case, the image may include both video and graphics.

[0091] Since the video signal and the graphics signal are combined by the mixing unit 170 and output to the display unit 110, the video and graphics can be output on a single screen of the display unit 110.

[0092] The main processing unit 180 performs control for operating all components of the display device 100. The main processing unit 180 may include a control program that allows such control operations to be performed, a non-volatile memory in which the control program is installed, a volatile memory (such as DRAM) in which at least a portion of the installed control program is loaded, and at least one processor (such as a microprocessor, application processor, or central processing unit CPU) that executes the loaded control program.

[0093] The processor included in the main processing unit 180 may be single-core, dual-core, triple-core, quad-core, and / or multi-core. Furthermore, the processor, ROM, and RAM are interconnected via an internal bus.

[0094] In some implementations, multiple processors may be configured. For example, the display device 100 may be configured with a separate subprocessor that operates in a sleep mode, which only supplies standby power and does not function as a display device.

[0095] In some implementations, the main processing unit 180 may execute application 142 to determine geometric information for representing the video and graphics. The geometric information may include size and position information for each of the video and graphics as parameters for representing the video and graphics. For example, the geometric information may include coordinate values ​​(x, y, w, h).

[0096] Geometric information may include a start point and an end point. Video can be controlled to be output within the interval between the start and end points. Graphics may include multiple planes, and each plane can be controlled to be output within the interval between the start and end points.

[0097] In some implementations, the main processing unit 180 may be implemented as being included in a main system-on-a-chip (SoC) mounted on a PCB embedded in the display device 100.

[0098] The time required for the video signal to be processed in the video processing unit 150 and for the video to be output to the display unit 110 may be longer than the time required for the graphics signal to be processed in the graphics processing unit 160 and for the graphics to be output to the display unit 110. In other words, the video path used to process the video signal may be longer than the graphics path used to process the graphics signal.

[0099] The display device 100 can control the time required for video and graphics to be output to the display unit 110 to be the same. Therefore, video and graphics can be synchronized and output to the display unit 110.

[0100] Figure 4 This is a block diagram schematically illustrating examples of software and hardware used for outputting images in a display device according to some embodiments. Figure 5 This illustrates according to some embodiments. Figure 4 A block diagram of an example configuration of the hardware shown.

[0101] exist Figure 4 In this context, the configuration of the display device 200 can be divided into software (SW) and hardware (HW). In some embodiments, the display device 200 can be integrated with the above-described... Figures 1 to 3 The display device described herein is similar.

[0102] The software (SW) may include application 210. Application 210 may include video driver 212 and graphics driver 214. In some embodiments, video driver 212 may be a video driver and graphics driver 214 may be a graphics driver. The hardware (HW) may include video processing unit 220, graphics processing unit 230, memory unit 240, mixing unit 250, and display unit 260.

[0103] Application 210 can request the rendering of video and graphics.

[0104] The video driver 212 can configure video geometry for representing the video corresponding to the video signal, and can send video geometry information to the video processing unit 220. The video geometry information may include size and position information for the video. Specifically, the video signal may include multiple video frames, and the video geometry information may include the size and position of each of the multiple video frames.

[0105] exist Figure 4 and Figure 5 In this context, the video processing unit 220 may include a multiplexer (MUX), a video quality block, and an FRC quality block. Each of the video quality block and the FRC quality block may include at least one video processing configuration. That is, the video processing unit 220 may include multiple video processing components. A video path may correspond to a multiplexer (MUX) or an FRC quality block.

[0106] Signals output from HDMI, Video Decoder (VDEC), Audio-Visual (AV), and Digital Television (DTV) can be input to the video processing unit 220. Specifically, the signals can be input to a multiplexer (MUX), and the multiplexer (MUX) can output one of the signals to a video quality block. In this case, one signal can correspond to a video signal, and the video signal can include multiple video frames.

[0107] A video quality block performs quality processing on the input video signal. For example, a video quality block may include multiple video processing components, and these components may perform different quality processing operations on the video signal.

[0108] The video quality block can normalize the video signal to meet required standard conditions. Furthermore, the video quality block can scale the video signal based on first geometric information received from the video driver 212, and can write the video signal into the memory unit 240. Specifically, multiple video frames can be stored in the video buffer 242 included in the memory unit 240.

[0109] Furthermore, the video quality block can perform operations such as noise reduction of multiple video frames stored in the video buffer 242 or processing of user-set image quality. The video quality block can output the video signal that has undergone quality processing to the FRC quality block.

[0110] The FRC quality block can perform frame rate conversion (FRC) of video signals. For example, the FRC quality block may include multiple video processing components, and these components may perform different frame rate conversion operations on the video signals. The FRC quality block can convert the number of frames per second of a video image, and the video image with the converted frame rate per second can be output to the mixing unit 250.

[0111] exist Figure 4 In this context, the graphics driver 214 can configure the graphics geometry for representing the graphics corresponding to the graphics signal, and can send the graphics geometry information to the graphics processing unit 230. The graphics geometry information may include size information and position information for the graphics. Specifically, the graphics signal may include multiple graphics frames, and the graphics geometry information may include the size and position of each of the multiple graphics frames.

[0112] exist Figure 4 and Figure 5 In this configuration, the graphics processing unit 230 may include multiple graphics processing components. These components may include a graphics processing unit (GPU), a central processing unit (CPU), and a graphics quality block. The CPU may be connected to... Figure 3 This corresponds to a portion of the main processing unit 180. The graphics path may correspond to a graphics processing unit (GPU) or a graphics quality block.

[0113] The graphics processing unit (GPU) and the central processing unit (CPU) can form multiple planes to form a graphics frame. These planes may include a main plane and overlay planes. The GPU and CPU can use graphics geometry information received from the graphics driver 214 to set graphics data for forming the multiple planes and store the graphics data in the memory unit 240. Specifically, the set graphics data can be stored in the graphics buffer 244.

[0114] The graphics quality block can output multiple planes included in a single graphics frame from the graphics buffer 244. The graphics quality block can combine multiple output planes into a single graphics frame to improve quality, and thus can output that frame to the mixing unit 250. The graphics quality block may include multiple graphics processing components, and the multiple graphics processing components can perform different quality processing operations on the planes or graphics frames.

[0115] The mixing unit 250 can mix video frames provided by the video processing unit 220 and graphics frames provided by the graphics processing unit 230, and output the mixed video and graphics frames to the display unit 260. Therefore, the corresponding video frames and the corresponding graphics frames can be output to the display unit 260 together.

[0116] The video path of the video processing unit 220 may be longer than the graphics path of the graphics processing unit 230. In other words, the time required for the video processing unit 220 to process the video may be longer than the time required for the graphics processing unit 230 to process the graphics. This means that a problem may occur where the video and graphics are not synchronized and output to the display unit 260.

[0117] In some implementations, the display device 200 can control the time required for video and graphics to be output to the display unit 260 to be the same, so that the video and graphics can be synchronized and output to the display unit 260.

[0118] In some implementations, when the video processing unit 220 processes a video signal, the time required for the video to be output to the display unit 260 can be shortened by omitting or disabling at least one of the plurality of video processing components included in the video processing unit 220, or by executing, enabling, or using fewer video processing components than all of the plurality of video processing components. For example, at least one of the plurality of video processing components included in the video quality block and the FRC quality block can be omitted.

[0119] In some implementations, when the graphics processing unit 230 processes graphics signals, the number of graphics buffers 244 can be increased to extend the time required for graphics to be output to the display unit 260. For example, the number of graphics buffers 244 can be increased such that the time required for graphics to be output to the display unit 260 is the same as the time required for video to be output to the display unit 260.

[0120] Figure 6 This is a flowchart illustrating an example of operations for controlling the synchronization and output of video and graphics in a display device according to some embodiments. Figure 6 In this context, the display device may include a video processing unit, a graphics processing unit, a mixing unit, a display unit, a memory unit, and a main processing unit.

[0121] The video processing unit processes video signals, and the graphics processing unit processes graphics signals. The mixing unit can mix the video signal corresponding to the processed video signal and the graphics signal corresponding to the processed graphics signal. The display unit can output the mixed video and graphics.

[0122] The memory unit may include a video buffer for storing the video signal processed by the video processing unit and a graphics buffer for storing the graphics signal processed by the graphics processing unit. The main processing unit can control the video processing unit, graphics processing unit, mixing unit, display unit, and memory unit. Furthermore, the main processing unit can execute applications stored in the memory unit.

[0123] The video signal may include multiple video frames, and the graphics signal may include multiple graphics frames. In some embodiments, the display device may be associated with the above-mentioned... Figures 1 to 5 The display device described herein is similar. The following section describes the operation of the display device for controlling the synchronization and output of video and graphics.

[0124] Users can input commands into the application (S100). Figure 2 In this context, where video and graphics are output together on a display device, the user can input a command to switch the video to full screen into the application.

[0125] Based on the user's command, the application can request the rendering of video and graphics (S110). In other words, the application can sequentially request the rendering of corresponding synchronized and output video frames and graphics frames.

[0126] The application can send video geometry information for video to the video processing unit, and can also send graphic geometry information for graphics to the graphics processing unit. The video processing unit can use the video geometry information to process the video signal, and the graphics processing unit can use the graphic geometry information to process the graphic signal (S120).

[0127] Reference Figure 5 Video signals can be processed by multiple video processing components of the video processing unit. Graphics signals can be processed by multiple graphics processing components of the graphics processing unit. Video and graphics signals can be processed frame by frame according to a synchronization signal, and the synchronization signal can be a vertical synchronization signal for a display or display unit.

[0128] The video corresponding to the processed video signal and the graphic corresponding to the processed graphic signal can be mixed, and the mixed video and graphic can be output to the display unit (S130). In other words, based on the vertical synchronization signal for the display unit, multiple video frames included in the video signal and multiple graphic frames included in the graphic signal can be output sequentially.

[0129] The main processing unit can calculate the video latency time and the graphics latency time (S140). The video latency time for each of the multiple video frames can be calculated using the video rendering request time for rendering each of the multiple video frames and the video output time (or video frame output time) for outputting each of the multiple video frames to the display unit. For example, the video latency time can be the time from the video rendering request time to the video output time.

[0130] The graphics latency for each of the multiple graphics frames can be calculated using the graphics rendering request time for each request to render in the multiple graphics frames and the graphics output time (or graphics frame output time) for outputting each of the multiple graphics frames to the display unit. For example, the graphics latency can be the time from the graphics rendering request time to the graphics output time.

[0131] The main processing unit can determine whether the video delay time is the same as the graphics delay time (S150). When the video delay time and the graphics delay time are the same ("Yes" in S150), the video frame and the graphics frame can be synchronized and output sequentially based on the vertical synchronization signal (S170).

[0132] When the video delay time and the graphics delay time are not the same ("No" in S150), the main processing unit can compensate for at least one of the video delay time and the graphics delay time (S160). Since the video delay time and the graphics delay time are controlled to be the same, the video frame and the graphics frame can be synchronized based on the vertical synchronization signal and output sequentially (S170).

[0133] When the output of video and graphics is not terminated ("No" in S180), the processing and output of video and graphics signals, the calculation of video delay time and graphics delay time, and the synchronization and output of video and graphics frames (S120 to S170) can be repeatedly performed.

[0134] In the following text, reference will be made to Figure 6 and Figure 7 Describe in detail the processing steps for calculating video latency and graphics latency.

[0135] Figure 7 This is a flowchart illustrating an example of the processing for calculating video latency and graphics latency according to some implementation methods. Figure 7 In this process, based on the user's command, the application may request the rendering of video and graphics (S200). In this case, the processing for calculating the video delay time corresponds to processes S211 to S215, and the processing for calculating the graphics delay time corresponds to processes S221 to S225.

[0136] When the process of calculating video latency is described, the main processing unit may confirm video information and video geometric information (S211). Video information may include bits per pixel (BPP), frames per second (FPS), and resolution. Video geometric information may include the size and position of each of the multiple video frames. Geometric information may include coordinate values ​​(x, y, w, h).

[0137] The main processing unit can store the video rendering request time for each requested rendering of multiple video frames of the video signal in metadata (context data) (S212). For example, the video rendering request time can be stored in the metadata in the form of a timestamp.

[0138] The main processing unit can process and output multiple video frames (S213), and in some embodiments, the processing of S213 can be combined with the above. Figure 6 The process described in S120 and S130 is similar.

[0139] The main processing unit can output the confirmed video frame time and the video rendering request time for each of the multiple video frames (S214). The main processing unit can use the video frame output time and the video rendering request time to calculate the video delay time (S215). Specifically, the video delay time can be the time from the video rendering request time to the video frame output time.

[0140] When the process of calculating the graphics delay time is described, the main processing unit may confirm the graphics geometry information (S221). The graphics geometry information may include the size and position of each of the multiple graphics frames of the graphics. The geometry information may include coordinate values ​​(x, y, w, h).

[0141] The main processing unit can store the graphics rendering request time for each of the multiple graphics frames for the graphics signal in metadata (context data) (S222). For example, the graphics rendering request time can be stored in the metadata in the form of a timestamp.

[0142] The main processing unit can process and output multiple graphics frames (S223), and in some embodiments, the processing of S223 can be combined with the above. Figure 6 The process described in S120 and S130 is similar.

[0143] The main processing unit can confirm the output time of each graphics frame and the graphics rendering request time in multiple graphics frames (S224). The main processing unit can use the graphics frame output time and the graphics rendering request time to calculate the graphics delay time (S225). Specifically, the graphics delay time can be the time from the graphics rendering request time to the graphics frame output time.

[0144] Figure 8This is a flowchart illustrating an example of the processing for calculating video latency and graphics latency according to some implementation methods. Regarding video rendering request time and graphics rendering request time, Figure 7 and Figure 8 The implementations shown differ from one another in that: Figure 7 In the embodiment shown, the time can be stored in metadata, while Figure 8 In the embodiment shown, the time can be calculated by analyzing the binarization pattern (or binarization mode) inserted into the frame.

[0145] Based on the user's command, the application may request the rendering of video and graphics (S300). In this case, the processing for calculating the video delay time corresponds to processes S311 to S315, and the processing for calculating the graphics delay time corresponds to processes S321 to S325.

[0146] When the process of calculating video delay time is described, the main processing unit can confirm video information and video geometric information (S311). The main processing unit can insert the output order of multiple video frames of the video signal as a video binarization pattern into each of the multiple video frames (S312). Specific details of the operation of inserting the video binarization pattern into each of the multiple video frames can be described in Korean Patent Application KR2023-0056893, which is incorporated herein by reference.

[0147] The main processing unit can process and output multiple video frames (S313). For each of the multiple video frames, the main processing unit can determine the video frame output time and analyze the video binarization pattern to calculate the video rendering request time (S314). The main processing unit can use the video frame output time and the video rendering request time to calculate the video delay time (S315). Specifically, the video delay time can be the time from the video rendering request time to the video frame output time.

[0148] When the process of calculating the graphics delay time is described, the main processing unit can confirm the graphics geometry information (S321). The main processing unit can insert the output order of multiple graphics frames of the graphics signal as a graphics binarization pattern into each of the multiple graphics frames (S322). The specific details of the operation of inserting the graphics binarization pattern into each of the multiple graphics frames can be described in Korean Patent Application KR2023-0056893, which is incorporated herein by reference.

[0149] The main processing unit can process and output multiple graphics frames (S323). For each of the multiple graphics frames, the main processing unit can determine the graphics frame output time and analyze the graphics binarization pattern to calculate the graphics rendering request time (S324). The main processing unit can use the graphics frame output time and the graphics rendering request time to calculate the graphics delay time (S325). Specifically, the graphics delay time can be the time from the graphics rendering request time to the graphics frame output time.

[0150] Figure 9 This is a flowchart illustrating an example of a process for compensating for video delay time according to some implementation methods. Figure 10 This is a flowchart illustrating an example of a process for compensating for video delay time according to some implementation methods.

[0151] In some embodiments, the display device may include a video processing unit, a graphics processing unit, a mixing unit, a display unit, a memory unit, and a main processing unit. The video processing unit may include multiple video processing components, and the graphics processing unit may include multiple graphics processing components.

[0152] Reference Figure 5 Multiple video processing components may include a multiplexer (MUX) or a frame rate converter (FRC). Multiple graphics processing components may include a graphics processing unit (GPU) or a quality enhancement unit (QE). The number of multiple video processing components may be greater than the number of multiple graphics processing components.

[0153] Video and graphics signals can be processed for each frame according to a synchronization signal, which can be a vertical synchronization signal for the display or display unit. Therefore, since video signals can be processed by more video processing components, video latency can be longer than graphics latency.

[0154] Figure 9 and Figure 10 This can be compared to an example of processing that reduces video latency based on geometric information from multiple video frames. The geometric information can include the size and position of each of the multiple video frames. Figure 9 The following will show an example of a process that compensates for video latency by changing the frequency based on geometric information.

[0155] The main processing unit can calculate the frequency of change of geometric information for multiple video frames (S400). The frequency of change of geometric information can refer to the number of times the frame size or frame position changes during a unit of time. The frequency of change of geometric information can be included in the video information, can be calculated from the video information, or can be calculated by measuring the number of changes per unit of time using a counter.

[0156] The main processing unit can determine whether the frequency of change of geometric information is greater than or equal to the reference frequency (S410). For example, the reference frequency could be the case where the frame size or frame position changes 10 times within 1 second.

[0157] When the frequency of change of geometric information is not greater than or equal to the reference frequency (e.g., the frequency of change of geometric information is less than the reference frequency) ("No" in S410), the main processing unit can fully process multiple video processing components (S420). In other words, the video signal can be processed by all of the multiple video processing components.

[0158] The video delay time can be maintained in the same way (S430), and video frames can be output while maintaining the video delay time (S470). In other words, since the delay time difference is maintained, video and graphics can be output without synchronization, which can lead to noise.

[0159] When the frequency of change of geometric information is greater than or equal to the reference frequency (S410 "Yes"), the main processing unit can calculate the delay time difference, which is the difference between the video delay time and the graphics delay time (i.e., video-graphics delay time) (S440).

[0160] The main processing unit can perform partial processing by omitting at least one of multiple video processing components (S450). See reference. Figure 5 The main processing unit may omit at least one of the multiple video processing components included in the video quality block and the FRC quality block.

[0161] The omitted components can be determined by the delay time difference, and the required frame delay and delay time difference in the omitted components can be the same as each other. That is, the video delay time is shortened by the delay time difference (S460). Video frames can be output with the shortened video delay time (S470). In other words, since there is no delay time difference, the video and graphics can be synchronized and output.

[0162] Processing steps S400 through S470 can be repeated until the output of video and graphics is terminated.

[0163] exist Figure 10 The example shown is an example of a process that compensates for video latency based on the size of multiple video frames.

[0164] The main processing unit can determine whether the size of the video frame is less than or equal to a reference size (S500). For example, the reference size may correspond to half the size of the display unit. Specifically, the reference horizontal size may correspond to half the horizontal size of the display unit, and the reference vertical size may correspond to half the vertical size of the display unit.

[0165] When the size of the video frame is not less than or equal to the reference size (for example, the size of the video frame is greater than the reference size) ("No" in S500), the main processing unit can fully process multiple video processing components (S510). In other words, the video signal can be processed by all of the multiple video processing components.

[0166] The video delay time can be maintained in the same way (S520), and video frames can be output while maintaining the video delay time (S560). In other words, since the delay time difference is maintained, video and graphics can be output without synchronization, which can lead to noise.

[0167] When the size of the video frame is less than or equal to the reference size (S500 "Yes"), the main processing unit can calculate the delay time difference, which is the difference between the video delay time and the graphics delay time (S530).

[0168] The main processing unit can perform partial processing by omitting at least one of multiple video processing components (S540). See reference. Figure 5 The main processing unit may omit at least one of the multiple video processing components included in the video quality block and the FRC quality block.

[0169] The omitted components can be determined by the delay time difference, and the required frame delay and delay time difference in the omitted components can be the same as each other. That is, the video delay time can be shortened by the delay time difference (S550). The video frame can be output with the shortened video delay time (S560). In other words, since there is no delay time difference, the video and graphics can be synchronized and output.

[0170] Processing steps S500 through S560 can be repeated until the output of video and graphics is terminated.

[0171] Figure 11 This is a flowchart illustrating an example of a process for compensating for graphics latency according to some implementation methods. Figure 11 In this context, the display device may include a video processing unit, a graphics processing unit, a mixing unit, a display unit, a memory unit, and a main processing unit. The memory unit may include a video buffer for storing video signals processed by the video processing unit and a graphics buffer for storing graphics signals processed by the graphics processing unit.

[0172] Video and graphics signals can be processed for each frame according to the vertical synchronization signal. Video and graphics buffers can be in a first-in, first-out (FIFO) format to store processed frames. That is, video and graphics frames can be stored sequentially in the video and graphics buffers and then output. Therefore, as buffers are added, frame delay can increase until the stored frames are output.

[0173] Figure 11 This corresponds to the process of extending the graphics delay time by reflecting an additional graphics buffer in the graphics buffer.

[0174] The main processing unit can calculate the delay time difference, which is the difference between the video delay time and the graphics delay time (S600). The main processing unit can allocate an additional graphics buffer corresponding to the delay time difference (S610). In other words, the frame delay and the delay time difference required in the additional graphics buffer can be the same.

[0175] The main processing unit can reflect the additional graphics buffer in the graphics buffer (S620), and the graphics delay time can be extended by the delay time difference (S630). The graphics frame can be output with the extended graphics delay time (S640). That is, since there is no delay time difference, the video and graphics can be synchronized and output.

[0176] Processing steps S600 through S640 can be repeated until the output of video and graphics is terminated.

[0177] Figures 12 to 14 This is a diagram illustrating examples of vertical synchronization signals, video signals, and graphics signals according to some implementation methods. Figures 12 to 14 In this context, the display device can process each of the video and graphics signals for each frame based on a synchronization signal. In this case, the synchronization signal can be a vertical synchronization signal for the display unit.

[0178] exist Figures 12 to 14 In this context, the rendering request time Tr for each rendering request in the video and graphics signals can be the same for each other. The rendering request time Tr can correspond to the interval between the first pulse time T1 and the second pulse time T2 of the vertical synchronization signal.

[0179] first, Figure 12 It can show the video and graphics signals of a display device in which video and graphics delay times are not compensated.

[0180] The graphics signal can be processed in the graphics processing unit, and the first graphics frame G1 can be output to the display unit at the second pulse time T2 of the vertical sync signal. The graphics delay time can be the time from the rendering request time Tr to the second pulse time T2. The video signal can be processed in the video processing unit, and the first video frame V1 can be output to the display unit at the fourth pulse time T4 of the vertical sync signal. The video delay time can be the time from the rendering request time Tr to the fourth pulse time T4.

[0181] The difference between video latency and graphics latency can be calculated as the latency difference. Figure 12 In this context, the time from the second pulse time T2 to the fourth pulse time T4 of the vertical synchronization signal can be considered as a delay time difference. Video delay time can be longer than graphics delay time.

[0182] The number of video processing components can be greater than the number of graphics processing components. Therefore, the frame latency required by multiple video processing components can be longer than that required by multiple graphics processing components. In other words, video and graphics can be output without synchronization, which can lead to noise.

[0183] Figure 13 Examples of video and graphics signals used by a display device to perform compensation for video latency can be shown. Figure 9 and Figure 10 In this process, video latency can be shortened based on geometric information from multiple video frames.

[0184] exist Figure 13 In this process, the display device can shorten the video delay time difference by omitting at least one of multiple video processing components. Therefore, the first video frame V1 and the first graphic frame G1 can be output to the display unit at the second pulse time T2 of the vertical synchronization signal. That is, the video and graphics can be synchronized and output.

[0185] Figure 14 Examples of video and graphics signals used by a display device to perform compensation for graphics latency can be shown. Figure 11 In this context, the graphics delay time can be extended based on the delay time difference.

[0186] exist Figure 14 In this configuration, the display device can allocate an additional graphics buffer corresponding to the delay time difference, which can be reflected in the graphics buffer and can extend the graphics delay time by increasing the delay time difference. Therefore, the first video frame V1 and the first graphics frame G1 can be output to the display unit at the fourth pulse time T4 of the vertical synchronization signal. That is, the video and graphics can be synchronized and output.

[0187] Figure 15This is a block diagram schematically illustrating an example configuration of a display device according to some implementations. Figure 15 The display device 300 may include a first semiconductor chip 310, a second semiconductor chip 320, a mixing unit 330, and a display unit 340. The display device 300 may also include an interface unit and a user input unit.

[0188] The first semiconductor chip 310 and the second semiconductor chip 320 can be display driver devices for driving the display device 300. The first semiconductor chip 310 can be a mobile application processor (mobile AP) that can be provided in the form of a system-on-a-chip (SoC). The second semiconductor chip 320 can be provided in the form of a digital television system-on-a-chip (DTV SoC).

[0189] The first semiconductor chip 310 may include a first video processing unit 312, a first graphics processing unit 314, a first main processing unit 316, and a first memory unit 318. The first video processing unit 312 can process video signals, and the first graphics processing unit 314 can process graphics signals. The first main processing unit 316 can control the first video processing unit 312 and the first graphics processing unit 314. The first memory unit 318 may include a first video buffer for storing processed video signals and a first graphics buffer for storing processed graphics signals.

[0190] The second semiconductor chip 320 may include a second video processing unit 322, a second graphics processing unit 324, a second main processing unit 326, and a second memory unit 328. The second video processing unit 322 can process the video signal processed by the first video processing unit 312. The second graphics processing unit 324 can process the graphics signal processed by the first graphics processing unit 314. The second main processing unit 326 can control the second video processing unit 322 and the second graphics processing unit 324. The second memory unit 328 may include a second video buffer for storing the processed video signal and a second graphics buffer for storing the processed graphics signal.

[0191] In some implementations, the first semiconductor chip 310 may receive rendering requests from an application. The application may send video geometry information for representing video and graphic geometry information for representing graphics to the first main processing unit 316.

[0192] The first main processing unit 316 can calculate the video delay time of each of the multiple video frames using the video rendering request time for each requested rendering of the multiple video frames for the video signal and the video output time for each of the multiple video frames to be output to the display unit.

[0193] The first main processing unit 316 can calculate the graphics delay time of each of the multiple graphics frames using the graphics rendering request time for each of the multiple graphics frames requested to be rendered for the graphics signal and the graphics output time for each of the multiple graphics frames to be output to the display unit.

[0194] The first main processing unit 316 and the second main processing unit 326 can shorten the video delay time by omitting at least a portion of the plurality of first video processing components included in the first video processing unit 312 and the plurality of second video processing components included in the second video processing unit 322.

[0195] The first main processing unit 316 may allocate an additional graphics buffer corresponding to a delay time difference, which is the difference between the video delay time and the graphics delay time. The first main processing unit 316 and the second main processing unit 326 may extend the graphics delay time by allocating the additional graphics buffer to at least one of the first graphics buffer and the second graphics buffer. In other words, a graphics buffer (e.g., an additional graphics buffer) may be added to at least one of the first graphics buffer and the second graphics buffer.

[0196] In some implementations, the first main processing unit 316 can extend the graphics delay time by reflecting an additional graphics buffer to the first graphics buffer. The graphics signal with the extended graphics delay time can then be sent from the first graphics processing unit 314 to the second graphics processing unit 324 via the display port.

[0197] In some implementations, the graphics signal can be transmitted from the first graphics processing unit 314 to the second graphics processing unit 324 via a display port. The second main processing unit 326 can then extend the graphics delay time by reflecting an additional graphics buffer to the second graphics buffer.

[0198] In some implementations, a portion of the additional graphics buffer may be reflected in the first graphics buffer, and another portion of the additional graphics buffer may be reflected in the second graphics buffer. In other words, the first main processing unit 316 can extend a portion of the graphics delay time by reflecting a portion of the additional graphics buffer in the first graphics buffer. The graphics signal with the extended graphics delay time can be sent to the second graphics processing unit 324 via the display port. The second main processing unit 326 can extend a portion of the graphics delay time by reflecting another portion of the additional graphics buffer in the second graphics buffer.

[0199] The first main processing unit 316 can shorten the video delay time difference based on the geometric information of multiple video frames.

[0200] In some embodiments, the first main processing unit 316 can shorten the video delay time difference by omitting at least one of the plurality of first video processing components included in the first video processing unit 312. The video signal with the shortened video delay time can be sent to the second video processing unit 322 through the display port.

[0201] In some implementations, the video signal can be transmitted from the first video processing unit 312 to the second video processing unit 322 via a display port. The second main processing unit 326 can then shorten the video delay time difference by omitting at least one of the plurality of second video processing components included in the second video processing unit 322.

[0202] In some embodiments, the first main processing unit 316 may shorten a portion of the video delay time difference by omitting at least one of the plurality of first video processing components included in the first video processing unit 312. The video signal with the shortened video delay time may be transmitted to the second video processing unit 322 via a display port. The second main processing unit 326 may shorten another portion of the video delay time difference by omitting at least one of the plurality of second video processing components included in the second video processing unit 322.

[0203] The mixing unit 330 can mix video frames provided by the second video processing unit 322 and graphics frames provided by the second graphics processing unit 324. The mixed video frames and graphics frames can be output to the display unit 340. Therefore, corresponding video frames and corresponding graphics frames can be synchronously and sequentially output to the display unit 340.

[0204] While this disclosure contains numerous details of specific implementations, these details should not be construed as limiting the scope of the claims, their equivalents, and the appended claims. Certain features described in the context of individual implementations in this disclosure may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may be implemented individually or in any suitable sub-combination in multiple implementations. Furthermore, although features may be described above as functioning in certain combinations, one or more features from a combination may be removed from that combination in some cases, and the combination may involve sub-combinations or variations thereof.

Claims

1. A display device, comprising: A video processor is configured to process video signals; The graphics processor is configured to process graphics signals; The mixing processor is configured to mix processed video signals and processed graphics signals to provide a mixed video and graphics signal; The display device is configured to output mixed video and graphics signals; as well as The main processor is configured to control the video processor, graphics processor, mixed-signal processor, and display device. The video signal includes multiple video frames, and the graphics signal includes multiple graphics frames. The main processor is configured as follows: For each of the plurality of video frames, the corresponding video delay time is calculated using both the video rendering request time for the video frame to be rendered and the video output time for the video frame to be output to the display device. For each of the plurality of graphics frames, the corresponding graphics latency is calculated using both the graphics rendering request time for requesting rendering of the graphics frame and the graphics output time for outputting the graphics frame to the display device; and The plurality of video frames and the plurality of graphic frames are synchronized by compensating for at least one video delay time, compensating for at least one graphic delay time, or compensating for at least one video delay time and at least one graphic delay time.

2. The display device according to claim 1, in, For each video frame, the corresponding video latency is the time from the video rendering request time for that video frame to the video output time for that video frame. For each graphics frame, the corresponding graphics latency is the time from the graphics rendering request time for the graphics frame to the graphics output time for the graphics frame.

3. The display device according to claim 1, wherein, The main processor is configured to store video rendering request times and graphics rendering request times in context data.

4. The display device according to claim 1, wherein, The main processor is configured as follows: The output order of the multiple video frames is used as a video binarization pattern and inserted into each of the multiple video frames. The video rendering request time is calculated by analyzing the video binarization pattern. as well as The output order of the plurality of graphics frames is inserted as a graphics binarization pattern into each of the plurality of graphics frames, and the graphics rendering request time is calculated by analyzing the graphics binarization pattern.

5. The display device according to claim 1, in, A video processor includes multiple video processing components. The graphics processor includes multiple graphics processing components, and The number of the plurality of video processing components is greater than the number of the plurality of graphics processing components.

6. The display device according to claim 5, wherein, For at least one of the plurality of video frames, the video latency is longer than the graphics latency.

7. The display device according to claim 5, wherein, The main processor is configured as follows: Calculate the video-to-graphics delay time, where the video-to-graphics delay time is the difference between the first video delay time and the first graphics delay time; and Based on the video-graphics delay time, compensation is made for the first video delay time, compensation is made for the first graphic delay time, or compensation is made for both the first video delay time and the first graphic delay time.

8. The display device according to claim 7, wherein, The main processor is configured to compensate for the first video delay time based on geometric information from the plurality of video frames.

9. The display device according to claim 8, wherein, The geometric information includes the size and position of each of the plurality of video frames.

10. The display device according to claim 9, wherein, The main processing unit is configured as follows: Calculate the frequency of change of geometric information of the plurality of video frames; and Based on the fact that the frequency of change of geometric information is greater than or equal to the reference frequency, the delay time of the first video is shortened.

11. The display device according to claim 10, wherein, The main processor is configured to reduce the first video latency by enabling fewer video processing components than all of the plurality of video processing components combined with the video-to-graphics latency.

12. The display device according to claim 9, wherein, The main processor is configured to shorten the first video delay time based on the fact that the size of the plurality of video frames is less than or equal to a reference size.

13. The display device according to claim 12, wherein, The main processor is configured to reduce the first video latency by enabling fewer video processing components than all of the plurality of video processing components combined with the video-to-graphics latency.

14. The display device according to claim 7, further comprising: The memory circuit includes a video buffer and a graphics buffer, the video buffer being configured to store the video signal processed by the video processor, and the graphics buffer being configured to store the graphics signal processed by the graphics processor.

15. The display device according to claim 14, wherein, The main processor is configured to allocate an additional graphics buffer corresponding to the video-graphics latency, and to include the additional graphics buffer in the graphics buffer to extend the graphics latency.

16. A display device, comprising: A video processor is configured to process video signals comprising multiple video frames, and the video processor includes multiple video processing components; A graphics processor is configured to process graphics signals comprising multiple graphics frames, and the graphics processor includes multiple graphics processing components; The mixing processor is configured to mix processed video signals and processed graphics signals to provide a mixed video and graphics signal; The display device is configured to output mixed video and graphics signals; as well as The main processor is configured to control the video processor, graphics processor, mixed-signal processor, and display device. The number of the plurality of video processing components is greater than the number of the plurality of graphics processing components. The main processor is configured as follows: For each of the plurality of video frames, the corresponding video latency is shortened by using the time it takes to output each of the plurality of video frames to the display device; or For each of the plurality of graphic frames, the time taken to output each of the plurality of graphic frames to the display device is used to extend the corresponding graphic delay time.

17. The display device according to claim 16, wherein, For at least one of the plurality of video frames, the video latency is longer than the graphics latency.

18. The display device according to claim 17, in, The difference between video latency and graphics latency is called the latency difference, and The main processor is configured to: reduce the video delay time difference by enabling fewer video processing components than all of the multiple video processing components, based on the change frequency of the geometric information of the multiple video frames being greater than or equal to a reference frequency or the size of the multiple video frames being less than or equal to a reference size.

19. The display device according to claim 17, further comprising: The memory circuit includes a video buffer and a graphics buffer. The video buffer is configured to store the video signal processed by the video processor, and the graphics buffer is configured to store the graphics signal processed by the graphics processor. The difference between video latency and graphics latency is the latency difference, and The main processor is configured to allocate an additional graphics buffer corresponding to the delay time difference and include the additional graphics buffer in the graphics buffer to extend the graphics delay time.

20. A display driver device, comprising: A first semiconductor chip includes: a first video processor configured to process a video signal comprising multiple video frames; a first graphics processor configured to process a graphics signal comprising multiple graphics frames; a first main processor configured to control the first video processor and the first graphics processor; and a first memory circuit including a first video buffer configured to store the video signal processed by the first video processor and a first graphics buffer configured to store the graphics signal processed by the first graphics processor; and The second semiconductor chip includes: a second video processor configured to process a video signal processed by the first video processor; a second graphics processor configured to process a graphics signal processed by the first graphics processor; a second main processor configured to control the second video processor and the second graphics processor; and a second memory circuit including a second video buffer configured to store the video signal processed by the second video processor and a second graphics buffer configured to store the graphics signal processed by the second graphics processor. The first main processor is configured as follows: For each of the plurality of video frames, the corresponding video delay time is calculated using the time at which each of the plurality of video frames is output; and For each of the plurality of graphics frames, the corresponding graphics latency time is calculated using the time it takes to output each of the plurality of graphics frames. The first and second main processors are configured as follows: The video delay time is shortened by omitting at least a portion of the plurality of first video processing components included in the first video processor and the plurality of second video processing components included in the second video processor; or The graphics latency is extended by including the graphics buffer in at least one of the first graphics buffer and the second graphics buffer.

Images