Display device box and video frame control method

Abstract

This invention provides a display device box and a video frame control method. The display device box acquires the current video frame generated by decoding video data and the corresponding current decoding frame rate through a controller connected to a communicator. Simultaneously, it determines the video refresh time based on the current decoding frame rate. The controller determines the time difference between acquiring the current video frame and acquiring the video frame to be displayed. The video frame to be displayed is the video frame to be sent to the display device, and it determines whether to send the current video frame to the display device for display based on the video refresh time and the time difference. This allows for selective frame dropping based on the relationship between the video refresh time and the time difference, thereby overcoming the problem of uneven frame dropping in current frame dropping methods, which leads to intermittent image stuttering during video playback.

Description

Technical Field

[0001] This invention relates to the field of display technology, and more particularly to a display device box and a video frame control method. Background Technology

[0002] With the continuous development of intelligent display devices, current display devices generally come with built-in network modules, allowing direct viewing of online videos. Early display devices, however, did not have network modules. To enable these devices to watch online videos, display device boxes, or set-top boxes, were developed. These boxes include a network module, which can receive online video data and output it to the display device in the form of HDMI (High Definition Multimedia Interface) signals.

[0003] After receiving network video data, the set-top box decodes it, typically at frame rates of 25fps, 30fps, or 60fps. Simultaneously, the display device's refresh rate is usually 25Hz, 30Hz, or 60Hz. When the decoded frame rate and the display device's refresh rate do not match, dropped frames will appear in the displayed network video.

[0004] Currently, the common method to address this frame dropping issue is to set a video frame buffer. When the buffer is full, frames are dropped. However, this method results in uneven frame dropping, causing intermittent image stuttering during video playback. Summary of the Invention

[0005] This invention provides a display device box and a video frame control method to overcome the problem of uneven frame dropping caused by current frame dropping methods, which leads to intermittent image stuttering during video playback.

[0006] In a first aspect, embodiments of the present invention provide a display device box, comprising:

[0007] A communicator is used to establish a communication link between the controller and the display device;

[0008] The controller connected to the communicator is configured as follows:

[0009] Obtain the current video frame generated by decoding video data and the corresponding current decoding frame rate;

[0010] The video refresh time is determined based on the current decoding frame rate;

[0011] Determine the time difference between acquiring the current video frame and acquiring the video frame to be displayed; wherein, the video frame to be displayed is the video frame to be sent to the display device;

[0012] Whether to send the current video frame to the display device for display is determined based on the video refresh time and the time difference.

[0013] In some embodiments of this application, the controller is specifically configured as follows:

[0014] The video output frame rate is determined based on the current decoding frame rate;

[0015] The video refresh time is determined based on the video output frame rate.

[0016] In some embodiments of this application, the controller is provided with a video frame buffer; the video frame buffer is used to buffer a preset number of video frames generated after decoding the video data, and to send the buffered video frames to the display device for display in sequence;

[0017] The controller is specifically configured as follows:

[0018] The video refresh time is determined by multiplying the reciprocal of the video output frame rate and the preset frame number.

[0019] In some embodiments of this application, the controller is specifically configured as follows:

[0020] If it is determined that the time difference is less than the video refresh time, then the current video frame will not be sent to the display device for display.

[0021] If the time difference is determined to be greater than or equal to the video refresh time, the current video frame is sent to the display device for display.

[0022] In some embodiments of this application, the controller is further configured to:

[0023] Obtain a decoding frame rate adjustment instruction, and adjust the previous decoding frame rate of the current decoding frame rate according to the decoding frame rate adjustment instruction;

[0024] The adjusted previous decoding frame rate is set as the current decoding frame rate.

[0025] In some embodiments of this application, the decoding frame rate adjustment instruction includes video playback speed;

[0026] The controller is specifically configured as follows:

[0027] The current decoding frame rate is determined by multiplying the video playback speed by the previous decoding frame rate.

[0028] In some embodiments of this application, the controller is further configured to:

[0029] Monitor whether the video output frame rate of the current time period is the same as the video output frame rate of the previous time period;

[0030] If it is determined that the video output frame rate of the current time period is different from that of the previous time period, the video refresh time is updated according to the video output frame rate of the current time period.

[0031] In some embodiments of this application, the controller is further configured to:

[0032] Receive image display frame rate update information sent by the display device; the image display frame rate update information is generated when the display device detects that the image display frame rate of the display device in the current time period is different from the image display frame rate in the previous time period.

[0033] The video refresh time is updated based on the image display frame rate update information.

[0034] In some embodiments of this application, the controller is further configured to:

[0035] The video data is decoded according to the current decoding frame rate, and the decoded video data is processed for audio-visual synchronization.

[0036] In a second aspect, embodiments of the present invention provide a video frame control method, the method comprising:

[0037] Obtain the current video frame generated by decoding video data and the corresponding current decoding frame rate;

[0038] The video refresh time is determined based on the current decoding frame rate;

[0039] Determine the time difference between acquiring the current video frame and acquiring the video frame to be displayed; wherein, the video frame to be displayed is the video frame to be sent to the display device;

[0040] Whether to send the current video frame to the display device for display is determined based on the video refresh time and the time difference.

[0041] Thirdly, embodiments of the present invention provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the methods described in the first aspect and some embodiments thereof.

[0042] This embodiment provides a display device box and a video frame control method. The display device box acquires the current video frame generated by decoding video data and the corresponding current decoding frame rate through a controller connected to a communicator. Simultaneously, it determines the video refresh time based on the current decoding frame rate. The controller determines the time difference between acquiring the current video frame and acquiring the video frame to be displayed, and determines whether to send the current video frame to the display device for display based on the video refresh time and the time difference. This allows for selective frame dropping based on the relationship between the video refresh time and the time difference, thereby overcoming the problem of uneven frame dropping in current frame dropping methods, which leads to intermittent image stuttering during video playback. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 The diagram illustrates an operational scenario between a display device and a control device according to some embodiments;

[0045] Figure 2 The diagram illustrates, by way of example, a hardware configuration block diagram of a display device 200 according to some embodiments;

[0046] Figure 3 The diagram illustrates, by way of example, a hardware configuration block diagram of a control device 100 according to some embodiments;

[0047] Figure 4 The diagram illustrates, by way of example, a software configuration schematic of a display device 200 according to some embodiments;

[0048] Figure 5 The image above exemplarily illustrates a schematic diagram of an icon control interface display for an application in a display device 200 according to some embodiments;

[0049] Figure 6 The diagram illustrates a hardware configuration block diagram of a box according to some embodiments;

[0050] Figure 7 The image above exemplifies a scenario diagram of a display device and a display device box according to some embodiments;

[0051] Figure 8 The diagram illustrates a queuing scenario of video frames according to some embodiments;

[0052] Figure 9 The diagram illustrates, exemplarily, a component association diagram between a display device box and a television according to some embodiments;

[0053] Figure 10 The diagram above exemplarily illustrates a video frame queuing scenario in a video frame buffer according to some embodiments;

[0054] Figure 11 The diagram illustrates, by way of example, internal modules of a display device box according to some embodiments;

[0055] Figure 12 The diagram illustrates a display device box and a corresponding controller scenario according to some embodiments;

[0056] Figure 13 The diagram illustrates a video frame control method flowchart and partition diagram according to some embodiments of a display device box. Detailed Implementation

[0057] To make the objectives, implementation methods and advantages of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0058] Based on the exemplary embodiments described in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the appended claims. Furthermore, although the disclosures in this application are presented by way of one or more exemplary examples, it should be understood that each aspect of these disclosures can constitute a complete implementation on its own. It should be noted that the brief descriptions of terminology in this application are merely for the convenience of understanding the embodiments described below, and are not intended to limit the implementation of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0059] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device according to one or more embodiments of this application, such as... Figure 1 As shown, a user can operate the display device 200 via a mobile terminal 300 and a control device 100. The control device 100 can be a remote control, and communication between the remote control and the display device includes infrared protocol communication, Bluetooth protocol communication, wireless or other wired methods to control the display device 200. The user can input user commands through buttons on the remote control, voice input, control panel input, etc., to control the display device 200. In some embodiments, a mobile terminal, tablet computer, computer, laptop computer, and other smart devices can also be used to control the display device 200.

[0060] In some embodiments, the mobile terminal 300 can install software applications with the display device 200 to achieve connection and communication via network communication protocols, enabling one-to-one control operations and data communication. Audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 for synchronous display. The display device 200 also communicates with the server 400 via various communication methods. The display device 200 can communicate via a local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 can provide various content and interactive features to the display device 200. The display device 200 can be a liquid crystal display, an OLED display, or a projection display device. In addition to providing broadcast television reception functions, the display device 200 can also be equipped with a smart network television function that provides computer support.

[0061] Figure 2 An exemplary block diagram of the configuration of the control device 100 according to an exemplary embodiment is shown. Figure 2 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive user input commands and convert them into commands that the display device 200 can recognize and respond to, acting as an intermediary for interaction between the user and the display device 200. The communication interface 130 is used for external communication and includes at least one of a Wi-Fi chip, a Bluetooth module, NFC, or a replacement module. The user input / output interface 140 includes at least one of a microphone, a touchpad, a sensor, buttons, or a replacement module.

[0062] Figure 3 A hardware configuration block diagram of a display device 200 according to an exemplary embodiment is shown. For example... Figure 3The display device 200 shown includes at least one of the following: a tuner / demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface 280. The controller includes a central processing unit, a video processor, an audio processor, a graphics processor, RAM, ROM, and a first to nth interface for input / output. The display 260 can be at least one of a liquid crystal display, an OLED display, a touch display, and a projection display, and can also be a projection device and a projection screen. The tuner / demodulator 210 receives broadcast television signals via wired or wireless reception and demodulates audio and video signals, such as EPG data signals, from multiple wireless or wired broadcast television signals. The detector 230 is used to collect signals from the external environment or signals interacting with the external environment. The controller 250 and the tuner / demodulator 210 can be located in different separate devices; that is, the tuner / demodulator 210 can also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0063] In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display device 200. The user can input user commands through a graphical user interface (GUI) displayed on the monitor 260, and the user input interface receives the user input commands through the GUI. Alternatively, the user can input user commands by inputting specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0064] In some embodiments, a "user interface" is the medium through which an application or operating system interacts and exchanges information with a user, enabling the conversion between the internal form of information and a form acceptable to the user. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include at least one of the visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0065] Figure 4 This is a schematic diagram of the software configuration in a display device 200 according to one or more embodiments of this application, such as... Figure 4As shown, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the Android runtime and system library layer (referred to as the "System Runtime Layer"), and the kernel layer. The kernel layer contains at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, Wi-Fi driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0066] Figure 5 This is a schematic diagram showing the icon control interface of an application in a display device 200 according to one or more embodiments of this application, such as... Figure 5 As shown, the application layer contains at least one application whose corresponding icon control can be displayed on the screen, such as: live TV application icon control, video-on-demand application icon control, media center application icon control, application center icon control, game application icon control, etc. Live TV applications can provide live television from different signal sources. Video-on-demand applications can provide video from different storage sources. Unlike live TV applications, video-on-demand provides video display from certain storage sources. Media center applications can provide applications for playing various multimedia content. The application center can provide storage for various applications.

[0067] Figure 6 This is a hardware configuration block diagram of a box according to one or more embodiments of this application, such as Figure 6 As shown, the Box 500 provides analog audio and video interfaces, enabling current analog display devices to receive network video. It also provides a digital broadcast interface, offering interactive functionality. The Box 500 comprises several important modules:

[0068] Network Interface Module 501: The set-top box typically operates within a cable television network. Cable television networks use analog transmission, therefore digital signals must be modulated and demodulated for transmission over analog channels. Modulation and demodulation are critical components of the system. The network interface module performs channel demodulation and decoding functions, sending out a transmission stream containing audio, video, and other data information.

[0069] Source data transmission stream demultiplexer 502: The transport stream generally contains multiple audio and video streams and some data information. The transport stream demultiplexer is used to distinguish different programs, extract the corresponding audio and video streams and data streams, and send them to the MPEG II decoding module 503.

[0070] The MPEG II (Moving Picture Experts Group) decoding module 503. The MPEG II decoding module can be considered a core module outside the CPU. MPEG II digital transmission uses interleaved coding, requiring deinterleaving of the bitstream, separation of video, audio, and data bitstreams, and decoding of the video code. After these steps, the MPEG II bitstream becomes video and audio digital signals. The MPEG multiplexer combines the various program streams and data streams together, sending them as 188-byte frames of MPEG II data format to the video / audio decoder and post-processor 504.

[0071] 504 Audio / Video Decoder and Post-Processing: The MPEG II decoder decompresses the audio and video signals, and through the video encoder and audio digital-to-analog conversion, restores the analog audio and video signals, displays high-quality images on analog televisions, and provides multi-channel stereo programs.

[0072] Embedded processor CPU and memory module 505 and interface circuitry: The embedded processor CPU and memory module are used to store and run the software system and control various hardware modules. The interface circuitry provides a rich set of external interfaces, including a universal serial interface, an Ethernet interface, analog and digital audio / video interfaces, and data interfaces.

[0073] The various modules of the Box 500 work collaboratively during high-speed network access. Before entering the cable TV network, all signal sources undergo two levels of encoding. The first level is source encoding of the audio and video signals, encapsulating all sources into a transport stream. The second level is channel encoding for transmission. Corresponding to the headend, the Box first extracts the channel-coded signal from the transport layer, completes channel demodulation, then restores the compressed source-coded signal to recover the original audio and video stream, and simultaneously receives and interprets data services and various applications.

[0074] Box 500 selects a channel via network interface module 501, performs demodulation and channel decoding, and outputs MPEG II multi-program transport stream data, which is sent to source data transmission stream demultiplexer 502. Source data transmission stream demultiplexer 502 extracts a pre-packaged audio / video elementary stream data from the MPEG II transport stream data and sends it to MPEG II decoding module 503. MPEG II decoding module 503 extracts the video elementary stream data and sends it to audio / video decoder and post-processing 504. After extracting and decoding the MPEG II video data, it outputs it to analog encoder, encodes it into an analog video signal, and then outputs it through video output circuit. Audio elementary stream data is sent to audio / video decoder and post-processing 504. After extracting and decoding the MPEG II audio data, it outputs the audio data to audio digital-to-analog converter. Audio digital-to-analog converter outputs an analog stereo audio signal, which is then output through audio output circuit.

[0075] With the continuous development of display devices, the functions of smart display devices are becoming increasingly rich. Current smart TVs generally come with a built-in network module, allowing users to install applications to watch online videos. However, most display devices on the market are still older models that lack a network module. To address this functional deficiency in older display devices, display box devices have been developed. These boxes have a network module, can output HDMI signals (allowing connection to display devices with HDMI ports), and can also output other signals (connecting to display devices with other signal ports). Through pre-installed applications or downloaded applications, users can receive and play online videos on the display device, enabling them to watch online content.

[0076] The box connects to the display device and outputs images and sound via HDMI. The box's video output resolution and frame rate can be adjusted. Typical frame rates are 25Hz, 30Hz, 50Hz, and 60Hz. Playback video files come in various frame rates, with some sources reaching as high as 120fps (Frames Per Second). Each video frame exists as an image. To ensure smooth viewing, video files typically have a frame rate of 24fps or higher. Playback at double speed requires an even higher frame rate. For example, a 25fps video file played at 2x speed would have a decoding frame rate of 50fps. To ensure continuous, uninterrupted playback, the display device's refresh rate must be 50Hz or higher. When the display device's refresh rate cannot meet the playback requirements, dropped frames will occur. The common approach to handling dropped frames is to set up a video frame buffer. When the buffer is full, frames are dropped, such as dropping all frames already in the buffer or dropping frames that are about to enter it. However, this method of dropping frames is highly random and uneven, leading to intermittent image stuttering during video playback.

[0077] Based on this, and building upon the above embodiments, to solve the aforementioned technical problems, the inventors discovered through research that when the video output frame rate of the box's output image is greater than the image refresh rate of the display device, the video frames output by the box will continuously queue and wait to be displayed, and these video frames will continuously accumulate, easily leading to random frame drops. To address this situation, the relationship between the time difference between the currently displayed video frame and the queued video frames and the video refresh time corresponding to the video output frame rate can be used to control whether a particular video frame in the queue is dropped. For example, when the time difference is less than the video refresh time, the video frame can be dropped. Since video frames with a time difference less than the video refresh time generally do not appear consecutively multiple times, the frame drops are more evenly distributed, preventing image stuttering.

[0078] Therefore, this application provides a display device box. This display device box obtains the current video frame and corresponding current decoding frame rate generated by decoding video data through a controller connected to a communicator. Simultaneously, it determines the video refresh time based on the current decoding frame rate. The controller determines the time difference between obtaining the current video frame and obtaining the video frame to be sent to the display device for display, and determines whether to send the current video frame to the display device for display based on the video refresh time and the time difference. This allows for selective control of frame dropping based on the relationship between the video refresh time and the time difference, thereby overcoming the uneven frame dropping issues caused by current frame dropping methods, which result in intermittent image stuttering during video playback.

[0079] The display device box provided in this application includes a communicator and a controller, with the controller connected to the communicator. The communicator is used to establish a communication link between the controller and the display device. This communication link can be a wired communication link or a wireless communication link.

[0080] The controller acquires the current video frame generated from decoding video data and its corresponding current decoding frame rate. Simultaneously, after acquiring the current decoding frame rate, the controller determines the video refresh time based on that rate and the time difference between acquiring the current video frame and acquiring the video frame to be displayed. The video frame to be displayed is the one to be sent to the display device. Finally, the controller determines whether to send the current video frame to the display device for display based on the video refresh time and the time difference.

[0081] In some embodiments, such as Figure 7As shown, the display device box communicates with the display device. In practical applications, the display device box can be placed on top of the display device or around the display device. Placing it on top of the display device is more common; therefore, the display device box is also called a set-top box. The display device box and the display device can be connected via a wired connection using HDMI. The communicator includes an HDMI interface or other types of interfaces.

[0082] In some embodiments, video data is acquired by the display device box via a wireless network. After acquiring the video data, the display device box decodes the video data according to the current decoding frame rate. The decoding frame rate is generally the same as the video data frame rate; for example, if the video data frame rate is 50fps, the decoding frame rate is also 50fps. However, when the user controls the display device box to adjust the playback speed, the decoding frame rate will change accordingly. For example, if the playback speed is doubled, the decoding frame rate will change from 50fps to 100fps.

[0083] The video frame to be displayed refers to the video frame that is about to be sent to the display device for display. The video refresh time refers to the refresh time of the video frame. Each video frame is an image, therefore, the video refresh time can also be called the image refresh time.

[0084] Meanwhile, because video data is decoded in real time, the decoded video frames have a certain order, and there will be time differences between the video frames. Generally, the current video frame and the video frame to be displayed are adjacent, but if the display device box is set with a video frame buffer, there may be a certain number of video frames between the current video frame and the video frame to be displayed, or the decoding frame rate may be too high, which may also cause a certain number of video frames to be separated between the current video frame and the video frame to be displayed.

[0085] When the time difference is less than the video refresh time, it means the current video frame arrives too quickly. The video frame to be displayed won't be immediately sent to the display device, causing it to occupy the display queue. In this case, the current video frame is discarded. For example, if the decoding frame rate is 50 and the video refresh frame rate is 25, then the time to decode each frame is half the video refresh time. If the video refresh time is 1 / 25, the time to decode each frame is 1 / 50. Therefore, when one frame is refreshed, two video frames may already be waiting to be displayed. Figure 8As shown, the time difference between the current video frame and the video frame to be displayed is 1 / 50, which is less than the video refresh time. Therefore, the current video frame needs to be discarded. The time difference between the second frame and the video frame to be displayed is 1 / 25, so the second frame can be retained. Similarly, the time difference between the third frame and the second frame is less than the video refresh time, so the third frame will also be discarded. In existing frame dropping methods, due to queuing, frames are dropped randomly; for example, the current video frame and the second frame might both be dropped, only the third frame is retained. Therefore, the video frame control method in this embodiment can improve the uniformity of frame dropping.

[0086] In some embodiments, after determining the time difference, the controller can further determine the relationship between the time difference and the video refresh time. If the time difference is determined to be less than the video refresh time, the current video frame is not sent to the display device for display. If the time difference is determined to be greater than or equal to the video refresh time, the current video frame is sent to the display device for display.

[0087] Whether to send the current video frame to the display device for display is determined based on the video refresh time and the time difference. There are several cases. For example, if the time difference is less than the video refresh time, the current video frame will not be sent to the display device for display. If the time difference is greater than or equal to the video refresh time, the current video frame will be sent to the display device for display.

[0088] If the time difference is less than the video refresh time, it means that the current video frame arrived earlier. In this case, the video frame to be displayed will take some time to appear, so the video frame can be discarded.

[0089] If the time difference is greater than or equal to the video refresh time, it means that the current video frame arrives later. In this case, the video frame to be displayed will be displayed soon, so the video frame can be retained.

[0090] In some embodiments, the controller can determine the video output frame rate based on the current decoding frame rate, and determine the video refresh time based on the video output frame rate.

[0091] The current video frame refers to the most recently generated video frame during the decoding process. The video refresh time corresponds to the video output frame rate, which in turn corresponds to the current decoding frame rate. Therefore, the video refresh time can be determined based on the current decoding frame rate. Generally, the current decoding frame rate equals the video output frame rate, and the video refresh time is the reciprocal of the video output frame rate. If the display device box has a video frame buffer set, the video refresh time will change accordingly.

[0092] For example, refer to Figure 9This embodiment uses a television as an example to illustrate the video frame control process of a display device box. The display device box receives video data via a wireless network. In the diagram, the box controller decodes the video data according to the current decoding frame rate and determines the video refresh time based on the current decoding frame rate. The box controller determines the time difference between acquiring the current video frame and acquiring the video frame to be sent to the display device for display. Finally, the box controller determines whether to send the current video frame to the television communicator via the box communicator based on the video refresh time and the time difference. The television communicator sends the received video frame to the television controller, which processes the video frame accordingly and then sends it to the display for display.

[0093] In some embodiments, the controller includes a video frame buffer. The video frame buffer is used to buffer a preset number of video frames generated after decoding the video data, and then sequentially sends the buffered video frames to the display device for display. For example, the video frame buffer can buffer 3 frames, 5 frames, or other number of video frames, thereby increasing the buffering time.

[0094] Meanwhile, the video refresh time can be determined by the controller by multiplying the reciprocal of the video output frame rate and the preset frame rate.

[0095] In some embodiments, such as Figure 10 As shown, when the preset frame count is 3 frames, the video frame buffer will queue the 3 video frames it contains and send them to the display device. Assuming the current video frame is the 4th frame outside the video frame buffer, the video frame to be displayed is the 1st frame in the video frame buffer. At this time, the time difference is the time difference between the 1st and 4th video frames. The video frame buffer is the product of the reciprocal of the video output frame rate and 3. In this embodiment, due to the video frame buffer, there is generally a preset frame difference between the current video frame and the video frame to be displayed. Therefore, the video refresh time can be set as the product of the reciprocal of the video output frame rate and the preset frame count. When the time difference is less than this video refresh time, it means the current video frame arrives too quickly, and the video frames in the video frame buffer will not immediately send the 1st frame to the display device for display, thus preventing the current video frame from entering the video frame buffer. In this case, the current video frame is discarded, and the next video frame enters the video frame buffer to queue. Existing frame dropping methods may discard all three frames in the video frame buffer to display the current frame. Therefore, compared to existing frame dropping methods, the video frame control method in this embodiment can further improve the uniformity of frame dropping.

[0096] In some embodiments, the controller also has the function of performing audio-visual synchronization processing on the video data. The controller can decode the video data according to the current decoding frame rate and perform audio-visual synchronization processing on the decoded video data, thereby synchronizing the audio and video of the video data and improving the user's viewing experience.

[0097] Audio-video synchronization can be achieved using either the audio master or video master mode. The goal of audio-video synchronization is to render audio and video with the same predetermined timing at the same point in time. This involves a time reference issue, and two common methods are:

[0098] Audio master: Based on the audio timestamp, video data is synchronized to the audio timestamp.

[0099] Video master: Based on the video timestamp, audio data is synchronized to the video timestamp.

[0100] The main difference between the two methods is whether the reference is video or audio.

[0101] In some embodiments, since the video output frame rate changes in real time, a controller can be used to monitor whether the video output frame rate of the current time period is the same as that of the previous time period. If it is determined that the video output frame rate of the current time period is different from that of the previous time period, the video refresh time is updated according to the video output frame rate of the current time period. This allows for timely adjustment of the video refresh time.

[0102] In this embodiment, the current time period can be 1 second, 500 milliseconds, etc., and monitoring can be implemented by creating a sub-thread. The purpose of monitoring is to adjust the video refresh time more promptly when the video output frame rate changes, thus avoiding uneven frame drops.

[0103] In some embodiments, to reduce system overhead caused by frequent queries, the controller may also receive image display frame rate update information sent by the display device. This image display frame rate update information is generated when the display device detects a difference between the current image display frame rate and the previous image display frame rate.

[0104] Therefore, the controller can update the video refresh time based solely on the image display frame rate update information.

[0105] In this embodiment, the image display frame rate refers to the refresh rate of the image displayed by the display device, where each displayed image is a video frame. The image display frame rate is related to the screen refresh rate of the display device and the video output frame rate of the display device box. A callback function can be set on the display device side. When the image display frame rate of the display device in the current time period is different from that in the previous time period, the callback function is triggered. This allows for more timely adjustment of the video refresh time when the video output frame rate changes, while also reducing system overhead caused by frequent queries.

[0106] For example, refer to Figure 11 The diagram shows the controller and communicator of the display device box. The controller includes a decoding module, an audio-visual synchronization module, a video frame control module, and a video frame buffer. When the display device box receives video data from the network, the decoding module decodes the video data according to the current decoding frame rate. After decoding, the video frames are sent to the audio-visual synchronization module for audio-visual synchronization processing, ensuring that audio and video with the same predetermined time are rendered at the same point in time, thus preventing image and sound misalignment. After audio-visual synchronization processing, the video frame control module determines whether to discard the video frame or send it to the video frame buffer. If it is determined to send it to the video frame buffer, the video frame is sent to the video frame buffer, and the video frames are then sent to the communicator in sequence within the video frame buffer. The communicator then sends the video frame to the display device for display.

[0107] In some embodiments, users may adjust the video playback rate according to their viewing needs. After the user issues a decoding frame rate adjustment command via a remote control or other device, the controller can obtain the decoding frame rate adjustment command and adjust the previous decoding frame rate according to the current decoding frame rate.

[0108] At this point, the controller can set the adjusted previous decoding frame rate as the current decoding frame rate to meet the user's needs.

[0109] In this embodiment, the previous decoding frame rate is relative to the current decoding frame rate; it refers to adjusting the previous decoding frame rate. For example... Figure 12 As shown, Figure 12 An example of one model of a display device box is given. Figure 12 The upper-middle area is the display device box. Circles filled with a left-hand diagonal line within the display device box represent the power switch, while circles without the diagonal line represent indicator lights. In actual use, you need to first turn on the display device box and then use the remote control to activate it again, turning the indicator light green. Figure 12The lower middle area is for the remote control. Decoding frame rate adjustment commands can be obtained from the control device that is paired with the display device box. For example, the user can control the video playback speed through the control device that is paired with the display device box, such as the remote control, thereby adjusting the previous decoding frame rate.

[0110] In some embodiments, the decoding frame rate adjustment instruction includes the video playback speed.

[0111] When the controller receives a decoding frame rate adjustment command, in order to synchronize with the user's desired video playback speed, the controller determines the current decoding frame rate by multiplying the video playback speed by the previous decoding frame rate. Video playback speeds can vary, such as 0.5x, 0.75x, or 2x, and the decoding frame rate will change accordingly with each change in playback speed.

[0112] For example, the following will provide a more detailed explanation of the program execution flow inside the display device box. The display device box includes a decoding module, an audio-visual synchronization module, and a video frame control module. Video frame control uses the system time as a reference and the image refresh time of the display device box as a reference. Based on the time it takes for a video frame to arrive at the video frame control module, a certain algorithm determines whether frame dropping is necessary. First, the meaning of the following control variables will be explained:

[0113] current_time: The system time when a video frame arrives at the video frame control module.

[0114] render_time: The system time when the video frame to be displayed arrives at the video frame control module.

[0115] diff_time: The system time difference between two video frames arriving at the video frame control module.

[0116] `fresh_time`: The refresh interval of the display device's image. This value is calculated from the device's image output frame rate (FR, short for Frame Rate, unit: Hz). `fresh_time` = 1 / FR (unit: seconds). For example, if the device's image output frame rate (FR) is 25Hz, the refresh time is 1 / 25 of a second (40ms), meaning one frame is refreshed and displayed every 40ms. If the device's image output frame rate is 60Hz, the refresh time is 1 / 60 of a second (approximately 16.7ms). When image buffering is used, `fresh_time` is N / FR (N is the number of buffered video frames). For example, when the video frame buffer can buffer N = 3 frames, and the image frame rate (FR) is 25Hz, the video buffering time is 3 / 25 of a second (120ms).

[0117] After receiving a video frame, the video frame control module first saves the system time of the acquired video frame into the variable `current_time`. It then determines whether the frame is the first video frame. If so, it assigns `render_time` to `current_time` and sends the frame to the video display module for display. If the frame is not the first video frame, it calculates the arrival time `current_time` and the system time difference `diff_time` between the frame and the frame to be displayed. If `diff_time` is greater than the video refresh time `fresh_time`, the frame is not dropped; `render_time` is updated to `current_time`, and the frame is sent to the video display module for display. If `diff_time` is less than or equal to the video refresh time `fresh_time`, the frame is dropped, thus completing the display control for this video frame.

[0118] like Figure 13 As shown in the embodiments of this application, a video frame control method is also provided, including:

[0119] Step S601: Obtain the current video frame and the corresponding current decoding frame rate generated by decoding the video data.

[0120] Step S602: Determine the video refresh time based on the current decoding frame rate.

[0121] Step S603: Determine the time difference between acquiring the current video frame and acquiring the video frame to be displayed; wherein, the video frame to be displayed is the video frame to be sent to the display device.

[0122] Step S604: Determine whether to send the current video frame to the display device for display based on the video refresh time and time difference.

[0123] The video frame control method provided in this embodiment acquires the current video frame generated by decoding video data and the corresponding current decoding frame rate. Simultaneously, it determines the video refresh time based on the current decoding frame rate. The controller determines the time difference between acquiring the current video frame and acquiring the video frame to be sent to the display device for display, and determines whether to send the current video frame to the display device for display based on the video refresh time and the time difference. This allows for selective frame dropping based on the relationship between the video refresh time and the time difference, thereby overcoming the problem of uneven frame dropping and intermittent image stuttering during video playback caused by current frame dropping methods.

[0124] In some embodiments, determining the video refresh time based on the current decoding frame rate includes:

[0125] The video output frame rate is determined based on the current decoding frame rate.

[0126] The video refresh time is determined based on the video output frame rate.

[0127] In some embodiments, the controller includes a video frame buffer. The video frame buffer is used to buffer a preset number of video frames generated after decoding the video data, and to send the buffered video frames sequentially to the display device for display.

[0128] Determining the video refresh time based on the video output frame rate includes:

[0129] The video refresh time is determined by multiplying the reciprocal of the video output frame rate and the preset frame rate.

[0130] In some embodiments, determining whether to send the current video frame to a display device for display based on the video refresh time and time difference includes:

[0131] If the time difference is determined to be less than the video refresh time, the current video frame will not be sent to the display device for display.

[0132] If the time difference is determined to be greater than or equal to the video refresh time, the current video frame is sent to the display device for display.

[0133] In some embodiments, before obtaining the current video frame generated by decoding the video data and the corresponding current decoding frame rate, the method further includes:

[0134] Obtain the decoding frame rate adjustment instruction, and adjust the previous decoding frame rate according to the current decoding frame rate.

[0135] The adjusted previous decoding frame rate is set as the current decoding frame rate.

[0136] In some embodiments, the decoding frame rate adjustment instruction includes the video playback speed.

[0137] Adjust the previous decoding frame rate according to the decoding frame rate adjustment instruction, and determine the adjusted previous decoding frame rate as the current decoding frame rate, including:

[0138] The current decoding frame rate is determined by multiplying the video playback speed by the previous decoding frame rate.

[0139] In some embodiments, the video frame control method further includes:

[0140] Monitor whether the video output frame rate of the current time period is the same as that of the previous time period.

[0141] If it is determined that the video output frame rate of the current time period is different from that of the previous time period, the video refresh time is updated according to the video output frame rate of the current time period.

[0142] In some embodiments, the video frame control method further includes:

[0143] Receive image display frame rate update information sent by the display device. The image display frame rate update information is generated when the display device detects that the image display frame rate in the current time period is different from the image display frame rate in the previous time period.

[0144] The video refresh time is updated based on the frame rate information displayed in the image.

[0145] In some embodiments, before obtaining the current video frame generated by decoding the video data and the corresponding current decoding frame rate, the method further includes:

[0146] The video data is decoded based on the current decoding frame rate, and the decoded video data is then processed for audio-visual synchronization.

[0147] The video frame control method implemented in the display device box provided in the above embodiments is similar to... Figures 6 to 9 The functions implemented by the display device box provided in the illustrated embodiment correspond to those of the previous embodiment, and will not be described in detail here.

[0148] This application also provides a computer-readable storage medium storing computer-executable instructions. When the processor executes the computer-executable instructions, it implements the video frame control method executed by the display device box in the above embodiments.

[0149] Additionally, this application embodiment may also provide a computer product that can implement the video frame control method executed by the display device box in the above embodiments.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0151] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the embodiments and various different variations of embodiments suitable for specific application considerations.

Claims

1. A display device box, characterized by, The method comprises: a communicator for establishing a communication link between the controller and the display device; a controller connected to the communicator and configured to: obtain a decoding frame rate adjustment instruction, and adjust a previous decoding frame rate of a current decoding frame rate according to the decoding frame rate adjustment instruction; determine the adjusted previous decoding frame rate as the current decoding frame rate; the decoding frame rate adjustment instruction comprises a video playback speed; and the controller is specifically configured to determine a product of the video playback speed and the previous decoding frame rate as the current decoding frame rate; obtain a current video frame generated by decoding video data and a corresponding current decoding frame rate; determine a video output frame rate according to the current decoding frame rate; determine a video refresh time according to the video output frame rate, wherein the video refresh time is a product of an inverse of the video output frame rate and a preset number of frames of a video frame buffer; determine a time difference between obtaining the current video frame and obtaining a to-be-displayed video frame, wherein the to-be-displayed video frame is a video frame to be sent to the display device; determine whether to send the current video frame to the display device for display according to the video refresh time and the time difference.

2. The display device box of claim 1, wherein, The controller is provided with a video frame buffer; the video frame buffer is used to buffer a preset number of video frames generated after decoding video data, and sequentially send the buffered video frames to the display device for display.

3. The display device box of claim 2, wherein, The controller is specifically configured to: if it is determined that the time difference is less than the video refresh time, the current video frame is not sent to the display device for display; if it is determined that the time difference is greater than or equal to the video refresh time, the current video frame is sent to the display device for display.

4. The display device box of claim 1, wherein, The controller is further configured to: monitor whether the video output frame rate of a current period is the same as the video output frame rate of a previous period; if it is determined that the video output frame rate of the current period is different from the video output frame rate of the previous period, update the video refresh time according to the video output frame rate of the current period.

5. The display device box of claim 1, wherein, The controller is further configured to: receive image display frame rate update information sent by the display device; the image display frame rate update information is generated when the display device monitors that the image display frame rate of a current period of the display device is different from the image display frame rate of a previous period; update the video refresh time according to the image display frame rate update information.

6. The display device box of claim 1, wherein, The controller is further configured to: decode the video data according to the current decoding frame rate, and perform audio-visual synchronization processing on the decoded video data.

7. A video frame control method characterized by comprising: The method comprises: obtain a current video frame generated by decoding video data and a corresponding current decoding frame rate; determine a video output frame rate according to the current decoding frame rate; determine a video refresh time according to the video output frame rate, wherein the video refresh time is a product of an inverse of the video output frame rate and a preset number of frames; determine a time difference between obtaining the current video frame and obtaining a to-be-displayed video frame to be sent to the display device; determine whether to send the current video frame to the display device for display according to the video refresh time and the time difference. Before the current video frame generated by decoding the video data and the corresponding current decoding frame rate are acquired, the method further comprises: acquiring a decoding frame rate adjustment instruction, and adjusting a previous decoding frame rate of the current decoding frame rate according to the decoding frame rate adjustment instruction; determining the adjusted previous decoding frame rate as the current decoding frame rate; the decoding frame rate adjustment instruction comprises a video playing speed; the adjusting of the previous decoding frame rate of the current decoding frame rate according to the decoding frame rate adjustment instruction and the determining of the adjusted previous decoding frame rate as the current decoding frame rate comprise: determining a product of the video playing speed and the previous decoding frame rate as the current decoding frame rate.

Images