Video playing method, device and equipment and readable storage medium

Abstract

The invention relates to a video playing method and device, equipment and a readable storage medium, and the method comprises the steps: predicting the time when a to-be-rendered video frame is inserted into a frame buffer region under the condition that the to-be-rendered video frame does not exist in the frame buffer region after a vertical synchronization signal is received; and whether the video frame to be rendered can come before consumption of the rendering thread to execute the corresponding video frame rendering strategy is judged, so that the opportunity of video frame rendering in the refresh cycle is more accurately controlled, the risk that the video frame in the frame buffer area is not consumed and is covered by the newly inserted video frame is reduced, the video frame rendering efficiency is improved, and the user experience is improved. And the problems of video playing frame loss and lagging are reduced. Based on the video playing method, the smooth playing of the video in the video playing scene with the refresh rate set to be the same as the video frame rate is realized, and the power consumption of equipment is reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of multimedia technology, and in particular to a video playback method, device, equipment and readable storage medium. Background Art

[0002] Vertical sync technology eliminates screen tearing by matching the screen refresh rate with the video frame rate. When the video frame rate and screen refresh rate are consistent, each video frame can be displayed within one screen refresh cycle, so that each video frame is fully displayed on the screen, avoiding screen tearing.

[0003] Since the video frame rate may fluctuate in actual applications, in order to ensure that each video frame can be displayed within one screen refresh cycle, the screen refresh rate is usually set to twice the video frame rate or its multiple. For example, if the video frame rate is 30fps, the screen refresh rate can be set to 60Hz through double refresh, so that each video frame will be displayed within two consecutive screen refresh cycles to achieve smooth video playback.

[0004] However, the above method of setting the screen refresh rate to a multiple of the video frame rate will result in higher power consumption, while setting the screen refresh rate to the same value as the video frame rate is prone to problems such as video playback stuttering and unstable display, affecting the smoothness of the video and the user's viewing experience. Summary of the invention

[0005] In view of this, in order to solve the above technical problems, the present disclosure provides a video playback method, device, equipment and storage medium.

[0006] According to a first aspect of an embodiment of the present disclosure, a video playback method is provided, the method comprising:

[0007] Receiving a vertical synchronization signal, where the vertical synchronization signal is used to instruct a main thread of the application to render a first video frame to be rendered in a frame buffer;

[0008] In response to the first video frame not existing in the frame buffer, predicting an insertion time of the first video frame into the frame buffer according to timestamp information of the second video frame being inserted into the frame buffer and a frame rate of the corresponding video; the second video frame being a previous frame of the first video frame;

[0009] Execute rendering processing in the refresh cycle according to the relationship between the insertion time and the scheduling time of the rendering thread in the refresh cycle indicated by the vertical synchronization signal;

[0010] A rendering result corresponding to the rendering process is obtained, and the rendering result is displayed in a video playback area of ​​the application.

[0011] Optionally, after receiving the vertical synchronization signal, the method further includes:

[0012] Querying whether a message queue associated with the main thread includes a synchronization message to be processed, wherein the synchronization message is used to indicate that a video frame to be rendered has been inserted into the frame buffer;

[0013] If not, determining that the first video frame does not exist in the frame buffer; or,

[0014] Monitoring whether an update operation of inserting a video frame occurs in the frame buffer;

[0015] In response to the update operation not occurring before receiving a vertical synchronization signal, it is determined that the first video frame does not exist in the frame buffer.

[0016] Optionally, predicting the insertion time of the first video frame into the frame buffer according to the timestamp information of the second video frame being inserted into the frame buffer and the frame rate of the corresponding video includes:

[0017] Determine a time interval between the first video frame and the second video frame according to a frame rate of a video to which the second video frame belongs;

[0018] The insertion time is determined according to the timestamp information of the second video frame and the time interval.

[0019] Optionally, performing rendering processing in the refresh cycle according to a relationship between the insertion time and a scheduling time of a rendering thread in the refresh cycle indicated by the vertical synchronization signal includes:

[0020] In response to the insertion time being earlier than the scheduling time, setting a transition state to indicate that the rendering thread is ready to perform a rendering operation;

[0021] After the first video frame is inserted into the frame buffer, the rendering thread obtains the first video frame from the frame buffer and performs rendering processing on the first video frame.

[0022] Optionally, the method further comprises:

[0023] In response to the insertion time being later than the scheduling time, sending an empty frame command to the rendering thread;

[0024] After receiving the empty frame command, the rendering thread skips the rendering processing operation on the video frame view in the refresh cycle.

[0025] Optionally, after predicting the insertion time of the first video frame, the method further includes:

[0026] Obtaining a historical scheduling time of a rendering thread in a refresh cycle indicated by at least one historical signal before the vertical synchronization signal;

[0027] The scheduling time of the rendering thread within the refresh period indicated by the vertical synchronization signal is predicted according to the historical scheduling time.

[0028] Optionally, predicting the scheduling time of the rendering thread within the refresh period indicated by the vertical synchronization signal includes:

[0029] Determine the scheduling time interval between adjacent historical signals;

[0030] Determining an occurrence period of the rendering thread according to the scheduling time interval;

[0031] The scheduling time of the rendering thread in the refresh cycle indicated by the vertical synchronization signal is determined according to the generation cycle of the rendering thread and the historical scheduling time of the rendering thread in a previous historical refresh cycle of the refresh cycle.

[0032] According to a second aspect of an embodiment of the present disclosure, a video playback device is provided, the device comprising:

[0033] A signal receiving module, used for receiving a vertical synchronization signal, wherein the vertical synchronization signal is used for instructing a main thread of the application to render a first video frame to be rendered in a frame buffer;

[0034] an insertion time prediction module, configured to predict, in response to the absence of the first video frame in the frame buffer, an insertion time of the first video frame into the frame buffer according to timestamp information of the second video frame being inserted into the frame buffer and a frame rate of the corresponding video; the second video frame being a previous frame of the first video frame;

[0035] A rendering processing module, configured to perform rendering processing in the refresh cycle according to a relationship between the insertion time and a scheduling time of a rendering thread in the refresh cycle indicated by the vertical synchronization signal;

[0036] The display module is used to obtain the rendering result corresponding to the rendering process and display the rendering result in the video playback area of ​​the application.

[0037] Optionally, the device further comprises:

[0038] Querying whether a message queue associated with the main thread includes a synchronization message to be processed, wherein the synchronization message is used to indicate that a video frame to be rendered has been inserted into the frame buffer;

[0039] If not, determining that the first video frame does not exist in the frame buffer; or,

[0040] Monitoring whether an update operation of inserting a video frame occurs in the frame buffer;

[0041] In response to the update operation not occurring before receiving a vertical synchronization signal, it is determined that the first video frame does not exist in the frame buffer.

[0042] Optionally, the insertion time prediction module is specifically used for:

[0043] Determine a time interval between the first video frame and the second video frame according to a frame rate of a video to which the second video frame belongs;

[0044] The insertion time is determined according to the timestamp information of the second video frame and the time interval.

[0045] Optionally, the rendering processing module is specifically used for:

[0046] In response to the insertion time being earlier than the scheduling time, setting a transition state to indicate that the rendering thread is ready to perform a rendering operation;

[0047] After the first video frame is inserted into the frame buffer, the rendering thread obtains the first video frame from the frame buffer and performs rendering processing on the first video frame.

[0048] Optionally, the rendering processing module is specifically used for:

[0049] In response to the insertion time being later than the scheduling time, sending an empty frame command to the rendering thread;

[0050] After receiving the empty frame command, the rendering thread skips the rendering processing operation on the video frame view in the refresh cycle.

[0051] Optionally, the device further comprises:

[0052] An acquisition module, used for acquiring a historical scheduling time of a rendering thread in a refresh cycle indicated by at least one historical signal before the vertical synchronization signal;

[0053] The scheduling time prediction module is used to predict the scheduling time of the rendering thread within the refresh period indicated by the vertical synchronization signal according to the historical scheduling time.

[0054] Optionally, the scheduling time prediction module is specifically used to:

[0055] Determine the scheduling time interval between adjacent historical signals;

[0056] Determining an occurrence period of the rendering thread according to the scheduling time interval;

[0057] The scheduling time of the rendering thread in the refresh cycle indicated by the vertical synchronization signal is determined according to the generation cycle of the rendering thread and the historical scheduling time of the rendering thread in a previous historical refresh cycle of the refresh cycle.

[0058] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including: a processor and a memory; the memory is used to store a computer program; the processor is used to call the computer program to implement the above-mentioned video playback method.

[0059] According to a fourth aspect of an embodiment of the present disclosure, a readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned video playback method is implemented.

[0060] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0061] In the present disclosure, in the case where there is no video frame to be rendered in the frame buffer after receiving the vertical synchronization signal, by predicting the timing of inserting the video frame to be rendered into the frame buffer, and judging whether the video frame to be rendered can arrive before the rendering thread consumes it to execute the corresponding video frame rendering strategy, it is achieved to more accurately control the timing of video frame rendering within the refresh cycle, reduce the risk of the video frame in the frame buffer not being consumed and being overwritten by the newly inserted video frame, improve the efficiency of video frame rendering, and reduce the occurrence of frame loss and freeze in video playback. Based on this video playback method, smooth video playback is achieved in the video playback scenario where the refresh rate is set to the same as the video frame rate, and the power consumption of the device is reduced.

[0062] It should be understood that the above general description and the following detailed description are only exemplary and explanatory and cannot limit the present disclosure. In addition, any embodiment of the present disclosure does not need to achieve all the above effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0064] Figure 1 is a schematic diagram showing component interactions involved in video playback according to an exemplary embodiment;

[0065] Figure 2 is a flowchart of a video playback method according to an exemplary embodiment;

[0066] Figure 3 is a flow chart showing a method of predicting a video frame to be rendered and inserting it into a frame buffer according to an exemplary embodiment;

[0067] Figure 4 is a schematic diagram showing a method of performing a video frame rendering process in a current refresh cycle according to an exemplary embodiment;

[0068] Figure 5 is a flowchart showing a method of predicting the scheduling time of a rendering thread in a current refresh cycle according to an exemplary embodiment;

[0069] Figure 6 is a flowchart of a specific implementation method for predicting the scheduling time of a rendering thread in a current refresh cycle according to an exemplary embodiment;

[0070] Figure 7 It is a schematic diagram of video frame coverage and synchronous rendering in a refresh cycle in a scenario where the screen refresh rate and the video frame rate are set to be the same in the related art;

[0071] Figure 8 is a flowchart of a video playback in a scenario where a 30fps video is played based on a 30Hz screen refresh rate according to an exemplary embodiment;

[0072] Fig. 9 is a structural schematic diagram of a video playback device according to an exemplary embodiment;

[0073] Fig.10 The present invention is a block diagram of a terminal device according to an exemplary embodiment. DETAILED DESCRIPTION

[0074] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0075] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0076] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first classification threshold may also be referred to as the second classification threshold, and similarly, the second classification threshold may also be referred to as the first classification threshold. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0077] Currently, applications mainly use TextureView and surfaceView to implement video playback functions. Among them, the application is installed in an electronic device, such as Figure 1 As shown, the video playback of the application program interface on the screen of the electronic device based on the TextureView technology mainly involves the interaction between the video decoder, the view control SurfaceTexture under the TextureView technology, the display controller, the rendering component, and the surfaceFlinger system service of the electronic device.

[0078] Among them, the video decoder is used to decode the video source to be played into the original video frame data; the view control Surface Texture provides a frame buffer, which is used to store the video frame data to be rendered so that the rendering thread can obtain the video frame and perform rendering processing; the display controller of the electronic device sends a Vsync signal at a fixed frequency to notify the main thread to start rendering the next video frame; the rendering component is used to render the video frame and submit the rendering result to the hardware buffer; the SurfaceFlinger system service is used to synthesize the rendering result in the hardware buffer with other view surfaces of the screen, and control the display of the synthesized result in the video playback area of ​​the application interface of the electronic device.

[0079] See also Figure 1 As shown in the figure, taking the rendering and display process of a video frame as an example, the interaction process between the various components associated with the application is as follows:

[0080] After determining the target video source that the application needs to play, the application obtains the decoded video frame image by calling the video decoder, and inserts the video frames into the frame buffer of the control surface Texture frame by frame according to the video frame interval corresponding to the target video source frame rate;

[0081] After receiving the vertical synchronization signal, the main thread of the application can notify the rendering thread to perform rendering processing in the form of a rendering command or signal; the vertical synchronization signal is periodically sent to the main thread of the application by the display controller of the device according to the screen refresh rate;

[0082] When receiving the rendering signal, the rendering thread obtains the video frame ready for rendering from the frame buffer, calls the rendering component to implement the rendering of the video frame, and sends the rendering result to the hardware image buffer of the device;

[0083] After receiving the vertical synchronization signal from the display controller, the surfaceFlinger system service synthesizes the rendering result of the hardware image buffer with other view surfaces on the screen and displays it on the screen.

[0084] By repeating the above processing process for each video frame, video playback in the video playback area of ​​the application program interface on the electronic device is achieved.

[0085] Based on the above principle of using TextureView technology to achieve video playback, in related technologies, in order to ensure the smoothness of video playback, a video with a frame rate of 30fps is usually played at a device screen refresh rate of 60Hz. However, this method will bring about greater device power consumption; if a 30Hz screen refresh rate is used to play the 30fps video, there will be problems such as frame loss and playback freeze, which will affect the user experience.

[0086] To solve the above technical problems, the present disclosure provides a video playback method, which can be applied to but is not limited to applications supporting video playback functions installed on electronic devices such as mobile phones and tablets. The application uses a vertical synchronization signal to synchronize application rendering with screen refresh during the process of implementing the video playback function.

[0087] Figure 2 is a flowchart of a video playing method according to an exemplary embodiment. Figure 2 As shown, the method disclosed herein may include the following steps:

[0088] S201, receiving a vertical synchronization signal, where the vertical synchronization signal is used to instruct a main thread of an application to render a first video frame to be rendered in a frame buffer;

[0089] The vertical synchronization signal is a synchronization signal used in display devices to coordinate the timing of graphics rendering and screen display refresh to ensure the stability and smoothness of the image during the display process. The signal indicates when the display device starts the next screen refresh cycle. When the vertical synchronization signal is received, a new screen refresh cycle begins, and the thread associated with the application starts the next image rendering and screen display of the rendered image.

[0090] The vertical synchronization signal is divided into two parts in the Android system: Vsync-app and Vsync-sf. Among them, Vsync-app is for vertical synchronization at the application level. When the Vsync-app signal arrives, the application can start submitting image data to the rendering component for rendering. Vsync-sf is for vertical synchronization at the interface system level. When the Vsync-sf signal arrives, the SurfaceFlinger system service starts to mix and synthesize the image data of each window and send it to the display for display. In the embodiment of the present disclosure, the received vertical synchronization signal is the Vsync-app signal.

[0091] The vertical synchronization signal is periodically sent by the display controller of the electronic device on which the application is installed, and the sending period is the screen refresh rate of the device. The screen refresh rate of the device refers to the number of times the screen is refreshed per second, indicating the number of times the screen can update the display within one second. For example, the screen refresh rate of the device is 30Hz, which means that 30 Vsync-app signals are sent within one second, and the average sending time interval between the signals is 1 / 30ms.

[0092] The frame buffer refers to the buffer provided by the surface Texture associated with TextureView, which is used to store the video frames to be rendered. The application inserts the decoded video frames into the frame buffer for the rendering thread to consume the inserted video frames.

[0093] Video consists of a series of still image frames, and video playback is achieved by quickly displaying the image frames in the application interface. When the main thread of the application receives the vertical synchronization signal Vsync-app, it indicates that it has entered a new screen refresh cycle. During this refresh cycle, the next image frame of the video needs to be rendered, that is, the main thread needs to perform video frame rendering. In the Android system, when the main thread receives the Vsync-app signal, it can trigger the call of the doframe function to handle operations related to the application interface update, and notify the rendering thread to render the first video frame to be rendered in the frame buffer.

[0094] S202, in response to the first video frame not existing in the frame buffer, predicting an insertion time of the first video frame into the frame buffer according to timestamp information of the second video frame being inserted into the frame buffer and a frame rate of the corresponding video; the second video frame is a previous frame of the first video frame;

[0095] The frame rate of a video refers to the number of video frames played per second, usually expressed in fps (Frames Per Second), which indicates the number of video frames played in one second. A video consists of a series of still image frames, and the frame rate determines how fast these image frames are played continuously, thereby presenting a dynamic video effect. Common video frame rates include 24fps, 30fps, 60fps, etc.

[0096] The decoded video frames are inserted into the frame buffer at a fixed time interval, which depends on the video frame rate. For example, for a video with a video frame rate of 30fps, the time interval is 1 / 30ms.

[0097] After receiving the vertical synchronization signal, if the application determines that there is no video frame to be rendered in the frame buffer, the general practice is to process it as an empty frame. In the current refresh cycle, the rendering thread does not perform rendering processing operations on the video frame view, and performs normal rendering processing on other views displayed on the screen.

[0098] In the disclosed embodiment, based on the fact that the video frames inserted into the frame buffer are inserted in the order of the decoded video frames, if there is no video frame to be rendered in the frame buffer, the insertion time of the video frame to be rendered into the frame buffer can be predicted based on the timestamp information of the last time the video frame was inserted into the frame buffer and the insertion interval between each frame of the video. The insertion interval between each frame can be determined according to the video frame rate.

[0099] For example, the sixth video frame (hereinafter referred to as "video frame 6") needs to be rendered within the refresh cycle indicated by the vertical synchronization signal, but video frame 6 is not included in the current frame buffer. The time when video frame 6 is inserted into the frame buffer can be predicted based on the previous frame of video frame 6, that is, the timestamp information of video frame 5 inserted into the frame buffer and the insertion interval of the video frame.

[0100] S203, performing rendering processing in the refresh cycle according to the relationship between the insertion time and the scheduling time of the rendering thread in the refresh cycle indicated by the vertical synchronization signal;

[0101] The refresh cycle indicated by the vertical synchronization signal refers to the processing cycle between the vertical synchronization signal currently received by the main thread and the reception of the next vertical synchronization signal. During this refresh cycle, the main thread of the application interacts with other threads to implement the rendering processing of a frame of video, and each thread is called according to the set execution order and time point.

[0102] The rendering thread is used to request the first video frame to be rendered in the frame buffer and perform rendering processing on the video frame. For example, the rendering thread can call the functions provided by the graphics rendering tool OpenGL to create, transform and render the video frame. The scheduling time of the above rendering thread refers to the time point when the rendering thread requests the first video frame to be rendered from the frame buffer.

[0103] When there is no video frame to be rendered in the current frame buffer, when the estimated insertion time of the video frame to be rendered into the frame buffer is obtained, by comparing the estimated insertion time with the scheduling time of the rendering thread, it can be determined whether the video frame that has not yet been inserted into the frame buffer can arrive before the rendering thread consumes the frame buffer, thereby determining the corresponding rendering processing method based on different insertion situations to reduce the occurrence of problems such as frame loss and freeze in video playback.

[0104] S204, obtaining a rendering result corresponding to the rendering process, and displaying the rendering result in a video playback area of ​​the application.

[0105] In the current refresh cycle indicated by the received vertical synchronization signal, the rendering result within the refresh cycle is obtained, and the rendering result can be stored in the specified hardware image buffer, so that the rendering result can be synthesized with other view surfaces on the program interface in the refresh cycle used to display the rendering result, and the synthesized result can be sent to the specified area of ​​the application program interface for display.

[0106] In the present disclosure, in the case where there is no video frame to be rendered in the frame buffer after receiving the vertical synchronization signal, the timing of inserting the video frame to be rendered into the frame buffer is predicted. By judging whether the video frame to be rendered can arrive before the rendering thread consumes it, the corresponding video frame rendering strategy is adopted to more accurately determine the video frames that need to be rendered in the current refresh cycle, reduce the risk of the video frames in the frame buffer not being consumed and being overwritten by the newly inserted video frames, and by accurately controlling the rendering timing, reduce the skipping of the video frame view rendering processing operation in the refresh cycle, and postpone the timing of the delayed consumption of the video frame. Based on this video playback method, smooth video playback is achieved in the video playback scenario where the refresh rate is set to the same as the video frame rate, and the power consumption of the device is reduced.

[0107] In some embodiments, after the above step S201, the above video playback method may include the following steps to determine whether the first video frame exists in the frame buffer, the steps including:

[0108] Querying whether a message queue associated with the main thread includes a synchronization message to be processed, wherein the synchronization message is used to indicate that a video frame to be rendered has been inserted into the frame buffer;

[0109] If so, it is determined that the first video frame exists in the frame buffer; if not, it is determined that the first video frame does not exist in the frame buffer.

[0110] Among them, the message queue is a cache area associated with the main thread of the application and is used to store interaction messages between other threads and the main thread. After the application inserts each decoded video frame into the frame buffer, it can send a synchronization message to the main thread to notify the main thread that there is a video frame to be rendered, so that the main thread can render the inserted video frame after receiving the vertical synchronization signal.

[0111] Therefore, after receiving the vertical synchronization signal, the main thread can query whether the local message queue includes the synchronization message to be processed. If so, it means that there is a newly inserted video frame that needs to be rendered, and the video frame rendering processing method of the relevant technology can be used to perform rendering and display operations on the video frame; if the synchronization message does not exist, it means that no new video frame to be rendered is inserted into the frame buffer before the vertical synchronization signal is received, that is, there is no video frame to be rendered in the frame buffer.

[0112] In the embodiment of the present disclosure, the main thread is notified of the existence of video frames to be rendered by sending a synchronization message, so that the main thread can determine the video frames to be rendered based on the synchronization message after receiving the vertical synchronization signal, thereby avoiding waiting for the rendering and update operations of the video frames directly in the main thread, reducing the response pressure of the main thread, achieving more precise synchronization and rendering control, and improving the video playback effect and picture quality.

[0113] In some embodiments, after the above step S201, the above video playback method may also determine whether the first video frame exists in the frame buffer in the following manner, which may include the following steps:

[0114] Monitoring whether an update operation of inserting a video frame occurs in the frame buffer;

[0115] In response to the update operation not occurring before receiving a vertical synchronization signal, it is determined that the first video frame does not exist in the frame buffer.

[0116] That is, the main thread or other threads of the application program may be set to monitor the update processing of the frame buffer by registering a corresponding event listener or using a callback function to monitor the update operation of the frame buffer.

[0117] After the rendering thread consumes the frame buffer in the refresh cycle indicated by the last vertical synchronization signal, if an update operation of inserting a video frame occurs before the main thread receives the vertical synchronization signal, it means that a new video frame to be rendered is inserted into the frame buffer. If the update operation does not occur, it means that there is no video frame to be rendered in the frame buffer. After receiving the vertical synchronization signal, the main thread can determine whether the first video frame exists according to the feedback update monitoring result.

[0118] In the embodiment of the present disclosure, by monitoring the update operation of the frame buffer to determine whether there are video frames that need to be rendered, new video frames to be rendered can be obtained in a timely manner, avoiding the main thread from constantly polling or blocking waiting for new video frames, saving CPU resources, and enabling the main thread to process other tasks more efficiently.

[0119] In some embodiments, Figure 3 As shown, the prediction of the insertion time of the first video frame into the frame buffer according to the timestamp information of the second video frame being inserted into the frame buffer and the frame rate of the corresponding video in the above step S202 can be implemented as follows:

[0120] S301, determining a time interval between the first video frame and the second video frame according to a frame rate of a video to which the second video frame belongs;

[0121] S302: Determine the insertion time according to the timestamp information of the second video frame and the time interval.

[0122] The first video frame and the second video frame are two adjacent video frames, and the time interval depends on the frame rate of the video. The frame rate of the video represents the number of frames displayed per second, and the value of the time interval is the reciprocal of the frame rate of the video. For example, if the frame rate of the video is 30fps, then the interval between two adjacent video frames is 1 / 30 second, that is, about 33.33 milliseconds.

[0123] The timestamp information of each video frame when it is inserted into the frame buffer can be recorded in advance. Based on the fact that the video frames are inserted into the frame buffer at the time interval of each frame, the insertion time of the first video frame into the frame buffer can be predicted based on the timestamp information of the video frame most recently inserted into the frame buffer, which is the second video frame mentioned above, and the time interval.

[0124] For example, the first video frame is video frame 8, and the second video frame is the previous frame of video frame 8, that is, video frame 7. Assuming that the time when video frame 7 is inserted into the frame buffer is 199.98ms after the video starts playing, and the video frame rate is known to be 30fps, it is estimated that the time when video frame 8 is inserted into the frame buffer is 233.31ms after the video starts playing.

[0125] In the embodiment of the present disclosure, in the case where there are no video frames to be rendered in the frame buffer within the current refresh cycle, the time when the video frame to be rendered is inserted into the frame buffer is predicted based on the timestamp information of the video frame most recently inserted into the frame buffer and the frame interval between each video frame, thereby achieving precise control of the rendering timing within the current refresh cycle based on the predicted insertion time, thereby reducing the risk of frame loss and freezes in video playback.

[0126] See also Figure 4 As shown, in some embodiments, the aforementioned step S203 of performing the rendering process in the refresh cycle according to the relationship between the insertion time and the scheduling time of the rendering thread in the refresh cycle indicated by the vertical synchronization signal may include the following implementation steps:

[0127] S401, in response to the insertion time being earlier than the scheduling time, setting a transition state to indicate that the rendering thread is ready to perform a rendering operation;

[0128] The setting of the transition state refers to setting a notification state, which is used to indicate that the rendering thread is ready to perform a rendering operation. The state can be implemented by a synchronization mechanism between threads, such as using a conditional variable or a semaphore, etc. When the rendering thread detects the state, it is ready to perform a video frame rendering operation. For example, in an embodiment of the present disclosure, when the main thread detects that the insertion time is earlier than the scheduling time, it can send a rendering signal or an indication message to the rendering thread to indicate that the rendering thread is ready to perform a rendering operation.

[0129] S402: After the first video frame is inserted into the frame buffer, the rendering thread obtains the first video frame from the frame buffer and performs rendering processing on the first video frame.

[0130] That is, after obtaining the predicted insertion time of the video frame to be rendered into the frame buffer and the scheduling time of the rendering thread in the current refresh cycle indicated by the current vertical signal, by comparing the insertion time and the scheduling time, it can be determined whether the video frame to be rendered can be inserted into the buffer between the frame buffers consumed by the rendering thread.

[0131] If the insertion time is earlier than the scheduling time, it means that the first video frame to be rendered will be inserted into the frame buffer before the rendering thread obtains the video frame to be rendered from the frame buffer, so that when the rendering thread is scheduled, the first video frame to be rendered already exists in the frame buffer, and the rendering thread can obtain the first video frame to be rendered from the frame buffer and render the first video frame within the current refresh cycle.

[0132] Therefore, if the insertion time is earlier than the scheduling time, the main thread can set the transition state to make the rendering thread ready to perform the video frame rendering operation, and temporarily obtain the first video frame of the frame buffer at the scheduling time of the rendering thread, and perform video frame rendering processing.

[0133] In the disclosed embodiment, by comparing the predicted insertion time of the video frame to be rendered into the frame buffer and the scheduling time of the rendering thread in the current refresh cycle, it is determined whether the video frame to be rendered can be timely acquired and the rendering process can be performed in the current refresh cycle, thereby achieving precise control of the video frame rendering, reducing the idle time of the rendering thread, improving the rendering efficiency, reducing the risk of the video frame to be rendered not being consumed and being overwritten by the newly inserted video frame, and improving the smoothness of the video playback.

[0134] In some embodiments, with respect to the aforementioned step S203 of performing the rendering process in the refresh cycle, in response to the insertion time being later than the scheduling time, the rendering process may further include the steps of:

[0135] The main thread sends an empty frame command to the rendering thread; the empty frame command is used to indicate that there is no video frame that needs to be rendered in the current refresh cycle; after receiving the empty frame command, the rendering thread skips the rendering processing operation on the video frame view in the refresh cycle.

[0136] When the insertion of the video frame into the frame buffer is later than the scheduling time of the rendering thread, when the rendering thread requests the frame buffer for the video frame to be rendered, there is still no video frame to be rendered in the frame buffer at this time, then the rendering thread cannot obtain the video frame to be rendered in time within the current refresh cycle.

[0137] The main thread can control the rendering thread through an empty frame command or signal, so that the rendering thread skips the rendering processing operation of the video frame view in the current refresh cycle. That is, after receiving the empty frame command, the rendering thread will suspend the rendering processing of the video frame view in the current refresh cycle, and at the same time, the rendering thread performs normal rendering operations on other views displayed by the application interface on the electronic device screen.

[0138] When the rendering processing operation on the video frame view in the current refresh cycle is skipped, the rendering result of the previous video frame can be used as the rendering result of the current refresh cycle, or the rendering result of the default setting can be determined as the rendering result of the current refresh cycle.

[0139] In the embodiment of the present disclosure, in the case where the video frame to be rendered cannot be obtained in time within the current refresh cycle, the performance burden of the rendering thread is reduced by skipping the rendering processing of the video frame view within the current refresh cycle, and the video smoothness can be quickly restored.

[0140] In some embodiments, see Figure 5 As shown, after predicting the insertion time of the first video frame in the above step S202, the video playback method may further include the following steps to predict the scheduling time of the rendering thread within the refresh cycle indicated by the vertical synchronization signal:

[0141] S501, obtaining a historical scheduling time of a rendering thread in a refresh cycle indicated by at least one historical signal before the vertical synchronization signal;

[0142] That is, for at least one historical signal before the currently received vertical synchronization signal, the time point at which the rendering thread requests the frame buffer for the video frame to be rendered in the refresh cycle indicated by each historical signal is obtained, and it is determined as the historical scheduling time of the rendering thread in the refresh cycle.

[0143] S502: predicting the scheduling time of a rendering thread within a refresh period indicated by the vertical synchronization signal according to the historical scheduling time.

[0144] According to the historical scheduling time corresponding to each historical signal, the occurrence cycle of the scheduling time of the rendering thread can be determined, so that the scheduling time of the rendering thread in the refresh cycle indicated by the current vertical synchronization signal can be predicted based on the occurrence cycle.

[0145] In the case of obtaining the historical scheduling time of the rendering thread in the refresh cycle indicated by the previous historical signal of the current vertical synchronization signal, the scheduling time of the rendering thread in the refresh cycle indicated by the current vertical synchronization signal can be predicted based on the historical scheduling time corresponding to the previous historical signal and the sending period of the vertical synchronization signal. The sending period of the vertical synchronization signal can be taken as the inverse of the screen refresh rate.

[0146] In the disclosed embodiments, the scheduling time of the rendering thread in the current refresh cycle is predicted by observing the occurrence pattern of the scheduling time of the rendering thread in the historical refresh cycle, thereby improving the prediction accuracy of the scheduling time of the rendering thread. Based on the predicted scheduling time, it is determined whether the video frame to be rendered can be obtained in time in the current refresh cycle, which helps to reduce the occurrence of video playback freezes, frame drops and the like.

[0147] In some embodiments, when obtaining historical scheduling times corresponding to more than two historical signals, the historical signals are adjacent to each other. For the aforementioned step S502, the scheduling time of the rendering thread in the refresh cycle indicated by the vertical synchronization signal is predicted, such as Figure 6 As shown, this can be achieved as follows:

[0148] S601, determining a scheduling time interval between adjacent historical signals;

[0149] That is, for a plurality of adjacent historical signals, the scheduling time interval corresponding to the group of adjacent historical signals can be determined according to the historical scheduling time of the rendering thread within the refresh cycle indicated by each group of adjacent historical signals;

[0150] S602, determining an occurrence period of the rendering thread according to the scheduling time interval;

[0151] For the multiple scheduling time intervals obtained, the occurrence cycle of the rendering thread can be determined by calculating the average value; alternatively, data cleaning can be performed on the multiple scheduling time intervals, such as removing an abnormal value, removing a maximum value and a minimum value, and the occurrence cycle can be determined based on the multiple scheduling time intervals after cleaning.

[0152] S603: Determine the scheduling time of the rendering thread in the refresh cycle indicated by the vertical synchronization signal according to the generation cycle of the rendering thread and the historical scheduling time of the rendering thread in the previous historical refresh cycle of the refresh cycle.

[0153] For example, the current vertical synchronization signal is Vsync-app signal 10, and the first five vertical synchronization signals of the signal are Vsync-app signal 5 to Vsync-app signal 9 respectively. Then, four scheduling time intervals can be determined according to the historical scheduling time of the rendering thread in the five historical refresh cycles indicated by Vsync-app signal 5 to Vsync-app signal 9; the average value of the four scheduling time intervals is determined as the occurrence cycle of the rendering thread; the occurrence cycle is superimposed on the historical scheduling time of the rendering thread in the historical refresh cycle indicated by the Vsync-app signal 9, and the superposition result can be used as the scheduling time of the rendering thread in the refresh cycle indicated by the Vsync-app signal 10.

[0154] In an embodiment of the present disclosure, the occurrence cycle of the rendering thread is determined according to the interval of the historical scheduling time of the rendering thread in multiple historical refresh cycles, and the scheduling time of the rendering thread in the current refresh cycle is predicted based on the occurrence cycle, taking into account the fluctuation of the scheduling of the rendering thread in the refresh cycle during the actual video playback process, so that the predicted scheduling time of the rendering thread is more accurate.

[0155] In order to enable those skilled in the art to more clearly understand the video playback method provided by the present disclosure, the method of the present disclosure is exemplified below by taking a mobile phone screen with a refresh rate of 30 Hz and playing a video with a frame rate of 30 fps in an application as an example.

[0156] In the actual screen refresh rate setting, due to the limitation of device hardware, the refresh rate of the mobile phone cannot be strictly guaranteed to be 30hz, and it may be a slightly larger value such as about 30.4hz. Taking the screen refresh rate of the mobile phone as 30.4hz as an example, the periodic interval of the Vsync-app signal sent by the display controller of the mobile phone is 32.88ms, and the inter-frame interval of the 30fps video is 33.33ms without considering the fluctuation of the video frame.

[0157] See also Figure 7 As shown, for the case where the periodic interval of the Vsync-app signal is less than the inter-frame interval of the video, in each refresh period indicated by the Vsync-app signal, according to the relevant technology, after receiving the Vsync-app signal, the frame buffer is queried to see if there are video frames to be rendered, and then the video frame rendering process is performed. If there are no video frames to be rendered in the frame buffer, the rendering operation of the video frame view is suspended. The refresh period can be divided into an overwriting risk period and a synchronous rendering period according to the time point when the video frame to be rendered is inserted into the frame buffer. Among them, the overwriting risk period refers to the period from the receipt of the Vsync-app signal to the scheduling of the rendering thread. The insertion of the video frame to be rendered into the frame buffer during this period may cause the old video frame that has not been consumed to be overwritten; the period other than the overwriting risk period in the refresh cycle is the synchronous rendering period. If the video frame needs to be rendered in the current refresh period, the video frame to be rendered inserted into the frame buffer during this period will be consumed normally after receiving the next Vsync-app signal.

[0158] Taking the refresh cycles 8 and 9 indicated by the Vsync-app signals 8 and 9 in the figure as an example, assuming that the refresh cycles 8 and 9 are used to render video frames 9 and 10 respectively, the cycle length of the refresh cycle is 32.88ms. Without considering the fluctuation of the video frames, the time interval between time points A and B and time points C and D is 33.33ms. Based on the fact that the periodic interval of the Vsync-app signal is smaller than the inter-frame interval of the video, assuming that video frame 9 is inserted into the frame buffer at time point A shown in the figure, and video frame 10 is correspondingly inserted into the frame buffer at time point B shown in the figure, then according to the video frame rendering processing method of the above-mentioned related technology, video frame 9 is inserted after receiving the Vsync-app signal 8, then video frame 9 cannot be rendered in refresh cycle 8, but is rendered in refresh cycle 9. Since video frame 10 is inserted before the rendering thread of refresh cycle 9 consumes the frame buffer, video frame 9 is not consumed when video frame 10 is inserted into the frame buffer, and will be overwritten by video frame 10, which may cause continuous frame loss and freezes. Problems; if video frame 9 is inserted at point C, video frame 10 will be inserted at point D accordingly, and the rendering of video frames 9 and 10 will be completed in refresh cycle 9 and refresh cycle 10 respectively.

[0159] In order to solve the above-mentioned playback stuttering and frame loss problems, the present disclosure provides a video playback method, which changes the processing method of the video frames to be rendered based on a prediction method. During the video playback process, the application calls the video decoder to obtain the decoded data frame of the video source, and inserts the decoded video frame into the frame buffer according to the inter-frame interval of the video as the target video frame to be rendered, and after inserting each video frame into the frame buffer, notifies the main thread in the form of a thread synchronization message, so that the main thread can process the target video frame to be rendered in time after receiving the Vsync-app signal.

[0160] like Figure 8 As shown, from the perspective of the main thread of the application, the method may include the following steps:

[0161] S801, in response to receiving a Vsync-app signal from a mobile phone display controller, querying whether an associated message queue contains the synchronization message to be processed;

[0162] The synchronization message is used to notify the main thread of the application program of the video frame to be rendered when the next Vsync-app signal arrives. If the message queue is queried after receiving the Vsync-app signal and does not include the synchronization information to be processed, it means that there is no video frame to be rendered in the current frame buffer.

[0163] S802, if the message queue does not include the synchronization message, execute step S803; otherwise, if the message queue includes the synchronization message, execute step S806;

[0164] S803, predicting the insertion time of the target video frame into the frame buffer according to the timestamp information of the previous frame of the target video frame being inserted into the frame buffer and the inter-frame interval;

[0165] S804, predicting the consumption time of the rendering thread in the current refresh cycle according to the historical consumption time of the video frames to be rendered in the frame buffer of the rendering thread in the historical refresh cycle before the current refresh cycle indicated by the Vsync-app signal;

[0166] S805, in response to the insertion time being earlier than the consumption time, setting a transition state and calling an interface to notify the rendering thread to be ready to perform video frame rendering processing;

[0167] If the insertion time is later than the consumption time, the main thread can notify the rendering thread through the thread communication mechanism to skip the rendering processing operation of the video frame view in the current refresh cycle, wait for the next refresh cycle to perform the rendering processing of the video frame, and return to execute step S801.

[0168] S806, the rendering thread consumes the frame buffer, renders the acquired video frame and hands it over to the underlying service of the mobile phone, and displays the rendering result on the application program interface.

[0169] In the disclosed embodiment, when there is no video frame rendered in the current refresh cycle in the frame buffer, the timing of inserting the video frame to be rendered into the frame buffer is predicted. By judging whether the video frame to be rendered can arrive before the rendering thread consumes it, the corresponding video frame rendering strategy is adopted, which reduces the occurrence of video frame coverage problems, postpones the timing of delayed consumption problems, and greatly shortens the interval of the above-mentioned coverage risk period. Based on this video playback method, compared with the related art that uses a multiple refresh method to play videos to maintain video smoothness, it achieves smooth video playback in a video playback scenario where the refresh rate is set to the same as the video frame rate, and reduces device power consumption.

[0170] It should be noted that the specific interaction implementation method between the threads in the above application program to implement the video playback scenario will vary according to the architecture and requirements of the application program. On the Android platform, mechanisms such as Handler, MessageQueue, HandlerThread, etc. can be used for inter-thread communication and message processing, and this application does not limit this.

[0171] It can be understood that the steps described in the above embodiments do not necessarily require a specific execution order or execution time. In some embodiments, the steps can be executed in a different order or time and still achieve the desired effect.

[0172] Corresponding to the embodiment of the aforementioned video playback method, Fig. 9 is a schematic diagram of the structure of a video playback device according to an exemplary embodiment. Fig. 9 As shown, the device includes a signal receiving module 910, an insertion time prediction module 920, a rendering processing module 930 and a display module 940, and the device may include:

[0173] A signal receiving module 910 is used to receive a vertical synchronization signal, where the vertical synchronization signal is used to instruct the main thread of the application to render a first video frame to be rendered in a frame buffer;

[0174] an insertion time prediction module 920, configured to predict, in response to the absence of the first video frame in the frame buffer, an insertion time of the first video frame into the frame buffer according to timestamp information of the second video frame being inserted into the frame buffer and a frame rate of the corresponding video; the second video frame being a previous frame of the first video frame;

[0175] A rendering processing module 930, configured to perform rendering processing in the refresh cycle according to a relationship between the insertion time and a scheduling time of a rendering thread in the refresh cycle indicated by the vertical synchronization signal;

[0176] The display module 940 is used to obtain the rendering result corresponding to the rendering process and display the rendering result in the video playback area of ​​the application.

[0177] In some embodiments, the apparatus further comprises:

[0178] Querying whether a message queue associated with the main thread includes a synchronization message to be processed, wherein the synchronization message is used to indicate that a video frame to be rendered has been inserted into the frame buffer;

[0179] If not, determining that the first video frame does not exist in the frame buffer; or,

[0180] Monitoring whether an update operation of inserting a video frame occurs in the frame buffer;

[0181] In response to the update operation not occurring before receiving a vertical synchronization signal, it is determined that the first video frame does not exist in the frame buffer.

[0182] In some embodiments, the insertion time prediction module is specifically used to:

[0183] Determine a time interval between the first video frame and the second video frame according to a frame rate of a video to which the second video frame belongs;

[0184] The insertion time is determined according to the timestamp information of the second video frame and the time interval.

[0185] In some embodiments, the rendering processing module is specifically used to:

[0186] In response to the insertion time being earlier than the scheduling time, setting a transition state to indicate that the rendering thread is ready to perform a rendering operation;

[0187] After the first video frame is inserted into the frame buffer, the rendering thread obtains the first video frame from the frame buffer and performs rendering processing on the first video frame.

[0188] In some embodiments, the rendering processing module is specifically used to:

[0189] In response to the insertion time being later than the scheduling time, sending an empty frame command to the rendering thread;

[0190] After receiving the empty frame command, the rendering thread skips the rendering processing operation on the video frame view in the refresh cycle.

[0191] In some embodiments, the apparatus further comprises:

[0192] An acquisition module, used for acquiring a historical scheduling time of a rendering thread in a refresh cycle indicated by at least one historical signal before the vertical synchronization signal;

[0193] The scheduling time prediction module is used to predict the scheduling time of the rendering thread within the refresh period indicated by the vertical synchronization signal according to the historical scheduling time.

[0194] In some embodiments, the scheduling time prediction module is specifically used to:

[0195] Determine the scheduling time interval between adjacent historical signals;

[0196] Determining an occurrence period of the rendering thread according to the scheduling time interval;

[0197] The scheduling time of the rendering thread in the refresh cycle indicated by the vertical synchronization signal is determined according to the generation cycle of the rendering thread and the historical scheduling time of the rendering thread in a previous historical refresh cycle of the refresh cycle.

[0198] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0199] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art may understand and implement it without creative work.

[0200] The embodiment of the present disclosure also provides a terminal device. The terminal device includes: a memory and a processor. The memory stores processor executable instructions, and the processor is configured to execute the executable instructions in the memory to implement the steps of video playback provided above. In the embodiment of the present disclosure, the terminal device can be an electronic device such as a mobile phone or a tablet computer with an application installed.

[0201] Fig.10 is a block diagram of a terminal device provided according to an exemplary embodiment. Fig.10As shown, the terminal device 1000 may include one or more of the following components: a processing component 1002, a memory 1004, a power component 1006, a multimedia component 1008, an audio component 1010, an input / output (I / O) interface 1012, a sensor component 1014, a communication component 1016, and an image acquisition component.

[0202] The processing component 1002 generally controls the overall operation of the terminal device 1000, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1002 may include one or more processors 1020 to execute instructions. In addition, the processing component 1002 may include one or more units to facilitate interaction between the processing component 1002 and other components. For example, the processing component 1002 may include a multimedia unit to facilitate interaction between the multimedia component 1008 and the processing component 1002.

[0203] The memory 1004 is configured to store various types of data to support operations on the terminal device 1000. Examples of such data include instructions for any application or method operating on the terminal device 1000, contact data, phone book data, messages, pictures, videos, etc. The memory 1004 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0204] The power supply component 1006 provides power to various components of the terminal device 1000. The power supply component 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the terminal device 1000.

[0205] The multimedia component 1008 includes a screen that provides an output interface between the terminal device 1000 and the target object. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the target object. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0206] The audio component 1010 is configured to output and / or input audio signals. For example, the audio component 1010 includes a microphone (MIC), and when the terminal device 1000 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1004 or sent via the communication component 1016. In some embodiments, the audio component 1010 also includes a speaker for outputting audio signals.

[0207] The I / O interface 1012 provides an interface between the processing component 1002 and a peripheral interface unit, such as a keyboard, a click wheel, a button, etc.

[0208] The sensor component 1014 includes one or more sensors for providing various aspects of status evaluation for the terminal device 1000. For example, the sensor component 1014 can detect the open / closed state of the terminal device 1000, the relative positioning of components, such as the display screen and keypad of the terminal device 1000, and the sensor component 1014 can also detect the position change of the terminal device 1000 or a component, the presence or absence of contact between the target object and the terminal device 1000, the orientation or acceleration / deceleration of the terminal device 1000, and the temperature change of the terminal device 1000. For another example, the sensor component 1014 also includes a light sensor, which is arranged below the OLED display screen.

[0209] The communication component 1016 is configured to facilitate wired or wireless communication between the terminal device 1000 and other devices. The terminal device 1000 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1016 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1016 also includes a near field communication (NFC) unit to facilitate short-range communication. For example, the NFC unit can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0210] In an exemplary embodiment, the terminal device 1000 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components.

[0211] In an exemplary embodiment, the present disclosure also provides a readable storage medium, which stores executable instructions. The executable instructions can be executed by a processor of a terminal device to implement the steps of the video playback method provided above. The readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0212] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the disclosure disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the above claims.

[0213] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A video playback method, characterized in that: The method comprises: Receiving a vertical synchronization signal, where the vertical synchronization signal is used to instruct a main thread of the application to render a first video frame to be rendered in a frame buffer; In response to the first video frame not existing in the frame buffer, predicting an insertion time of the first video frame into the frame buffer according to timestamp information of the second video frame being inserted into the frame buffer and a frame rate of the corresponding video; the second video frame being a previous frame of the first video frame; Execute rendering processing in the refresh cycle according to the relationship between the insertion time and the scheduling time of the rendering thread in the refresh cycle indicated by the vertical synchronization signal; A rendering result corresponding to the rendering process is obtained, and the rendering result is displayed in a video playback area of ​​the application.

2. The method according to claim 1, characterized in that: After receiving the vertical synchronization signal, the method further includes: Querying whether a message queue associated with the main thread includes a synchronization message to be processed, wherein the synchronization message is used to indicate that a video frame to be rendered has been inserted into the frame buffer; If not, determining that the first video frame does not exist in the frame buffer; or, Monitoring whether an update operation of inserting a video frame occurs in the frame buffer; In response to the update operation not occurring before receiving a vertical synchronization signal, it is determined that the first video frame does not exist in the frame buffer.

3. The method according to claim 1, characterized in that: The predicting the insertion time of the first video frame into the frame buffer according to the timestamp information of the second video frame being inserted into the frame buffer and the frame rate of the corresponding video includes: Determine a time interval between the first video frame and the second video frame according to a frame rate of a video to which the second video frame belongs; The insertion time is determined according to the timestamp information of the second video frame and the time interval.

4. The method according to claim 1, characterized in that: The performing of rendering processing in the refresh cycle according to the relationship between the insertion time and the scheduling time of the rendering thread in the refresh cycle indicated by the vertical synchronization signal comprises: In response to the insertion time being earlier than the scheduling time, setting a transition state to indicate that the rendering thread is ready to perform a rendering operation; After the first video frame is inserted into the frame buffer, the rendering thread obtains the first video frame from the frame buffer and performs rendering processing on the first video frame.

5. The method according to claim 4, characterized in that The method further comprises: In response to the insertion time being later than the scheduling time, sending an empty frame command to the rendering thread; After receiving the empty frame command, the rendering thread skips the rendering processing operation on the video frame view in the refresh cycle.

6. The method according to claim 1, characterized in that After predicting the insertion time of the first video frame, the method further comprises: Obtaining a historical scheduling time of a rendering thread in a refresh cycle indicated by at least one historical signal before the vertical synchronization signal; The scheduling time of the rendering thread within the refresh period indicated by the vertical synchronization signal is predicted according to the historical scheduling time.

7. The method according to claim 6, characterized in that The predicting the scheduling time of the rendering thread within the refresh cycle indicated by the vertical synchronization signal includes: Determine the scheduling time interval between adjacent historical signals; Determining an occurrence period of the rendering thread according to the scheduling time interval; The scheduling time of the rendering thread in the refresh cycle indicated by the vertical synchronization signal is determined according to the generation cycle of the rendering thread and the historical scheduling time of the rendering thread in a previous historical refresh cycle of the refresh cycle.

8. A video playback device, characterized in that: The device comprises: A signal receiving module, used for receiving a vertical synchronization signal, wherein the vertical synchronization signal is used for instructing a main thread of the application to render a first video frame to be rendered in a frame buffer; an insertion time prediction module, configured to predict, in response to the absence of the first video frame in the frame buffer, an insertion time of the first video frame into the frame buffer according to timestamp information of the second video frame being inserted into the frame buffer and a frame rate of the corresponding video; the second video frame being a previous frame of the first video frame; A rendering processing module, configured to perform rendering processing in the refresh cycle according to a relationship between the insertion time and a scheduling time of a rendering thread in the refresh cycle indicated by the vertical synchronization signal; The display module is used to obtain the rendering result corresponding to the rendering process and display the rendering result in the video playback area of ​​the application.

9. An electronic device, characterized in that: include: Processor, memory; The memory is used to store computer programs; The processor is used to call the computer program to implement the video playback method according to any one of claims 1 to 7.

10. A readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the video playback method according to any one of claims 1 to 7 is implemented.

Images