Timestamp correction method and related product

By correcting the capture timestamp of the video frame to be greater than the reference timestamp, the playback abnormality problem caused by incorrect video frame timestamps was resolved, improving the stability and smoothness of video playback.

CN119031215APending Publication Date: 2024-11-26SHUXING TECH (BEIJING) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411175617.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

Incorrect timestamps in video frame capture can easily lead to errors in video frame decoding and display timestamps, resulting in abnormalities such as stuttering, black screen, acceleration, and screen tearing during video playback.

Method used

By determining the reference timestamp and target timestamp of adjacent video frames, the time increment is determined based on the frame rate of the target video, and the target timestamp is corrected by the time increment to make it greater than the reference timestamp, thus ensuring the accuracy of the timestamp.

Benefits of technology

It improves the accuracy of video frame capture timestamps and reduces abnormal situations during video playback, such as stuttering, black screens, and acceleration.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119031215A_ABST
    Figure CN119031215A_ABST
Patent Text Reader

Abstract

The invention relates to a timestamp correction method and a related product. The timestamp correction method comprises the steps that a reference timestamp of a previous frame of a target video and a target timestamp of a current frame of the target video are determined respectively, the previous frame and the current frame are any two adjacent collected video frames, the previous frame is collected before the current frame, and both the reference timestamp and the target timestamp refer to collection timestamps of the video frames; under the condition that the target timestamp is smaller than or equal to the reference timestamp, a first time increment is determined based on the frame rate of the target video, the first time increment is a positive number, and the first time increment is in negative correlation with the frame rate; and correcting the target timestamp according to the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp. By adopting the timestamp correction method, the acquisition timestamp of the video frame can be more accurate.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of video processing, and in particular to a timestamp correction method and related products. Background Art

[0002] In the process of capturing and playing videos, the capture timestamp of the video frame may be wrong. The error of the capture timestamp of the video frame is likely to cause the error of the decoding timestamp and display timestamp of the video frame, which in turn may cause abnormalities such as freeze, black screen, acceleration, and screen distortion during video playback. Based on this, making the capture timestamp of the video frame more accurate is of great significance to reduce abnormalities during video playback. Summary of the invention

[0003] Based on this, it is necessary to provide a timestamp correction method and related products to address the above technical issues, so as to make the acquisition timestamp of the video frame more accurate. The related products include a video frame display method, a timestamp correction device, a video frame display device, a computer device, a computer-readable storage medium, and a computer program product.

[0004] In a first aspect, the present application provides a timestamp correction method, the method comprising:

[0005] Determine respectively a reference timestamp of a previous frame of a target video and a target timestamp of a current frame of the target video, wherein the previous frame and the current frame are any two adjacently captured video frames, and the previous frame is captured before the current frame, and both the reference timestamp and the target timestamp refer to the capture timestamp of the video frame;

[0006] When the target timestamp is less than or equal to the reference timestamp, determining a first time increment based on the frame rate of the target video, the first time increment is a positive number, and the first time increment is negatively correlated with the frame rate;

[0007] The target timestamp is corrected by the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp.

[0008] The timestamp correction method provided by the first aspect is adopted to respectively determine the reference timestamp of the previous frame and the target timestamp of the current frame that are adjacent to the previous frame. Wherein, the reference timestamp and the target timestamp both refer to the acquisition timestamp, and the previous frame is acquired before the current frame. In theory, the target timestamp of the current frame that is acquired later should be greater than the reference timestamp of the previous frame that is acquired earlier. That is, the target timestamp is more accurate when the target timestamp is greater than the reference timestamp. Therefore, when the target timestamp is less than or equal to the reference timestamp, it is necessary to correct the target timestamp of the current frame in time so that the target timestamp is greater than the reference timestamp and the target timestamp of the current frame is more accurate. In the case where the target timestamp is less than or equal to the reference timestamp, the timestamp correction method of the present invention determines a first time increment as a positive number based on the frame rate of the target video, and the first time increment is negatively correlated with the frame rate. Then, the target timestamp is corrected by the sum of the target timestamp and the first time increment so that the target timestamp is greater than the reference video frame. At this time, it is possible to make the target timestamp of the current frame more accurate. In addition, since the ideal time interval between any two adjacent captured video frames of the target video is 1 / frame rate. Therefore, this timestamp correction method determines the first time increment based on the frame rate of the target video, and the first time increment is negatively correlated with the frame rate. It can be understood that this timestamp correction method determines the first time increment based on the ideal time interval. Through this technical approach, when correcting the target timestamp based on the first time increment, not only can the target timestamp be greater than the reference timestamp, but also the actual time interval formed by the corrected target timestamp and the reference timestamp can be related to the ideal time interval, which is conducive to making the actual time interval more reasonable, and making the corrected target timestamp closer to the ideal target timestamp, that is, it is also conducive to improving the accuracy of the target timestamp of the current frame. The use of this timestamp correction method makes the target timestamp greater than the reference timestamp, improves the accuracy of the target timestamp of the current frame, and makes the acquisition timestamps of any two adjacent captured video frames increase, making the acquisition timestamp of the video frame more accurate.

[0009] In one embodiment, determining the first time increment based on the frame rate of the target video includes:

[0010] The product of the inverse of the frame rate of the target video and a first coefficient is used as the first time increment, where the first coefficient is a positive number less than 1;

[0011] The step of correcting the target timestamp by using the sum of the target timestamp and the first time increment so that the target timestamp is greater than the reference timestamp includes:

[0012] The target timestamp is iteratively corrected by using the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp.

[0013] In one of the embodiments, after correcting the target timestamp by the sum of the target timestamp and the first time increment so that the target timestamp is greater than the reference timestamp, the method further includes:

[0014] Determine a timestamp to be corrected of a target frame of the target video, where the target frame is any video frame captured after the current frame, and the timestamp to be corrected is a capture timestamp of the target frame;

[0015] The time stamp to be modified is modified by the sum of the time stamp to be modified and the first time increment.

[0016] In one embodiment, the method further comprises:

[0017] In a case where the target timestamp is greater than the reference timestamp, determining a time interval formed by the target timestamp and the reference timestamp;

[0018] In the case where the time interval does not belong to a preset interval, determining a second time increment based on the frame rate of the target video, the preset interval being a standard for determining whether the duration of the time interval is reasonable;

[0019] The target timestamp is corrected by the sum of the target timestamp and the second time increment, so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval.

[0020] In one of the embodiments, when the time interval does not belong to the preset interval, determining the second time increment based on the frame rate of the target video includes:

[0021] When the time interval is less than the minimum boundary value of the preset interval, the second time increment is determined based on the frame rate of the target video, the second time increment is a positive number, and the second time increment is negatively correlated with the frame rate.

[0022] In one embodiment, determining the second time increment based on the frame rate of the target video includes:

[0023] The product of the reciprocal of the frame rate of the target video and a second coefficient is used as the second time increment, where the second coefficient is a positive number less than 1;

[0024] The step of correcting the target timestamp by using the sum of the target timestamp and the second time increment so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval includes:

[0025] The target timestamp is iteratively corrected by using the sum of the target timestamp and the second time increment, so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval.

[0026] In one of the embodiments, when the time interval does not belong to the preset interval, determining the second time increment based on the frame rate of the target video includes:

[0027] When the time interval is greater than the maximum boundary value of the preset interval, the second time increment is determined based on the frame rate of the target video, the second time increment is a negative number, and the absolute value of the second time increment is negatively correlated with the frame rate.

[0028] In one embodiment, determining the second time increment based on the frame rate of the target video, wherein the second time increment is a negative number, comprises:

[0029] The product of the reciprocal of the frame rate of the target video and the third coefficient is used as the absolute value of the second time increment, the third coefficient is a positive number less than 1, and the second time increment is a negative number;

[0030] The step of correcting the target timestamp by using the sum of the target timestamp and the second time increment so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval includes:

[0031] The target timestamp is iteratively corrected by using the sum of the target timestamp and the second time increment, so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval.

[0032] In one of the embodiments, after the target timestamp is corrected by the sum of the target timestamp and the second time increment so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval, the method further includes:

[0033] Determine a timestamp to be corrected of a target frame of the target video, where the target frame is any video frame captured after the current frame, and the timestamp to be corrected is a capture timestamp of the target frame;

[0034] The time stamp to be corrected is corrected by the sum of the time stamp to be corrected and a target time increment, wherein the target time increment is the second time increment or the sum of the first time increment and the second time increment.

[0035] In one of the embodiments, before respectively determining the reference timestamp of the previous frame of the target video and the target timestamp of the current frame of the target video, the method further includes:

[0036] The original video is subjected to target processing to obtain the target video, wherein the target processing includes any one of acquisition processing, pre-processing, and encoding processing.

[0037] In a second aspect, the present application further provides a video frame display method, which is applied to a live broadcast terminal, and the method includes:

[0038] Determine respectively a reference timestamp of a previous frame of a target video and a target timestamp of a current frame of the target video, wherein the previous frame and the current frame are any two adjacently captured video frames, and the previous frame is captured before the current frame, and both the reference timestamp and the target timestamp refer to the capture timestamp of the video frame;

[0039] When the target timestamp is less than or equal to the reference timestamp, determining a first time increment based on the frame rate of the target video, the first time increment is a positive number, and the first time increment is negatively correlated with the frame rate;

[0040] Correcting the target timestamp by the sum of the target timestamp and the first time increment so that the target timestamp is greater than the reference timestamp;

[0041] In a case where the target timestamp is greater than the reference timestamp, determining a decoding timestamp and a display timestamp of the current frame based on the target timestamp;

[0042] The current frame and the decoding timestamp and the display timestamp of the current frame are sent to the service server in the live broadcast link, where the decoding timestamp refers to the time information for decoding the current frame, and the display timestamp refers to the time information for displaying the decoded current frame.

[0043] Using the video frame display method provided by the second aspect, the live broadcast terminal first determines the reference timestamp of the previous frame and the target timestamp of the current frame respectively. Wherein, the reference timestamp and the target timestamp both refer to the acquisition timestamp, and the previous frame is acquired before the current frame. Further, when the target timestamp is less than or equal to the reference timestamp, the live broadcast terminal determines a first time increment as a positive number based on the frame rate of the target video, and the first time increment is negatively correlated with the frame rate, and the target timestamp is corrected by the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference video frame, so that the target timestamp of the current frame is more accurate. When the target timestamp is greater than the reference timestamp, the target timestamp of the current frame is more accurate. It can be understood that the target timestamp greater than the reference timestamp at this time may be a target timestamp that is greater than the reference timestamp without correction, or may be a target timestamp that is initially less than or equal to the reference timestamp and is greater than the reference timestamp after correction. When the target timestamp is more accurate, the live broadcast terminal determines the decoding timestamp and display timestamp of the current frame based on the more accurate target timestamp. The decoding timestamp and display timestamp obtained at this time are also more accurate. Since the decoding timestamp refers to the time information for decoding the current frame, and the display timestamp refers to the time information for displaying the decoded current frame. Therefore, accurate decoding timestamps and display timestamps are conducive to ensuring that the current frame and other video frames are decoded and played normally, and are conducive to reducing abnormal situations during video playback. For example, if the decoding timestamp is too small, the current frame needs to be decoded earlier. At this time, the current frame may not be decoded because the reference frame that the current frame needs to refer to for decoding has not been decoded, which may affect the normal decoding of other video frames. When an error occurs in the decoding of a video frame, it is also difficult for the decoded video frame to be played normally, and the video playback may experience abnormal situations such as stuttering, black screen, acceleration, and flowery screen. For example, if the display timestamp is too large, the decoded current frame will be displayed later, and at this time, the video playback may also experience abnormal situations such as stuttering. The live broadcast terminal sends the current frame and accurate decoding timestamps and display timestamps to the service server in the live broadcast link, so that the current frame is decoded based on a more accurate decoding timestamp, and the decoded current frame is displayed based on a more accurate display timestamp. This helps reduce the possibility of abnormal video playback caused by live streaming, so that video frames can be decoded and displayed in an orderly and stable manner according to accurate decoding timestamps and display timestamps, which helps reduce abnormal situations during video playback and improve the live broadcast effect.

[0044] In a third aspect, the present application further provides a timestamp correction device, the device comprising:

[0045] A timestamp determination module, used to respectively determine a reference timestamp of a previous frame of a target video and a target timestamp of a current frame of the target video, wherein the previous frame and the current frame are any two adjacently captured video frames, and the previous frame is captured before the current frame, and the reference timestamp and the target timestamp both refer to the capture timestamp of the video frame;

[0046] a time increment determination module, configured to determine a first time increment based on a frame rate of the target video when the target timestamp is less than or equal to the reference timestamp, wherein the first time increment is a positive number and is negatively correlated with the frame rate;

[0047] The timestamp correction module is used to correct the target timestamp by using the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp.

[0048] In a fourth aspect, the present application further provides a video frame display device, which is applied to a live broadcast terminal, and the device includes:

[0049] A first determination module is used to respectively determine a reference timestamp of a previous frame of a target video and a target timestamp of a current frame of the target video, wherein the previous frame and the current frame are any two adjacently captured video frames, and the previous frame is captured before the current frame, and the reference timestamp and the target timestamp both refer to the capture timestamp of the video frame;

[0050] A second determining module is used to determine a first time increment based on the frame rate of the target video when the target timestamp is less than or equal to the reference timestamp, wherein the first time increment is a positive number and is negatively correlated with the frame rate;

[0051] A correction module, configured to correct the target timestamp by using the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp;

[0052] A third determining module, configured to determine a decoding timestamp and a display timestamp of the current frame based on the target timestamp when the target timestamp is greater than the reference timestamp;

[0053] The sending module is used to send the current frame and the decoding timestamp and the display timestamp of the current frame to the business server in the live broadcast link, wherein the decoding timestamp refers to the time information for decoding the current frame, and the display timestamp refers to the time information for displaying the decoded current frame.

[0054] In a fifth aspect, the present application also provides a computer device, comprising: a memory and a processor, wherein the memory stores program instructions; when the program instructions are executed by the processor, the processor executes the method as shown in the first aspect or any embodiment of the first aspect, or executes the method as shown in the second aspect or any embodiment of the second aspect.

[0055] In a sixth aspect, the present application also provides a computer-readable storage medium, in which a computer program is stored; when the computer program runs on one or more processors, the method shown in the first aspect or any embodiment of the first aspect is executed, or the method shown in the second aspect or any embodiment of the second aspect is executed.

[0056] In the seventh aspect, the present application also provides a computer program product, comprising a computer program or instructions; when the computer program or instructions are run on a computer, the computer is caused to execute the method as shown in the first aspect or any embodiment of the first aspect, or execute the method as shown in the second aspect or any embodiment of the second aspect.

[0057] It can be understood that the timestamp correction device provided in the third aspect, the video frame display device provided in the fourth aspect, the computer device provided in the fifth aspect, the computer readable storage medium provided in the sixth aspect, and the computer program product provided in the seventh aspect can be used to execute the timestamp correction method shown in the first aspect of the present application or any embodiment of the first aspect, or to execute the video frame display method shown in the second aspect of the present application or any embodiment of the second aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments are introduced as follows.

[0059] Figure 1 One of the flowcharts of the timestamp correction method provided in the embodiment of the present application;

[0060] Figure 2 The second flowchart of the timestamp correction method provided in the embodiment of the present application;

[0061] Figure 3 The third flowchart of the timestamp correction method provided in the embodiment of the present application;

[0062] Figure 4 A fourth flowchart of the timestamp correction method provided in an embodiment of the present application;

[0063] Figure 5A fifth flowchart of the timestamp correction method provided in an embodiment of the present application;

[0064] Figure 6 Flow chart six of the timestamp correction method provided in the embodiment of the present application;

[0065] Figure 7 Flow chart of the timestamp correction method provided in the embodiment of the present application.

[0066] Figure 8 A schematic diagram of a flow chart of a video frame display method provided in an embodiment of the present application;

[0067] Fig. 9 A schematic diagram of the structure of a video frame display system provided in an embodiment of the present application;

[0068] Fig.10 A schematic diagram of the structure of a timestamp correction device provided in an embodiment of the present application;

[0069] Fig.11 A schematic diagram of the structure of a video frame display device provided in an embodiment of the present application;

[0070] Fig.12 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to facilitate understanding of the embodiments of the present application, the embodiments of the present application will be described more fully below with reference to the relevant drawings. The preferred embodiments of the embodiments of the present application are given in the drawings. However, the embodiments of the present application can be implemented in many different forms and are not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the embodiments of the present application more thorough and comprehensive.

[0072] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the technical field of the embodiments of the present application. The terms used herein in the description of the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the embodiments of the present application.

[0073] The terms "first", "second", etc. used in this article are used to distinguish different objects, rather than to describe a specific order. For the purpose of distinguishing different objects, "first time value-added" can also be described as "second time value-added", and "second time value-added" can be described as "first time value-added". When used, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless the context clearly indicates otherwise. It should also be understood that the term "includes / comprising" specifies the presence of the stated features, wholes, steps, operations, parts or their fusions, but does not exclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, parts or their fusions.

[0074] In the process of capturing and playing videos, the capture timestamp of the video frame may be wrong. Since the decoding timestamp and display timestamp of the video frame marked when encoding the video frame need to be determined based on the capture timestamp of the video frame. Therefore, the capture timestamp of the video frame is wrong, which can easily lead to the decoding timestamp and display timestamp of the video frame. Since the decoding timestamp refers to the time information of decoding the video frame, and the display timestamp refers to the time information of displaying the decoded video frame. Therefore, the decoding timestamp and display timestamp of the video frame are wrong, which can easily lead to the decoding and display errors of the video frame. For example, if the decoding timestamp of the video frame is too small, the video frame needs to be decoded earlier. At this time, the video frame may not be decoded because the reference frame that the video frame needs to refer to for decoding has not been decoded. If it cannot be decoded, it cannot be displayed. When the video is played to this video frame, abnormal conditions such as a black screen may occur. If the video frame is decoded and the display timestamp of this video frame is too large, the video may be stuck before playing this video frame; if the display timestamp is too small, the video may be accelerated before playing this video frame. Based on the decoding reference relationship between each video frame, if the decoding timestamp of a video frame is wrong, it may cause the decoding errors of this video frame and many other video frames, and the decoding errors will also cause display errors. For example, if the decoding timestamp of the P frame (Predicted frame) is too small, the P frame needs to be decoded earlier. At this time, the P frame may not be decoded because the I frame (Intra-coded frame) that the P frame needs to refer to for decoding or the P frame that needs to be referenced for decoding has not been decoded. Furthermore, when other P frames or B frames (Bi-directional predicted frames) need to refer to this P frame for decoding, the P frame or B frame that needs to be referenced may be decoded incorrectly because the P frame cannot be decoded. When an error occurs in the decoding of a video frame, the undecoded video frame cannot be displayed, and the decoded video frame is also difficult to be displayed normally. Therefore, errors in the decoding timestamp and display timestamp of the video frame are likely to cause abnormal situations such as freeze, black screen, acceleration, and flower screen in video playback. It is understandable that, based on the different characteristics of the three types of video frames, I-frames, P-frames, and B-frames, errors in the acquisition timestamps, decoding timestamps, and display timestamps of different types of video frames have different effects on the decoding results and display results of other video frames, resulting in different degrees of abnormality when abnormalities occur in video playback. Based on this, making the acquisition timestamps of video frames more accurate to accurately determine the decoding timestamps and display timestamps of video frames is conducive to ensuring that this video frame and other video frames are decoded and played normally, which is of great significance to reducing abnormalities during video playback.

[0075] The embodiment of the present application provides a timestamp correction method that can make the acquisition timestamp of a video frame more accurate. The timestamp correction method can be applied to terminal devices such as mobile phones and computers. Figure 1As shown, the time stamp correction method includes the following steps S101 to S103.

[0076] S101, respectively determining a reference timestamp of a previous frame of a target video and a target timestamp of a current frame of the target video. The previous frame and the current frame are any two adjacently captured video frames, and the previous frame is captured before the current frame. The reference timestamp and the target timestamp both refer to the capture timestamp of the video frame.

[0077] The acquisition timestamp is usually generated by the terminal device during the video acquisition phase, and is a timestamp that exists in the video acquisition, encoding, transmission, decoding, rendering, etc. The acquisition timestamp of the video frame is stored in the data information of the video frame.

[0078] S102: When the target timestamp is less than or equal to the reference timestamp, determine a first time increment based on the frame rate of the target video. The first time increment is a positive number, and the first time increment is negatively correlated with the frame rate.

[0079] S103: Correct the target timestamp by the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp.

[0080] In theory, the target timestamp of the current frame that is captured later should be greater than the reference timestamp of the previous frame that is captured earlier. That is, the target timestamp is more accurate when it is greater than the reference timestamp. Therefore, when the target timestamp is less than or equal to the reference timestamp, the target timestamp of the current frame needs to be corrected in time to make the target timestamp greater than the reference timestamp and make the target timestamp of the current frame more accurate.

[0081] In the case where the target timestamp is less than or equal to the reference timestamp, the timestamp correction method determines a first time increment as a positive number based on the frame rate of the target video, and the first time increment is negatively correlated with the frame rate. Then, the target timestamp is corrected by the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference video frame. In this case, the target timestamp of the current frame can be made more accurate.

[0082] Optionally, the target timestamp can be corrected by the following formula:

[0083]

[0084] Among them, timestamp is the target timestamp before correction; timestamp fix is the corrected target timestamp; It is the first time increment, fps is the frame rate of the target video, and A is a positive number.

[0085] A can be set according to actual needs and business scenarios in order to make the corrected target timestamp larger than the reference video frame. For example, in practice, it is found that if the frame rate of the target video in the on-demand application scenario is 60fps, that is, 60 video frames are displayed per second, then when A is 1 / 3, the corrected target timestamp can be larger than the reference timestamp.

[0086] In addition, since the ideal time interval between any two adjacent captured video frames of the target video is 1 / frame rate. Therefore, the timestamp correction method determines the first time increment based on the frame rate of the target video, and the first time increment is negatively correlated with the frame rate. It can be understood that the timestamp correction method determines the first time increment based on the ideal time interval. Through this technical approach, when correcting the target timestamp based on the first time increment, not only can the target timestamp be greater than the reference timestamp, but the actual time interval formed by the corrected target timestamp and the reference timestamp can be related to the ideal time interval, which is conducive to making the actual time interval more reasonable and the corrected target timestamp closer to the ideal target timestamp, that is, it is also conducive to improving the accuracy of the target timestamp of the current frame.

[0087] In this embodiment, when the target timestamp of the current frame is less than or equal to the reference timestamp of the previous frame, the first time increment is determined by using the frame rate of the target video to correct the target timestamp. The target timestamp is made greater than the reference timestamp, which improves the accuracy of the target timestamp of the current frame, so that the acquisition timestamps of any two adjacent video frames are incremented, and the acquisition timestamp of the video frame is more accurate. In addition, the first time increment is determined based on the frame rate of the target video, and the first time increment is negatively correlated with the frame rate, and then the target timestamp is corrected based on the first time increment, which is conducive to making the corrected target timestamp closer to the ideal target timestamp, and is conducive to improving the accuracy of the target timestamp of the current frame. That is, the acquisition timestamp of the video frame can also be made more accurate. Furthermore, it is conducive to accurately determining the decoding timestamp and display timestamp of the video frame, which is conducive to ensuring that the video frame is decoded and played normally, which is conducive to reducing the possibility of abnormal video playback due to errors in the acquisition timestamp of the video frame, and is conducive to reducing abnormal situations such as black screen, acceleration, freeze, black screen, etc. during video playback.

[0088] In one embodiment, if Figure 2 As shown, the timestamp correction method includes the following steps S201 to S203. Among them, step S201 corresponds to step S101 in the above embodiment. The specific discussion of step S201 in this embodiment can refer to the specific discussion of step S101 in the above embodiment, which will not be repeated in this embodiment.

[0089] S201, respectively determine a reference timestamp of a previous frame of a target video and a target timestamp of a current frame of the target video. The previous frame and the current frame are any two adjacently captured video frames, and the previous frame is captured before the current frame. The reference timestamp and the target timestamp both refer to the capture timestamp of the video frame.

[0090] S202: When the target timestamp is less than or equal to the reference timestamp, the product of the inverse of the frame rate of the target video and the first coefficient is used as the first time increment. The first coefficient is a positive number less than 1, the first time increment is a positive number, and the first time increment is negatively correlated with the frame rate.

[0091] Alternatively, the first time increment can be expressed as:

[0092] Among them, fps is the frame rate of the target video; A is the first coefficient, 0 <A<1。

[0093] S203, iteratively correcting the target timestamp by using the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp.

[0094] The ideal time interval between any two adjacent captured video frames based on the target video is 1 / frame rate. In this embodiment, the first time increment is a positive number less than the ideal time interval, that is, the first time increment is a smaller positive number. In order to ensure that when the target timestamp is corrected by the sum of the target timestamp and the first time increment, the corrected target timestamp can be greater than the reference timestamp, an iterative correction method can be adopted. That is, the first time increment is added on the basis of the initial target timestamp to obtain the first corrected target timestamp; continue to judge whether the first corrected target timestamp is greater than the reference timestamp; if it is still less than or equal to the reference timestamp, the first time increment is added on the basis of the first corrected target timestamp to obtain the second corrected target timestamp; continue to judge whether the second corrected target timestamp is greater than the reference timestamp. Continue the above process until the corrected target timestamp meets the expectation and the target timestamp is greater than the reference timestamp, so that the target timestamp of the current frame is more accurate and the acquisition timestamp of the video frame is more accurate.

[0095] In this embodiment, the first time increment is set to a small positive number, and the target timestamp is iteratively corrected by the sum of the target timestamp and the first time increment, which not only makes the target timestamp greater than the reference timestamp, but also makes the acquisition timestamp of the video frame more accurate. At the same time, through micro-refined corrections multiple times, the actual time interval formed by the target timestamp and the reference timestamp can be effectively controlled to be more reasonable. In the case where the target timestamp is greater than the reference timestamp, if the actual time interval is too small, the target timestamp can be continuously corrected until the actual time interval is close to the ideal time interval; if the actual time interval has exceeded the ideal time interval, the correction of the target timestamp is terminated, so that the actual time interval is still close to the ideal time interval, thereby making the acquisition timestamp of the video frame more accurate.

[0096] In one embodiment, if Figure 3 As shown, the timestamp correction method further includes the following steps S301 to S305. Among them, steps S301 to S303 correspond to steps S101 to S103 in the aforementioned embodiment. The specific discussion of steps S301 to S303 in this embodiment can refer to the specific discussion of steps S101 to S103 in the aforementioned embodiment, which will not be repeated in this embodiment.

[0097] S301, respectively determining a reference timestamp of a previous frame of a target video and a target timestamp of a current frame of the target video. The previous frame and the current frame are any two adjacently captured video frames, and the previous frame is captured before the current frame. The reference timestamp and the target timestamp both refer to the capture timestamp of the video frame.

[0098] S302: When the target timestamp is less than or equal to the reference timestamp, determine a first time increment based on the frame rate of the target video. The first time increment is a positive number, and the first time increment is negatively correlated with the frame rate.

[0099] S303: Correct the target timestamp by the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp.

[0100] S304, determining the time stamp to be corrected of the target frame of the target video. The target frame is any video frame captured after the current frame, and the time stamp to be corrected is the capture time stamp of the target frame.

[0101] S305: Correct the time stamp to be corrected by using the sum of the time stamp to be corrected and the first time increment.

[0102] Since the acquisition timestamp of the video frame collected later is incremented based on the acquisition timestamp of the video frame collected earlier, in theory, the time stamp to be corrected of the target frame is determined based on the target timestamp before the current frame is corrected. For example, when the theoretical time interval is 10 milliseconds, the reference timestamp, the target timestamp, and the time stamp to be corrected are 40 milliseconds, 39 milliseconds, and 49 milliseconds, respectively. In the case where the target timestamp is wrong, the time stamp to be corrected will also be wrong. Based on this, after correcting the target timestamp of the current frame, it is necessary to postpone the correction of the time stamp to be corrected of the target frame collected after the current frame based on the correction amplitude of the target timestamp. For example, when the first time increment is 10 milliseconds, the target timestamp is corrected from 39 milliseconds to 49 milliseconds, and the time stamp to be corrected is corrected by postponing the correction with the first time increment, so that the time stamp to be corrected is added with the first time increment, so that the time stamp to be corrected is corrected from 49 milliseconds to 59 milliseconds. It should be understood that the target frame is any video frame captured after the current frame. Even if the target frame and the current frame are not two adjacently captured video frames, the method for correcting the to-be-corrected timestamp of the target frame is still the same as above.

[0103] It can be clearly known from the above examples that based on the correction amplitude of the target timestamp, the timestamp to be corrected of the target frame collected after the current frame is corrected in a postponed manner, that is, the timestamp to be corrected is corrected by the sum of the timestamp to be corrected and the first time increment, so that the timestamp to be corrected of the target frame can be made more reasonable. Furthermore, when determining whether the timestamp to be corrected of the target frame is greater than the acquisition timestamp of the adjacent video frame, it is possible to determine with a higher probability that the timestamp to be corrected is greater than the acquisition timestamp of the adjacent video frame, and this adjacent video frame refers to the video frame collected before the target frame and collected adjacent to the target frame. If the timestamp to be corrected of the target frame is not corrected in a postponed manner, in the above example, when determining whether the timestamp to be corrected of the target frame is greater than the acquisition timestamp of the adjacent video frame, the adjacent video frame of the target frame is the current frame, and the acquisition timestamp of the adjacent video frame has been corrected to 49 milliseconds, but the timestamp to be corrected of the target frame is still 49 milliseconds. Obviously, in this case, it is more likely to determine that the timestamp to be corrected is less than or equal to the acquisition timestamp of the adjacent video frame. At this time, the timestamp to be corrected needs to be further corrected to make it more accurate.

[0104] Exemplarily, when the target timestamp is iteratively corrected by the sum of the target timestamp and the first time increment so that the target timestamp is greater than the reference timestamp, the timestamp to be corrected is corrected by the sum of the timestamp to be corrected and the first time increment, which means that the timestamp to be corrected is iteratively corrected by the sum of the timestamp to be corrected and the first time increment. The number of times the timestamp to be corrected is iteratively corrected may be the same as the number of times the target timestamp is iteratively corrected.

[0105] In this embodiment, while making the target timestamp of the current frame more accurate, the timestamp to be corrected of the target frame collected after the current frame is corrected in a delayed manner based on the correction amplitude of the target timestamp, that is, the timestamp to be corrected is corrected by the sum of the timestamp to be corrected and the first time increment, which can make the timestamp to be corrected of the target frame more reasonable. It can reduce the probability of determining that the timestamp to be corrected is less than or equal to the acquisition timestamp of the adjacent video frame when determining whether the timestamp to be corrected of the target frame is greater than the acquisition timestamp of the adjacent video frame, can reduce the frequency of correction of the acquisition timestamp of the video frame, and can save resources.

[0106] In one embodiment, if Figure 4 As shown, the timestamp correction method further includes the following steps S401 to S406. Among them, steps S401 to S403 correspond to steps S101 to S103 in the aforementioned embodiment. The specific discussion of steps S401 to S403 in this embodiment can refer to the specific discussion of steps S101 to S103 in the aforementioned embodiment, which will not be repeated in this embodiment.

[0107] S401, respectively determine a reference timestamp of a previous frame of a target video and a target timestamp of a current frame of the target video. The previous frame and the current frame are any two adjacent captured video frames, and the previous frame is captured before the current frame. The reference timestamp and the target timestamp both refer to the capture timestamp of the video frame.

[0108] S402: When the target timestamp is less than or equal to the reference timestamp, determine a first time increment based on the frame rate of the target video. The first time increment is a positive number, and the first time increment is negatively correlated with the frame rate.

[0109] S403: Correct the target timestamp by the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp.

[0110] S404: When the target timestamp is greater than the reference timestamp, determine a time interval formed by the target timestamp and the reference timestamp.

[0111] As discussed above, an error in the acquisition timestamp of a video frame can easily lead to an error in the decoding timestamp and display timestamp of the video frame, which can easily lead to abnormal situations such as freeze, black screen, acceleration, and screen distortion in video playback. The timestamp correction method in the aforementioned embodiment corrects the target timestamp of the current frame for the error situation that the acquisition timestamps of two adjacent video frames do not increase, that is, the target timestamp of the current frame is less than or equal to the reference timestamp of the previous frame, so as to make the acquisition timestamp of the video frame more accurate. In fact, the acquisition timestamp error of the video frame includes, in addition to the acquisition timestamps of two adjacent video frames not increasing, the unreasonable time interval formed by the acquisition timestamps when the acquisition timestamps of two adjacent video frames increase. That is, it also includes the time interval being too long or too short. Different from the timestamp correction method in the aforementioned embodiment, the timestamp correction method in this embodiment further corrects the target timestamp of the current frame for the error situation that the time interval formed by the acquisition timestamp is unreasonable, so as to further improve the accuracy of the acquisition timestamp of the video frame and further reduce the abnormal situations that occur during video playback.

[0112] First, when the target timestamp is greater than the reference timestamp, determine the time interval formed by the target timestamp and the reference timestamp. It should be noted that the target timestamp that is greater than the reference timestamp at this time may be a target timestamp that is greater than the reference timestamp without correction, or a target timestamp that is initially less than or equal to the reference timestamp and is greater than the reference timestamp after correction. When the target timestamp is greater than the reference timestamp after correction, it may be a target timestamp obtained by iterative correction based on the first time increment.

[0113] Alternatively, the time interval can be determined by the following formula:

[0114] Δtimestamp=timestamp2-timestamp1

[0115] Among them, Δtimestamp is the time interval, timestamp2 is the target timestamp, and timestamp1 is the reference timestamp.

[0116] S405: If the time interval does not belong to the preset interval, determine a second time increment based on the frame rate of the target video. The preset interval is a standard for determining whether the duration of the time interval is reasonable.

[0117] The preset interval is a standard for determining whether the duration of the time interval is reasonable. Based on this, the preset interval can be set as an interval that includes the ideal time interval. It can be understood that the closer the maximum boundary value and the minimum boundary value of the preset interval are to the ideal time interval, the closer the duration of the time interval in the preset interval is to the ideal time interval, that is, the more reasonable it is. Furthermore, the more reasonable the target timestamp of the current frame is, the more accurate the acquisition timestamp of the video is.

[0118] The ideal time interval of the target video is You can set the preset interval to

[0119] Wherein, fps is the frame rate of the target video, B is a positive number less than 1, and the preset interval can be understood as the range from the ideal time interval as the midpoint, minus B times the ideal time interval to plus B times the ideal time interval. It can be understood that the smaller B is, the closer the time interval in the preset interval is to the ideal time interval.

[0120] S406: Correct the target timestamp by using the sum of the target timestamp and the second time increment, so that the time interval formed by the target timestamp and the reference timestamp belongs to a preset interval.

[0121] When the time interval formed by the target timestamp and the reference timestamp does not belong to the preset interval, it indicates that the time interval is unreasonable. At this time, the target timestamp of the current frame is corrected. Specifically, the second time increment is determined based on the frame rate of the target video, and the target timestamp is corrected by the sum of the target timestamp and the second time increment, so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval and meets the reasonable standard.

[0122] In this embodiment, when the target timestamp is greater than the reference timestamp, if the time interval formed by the target timestamp and the reference timestamp does not belong to the preset interval, the target timestamp is further corrected to make the time interval belong to the preset interval. This can make the actual time interval formed by the target timestamp and the reference timestamp closer to the ideal time interval, making the time interval more reasonable, and thus making the target timestamp of the current frame more reasonable and accurate, and further improving the accuracy of the video acquisition timestamp.

[0123] In one of the embodiments, step S405, when the time interval does not belong to the preset interval range, determines the second time increment based on the frame rate of the target video, including the following steps: when the time interval is less than the minimum boundary value of the preset interval range, determines the second time increment based on the frame rate of the target video, the second time increment is a positive number, and the second time increment is negatively correlated with the frame rate.

[0124] When the time interval is less than the minimum boundary value of the preset interval, it indicates that the time interval is too small, that is, the target timestamp is too small. At this time, the target timestamp needs to be increased to make the target timestamp more reasonable and accurate. Specifically, a second time increment is determined as a positive number based on the frame rate of the target video, and the second time increment is negatively correlated with the frame rate. Then, the target timestamp is corrected by the sum of the target timestamp and the second time increment, so that the time interval belongs to the preset interval, the target timestamp is increased, and the time interval formed by the target timestamp and the reference timestamp is increased.

[0125] Optionally, the target timestamp may be corrected by the following formula when the time interval is less than the minimum boundary value of the preset interval:

[0126]

[0127] Among them, timestamp is the target timestamp before correction; timestamp fix is the corrected target timestamp; Increment the second time, where fps is the frame rate of the target video and C is a positive number.

[0128] The purpose is to make the time interval formed by the corrected target timestamp and the reference video timestamp fall within the preset interval, and C can be set according to actual needs and business scenarios.

[0129] In addition, since the ideal time interval between any two adjacent captured video frames of the target video is 1 / frame rate. Therefore, the second time increment is determined based on the frame rate of the target video, and the second time increment is negatively correlated with the frame rate. It can be understood that the timestamp correction method determines the second time increment based on the ideal time interval. Through this technical approach, when correcting the target timestamp based on the second time increment, it can not only achieve that the time interval belongs to the preset interval interval, but also that the actual time interval formed by the target timestamp obtained after correction and the reference timestamp is related to the ideal time interval, which is also conducive to making the actual time interval more reasonable. Furthermore, it is conducive to making the target timestamp obtained after correction closer to the ideal target timestamp, and is conducive to improving the rationality and accuracy of the target timestamp of the current frame.

[0130] In this embodiment, when the target timestamp is greater than the reference timestamp, and the time interval formed by the target timestamp and the reference timestamp is less than the minimum boundary value of the preset interval, the second time increment determined as a positive number by the frame rate of the target video is used to correct the target timestamp, so as to increase the target timestamp and increase the time interval formed by the target timestamp and the reference timestamp. The rationality and accuracy of the target timestamp of the current frame can be improved, so that the acquisition timestamps of any two adjacent video frames are increased and increased reasonably and stably, making the acquisition timestamp of the video frame more accurate. Furthermore, it is conducive to accurately determining the decoding timestamp and display timestamp of the video frame, ensuring that the video frame is decoded and played normally, reducing the possibility of abnormal video playback due to errors in the acquisition timestamp of the video frame, and reducing abnormal situations such as black screen, acceleration, freeze, black screen, etc. during video playback.

[0131] In one embodiment, if Figure 5 As shown, the timestamp correction method further includes the following steps S501 to S506. Among them, steps S501 to S504 correspond to steps S401 to S404 in the aforementioned embodiment. The specific discussion of steps S501 to S504 in this embodiment can refer to the specific discussion of steps S401 to S404 in the aforementioned embodiment, which will not be repeated in this embodiment.

[0132] S501, respectively determine a reference timestamp of a previous frame of a target video and a target timestamp of a current frame of the target video. The previous frame and the current frame are any two adjacently captured video frames, and the previous frame is captured before the current frame. The reference timestamp and the target timestamp both refer to the capture timestamp of the video frame.

[0133] S502: When the target timestamp is less than or equal to the reference timestamp, determine a first time increment based on the frame rate of the target video. The first time increment is a positive number, and the first time increment is negatively correlated with the frame rate.

[0134] S503: Correct the target timestamp by using the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp.

[0135] S504: When the target timestamp is greater than the reference timestamp, determine the time interval formed by the target timestamp and the reference timestamp.

[0136] S505: When the time interval is less than the minimum boundary value of the preset interval, the product of the reciprocal of the frame rate of the target video and the second coefficient is used as the second time increment. The second coefficient is a positive number less than 1, the second time increment is a positive number, and the second time increment is negatively correlated with the frame rate. The preset interval is a standard for determining whether the duration of the time interval is reasonable.

[0137] Alternatively, the second time increment may be expressed as:

[0138] Among them, fps is the frame rate of the target video; C is the second coefficient, 0 <C<1。

[0139] S506: Iteratively correct the target timestamp using the sum of the target timestamp and the second time increment, so that the time interval formed by the target timestamp and the reference timestamp belongs to a preset interval.

[0140] The ideal time interval between any two adjacent captured video frames of the target video is 1 / frame rate. In this embodiment, the second time increment is a positive number less than the ideal time interval, that is, the second time increment is a smaller positive number. In order to ensure that when the target timestamp is corrected by the sum of the target timestamp and the second time increment, the time interval formed by the corrected target timestamp and the reference timestamp can be increased, and the time interval can be greater than the minimum boundary value of the preset interval interval and belong to the preset interval interval, an iterative correction method can be adopted. That is, the second time increment is added on the basis of the initial target timestamp to obtain the first corrected target timestamp; continue to judge whether the time interval formed by the first corrected target timestamp and the reference timestamp belongs to the preset interval interval; if the time interval is still less than the minimum boundary value of the preset interval interval, then the second time increment is added on the basis of the first corrected target timestamp to obtain the second corrected target timestamp; continue to judge whether the time interval formed by the second corrected target timestamp and the reference timestamp belongs to the preset interval interval. The above process is continued until the time interval formed by the corrected target timestamp and the reference timestamp is greater than the minimum boundary value of the preset interval and belongs to the preset interval, so that the target timestamp of the current frame is more reasonable and accurate, and the acquisition timestamp of the video frame is more accurate.

[0141] In this embodiment, the second time increment is set to a small positive number, and the target timestamp is iteratively corrected by the sum of the target timestamp and the second time increment, which can not only achieve that the time interval formed by the target timestamp and the reference time is greater than the minimum boundary value of the preset interval, but also achieve that the time interval belongs to the preset interval, so that the target timestamp of the current frame is more reasonable and more accurate, and the acquisition timestamp of the video frame is more accurate. At the same time, through micro-multiple fine corrections, the time interval formed by the target timestamp and the reference timestamp can be effectively controlled to be reasonable, so that the time interval is greater than the minimum boundary value of the preset interval and less than or equal to the maximum boundary value of the preset interval. It can avoid the situation where the time interval formed by the corrected target timestamp and the reference timestamp is too large due to the excessive second time increment, and is greater than the maximum boundary value of the preset interval, and avoid excessive correction resulting in the time interval still not belonging to the preset interval.

[0142] In one of the embodiments, in parallel with “step S405, when the time interval does not belong to the preset interval interval, determining the second time increment based on the frame rate of the target video, comprising the following steps: when the time interval is less than the minimum boundary value of the preset interval interval, determining the second time increment based on the frame rate of the target video, the second time increment is a positive number, and the second time increment is negatively correlated with the frame rate”, step S405, when the time interval does not belong to the preset interval interval, determining the second time increment based on the frame rate of the target video, comprises the following steps: when the time interval is greater than the maximum boundary value of the preset interval interval, determining the second time increment based on the frame rate of the target video, the second time increment is a negative number, and the absolute value of the second time increment is negatively correlated with the frame rate.

[0143] When the time interval is greater than the maximum boundary value of the preset interval, it indicates that the time interval is too large, that is, the target timestamp is too large. In this case, the target timestamp needs to be reduced to make the target timestamp more reasonable and accurate. Specifically, a second time increment is determined as a negative number based on the frame rate of the target video, and the absolute value of the second time increment is negatively correlated with the frame rate. Then, the target timestamp is corrected by the sum of the target timestamp and the second time increment, so that the time interval belongs to the preset interval, the target timestamp is reduced, and the time interval formed by the target timestamp and the reference timestamp is reduced.

[0144] Optionally, the target timestamp may be corrected by the following formula when the time interval is greater than the maximum boundary value of the preset interval:

[0145]

[0146] Among them, timestamp is the target timestamp before correction; timestamp fix is the corrected target timestamp; Add value to the second time; is the absolute value of the second time increment, where fps is the frame rate of the target video and D is a positive number.

[0147] The purpose is to make the time interval formed by the corrected target timestamp and the reference video timestamp fall within the preset interval, and D can be set according to actual needs and business scenarios.

[0148] In addition, since the ideal time interval between any two adjacent captured video frames of the target video is 1 / frame rate. Therefore, the second time increment is determined based on the frame rate of the target video, and the absolute value of the second time increment is negatively correlated with the frame rate. It can be understood that the timestamp correction method determines the second time increment based on the ideal time interval. Through this technical approach, when correcting the target timestamp based on the second time increment, not only can the time interval be made to belong to the preset interval interval, but also the actual time interval formed by the corrected target timestamp and the reference timestamp can be related to the ideal time interval, which is also conducive to making the actual time interval more reasonable. Furthermore, it is conducive to making the corrected target timestamp closer to the ideal target timestamp, and is conducive to improving the rationality and accuracy of the target timestamp of the current frame.

[0149] In this embodiment, when the target timestamp is greater than the reference timestamp, and the time interval formed by the target timestamp and the reference timestamp is greater than the maximum boundary value of the preset interval, the second time increment determined as a negative number by the frame rate of the target video is used to correct the target timestamp, so as to make the time interval belong to the preset interval, reduce the target timestamp, and reduce the time interval formed by the target timestamp and the reference timestamp. It can improve the rationality and accuracy of the target timestamp of the current frame, so that the acquisition timestamps of any two adjacent video frames are increased and increased reasonably and stably, making the acquisition timestamp of the video frame more accurate. Furthermore, it is conducive to accurately determining the decoding timestamp and display timestamp of the video frame, ensuring that the video frame is decoded and played normally, reducing the possibility of abnormal video playback due to errors in the acquisition timestamp of the video frame, and reducing abnormal situations such as black screen, acceleration, freeze, black screen, etc. during video playback.

[0150] In one embodiment, if Figure 6 As shown, the timestamp correction method further includes the following steps S601 to S606. Among them, steps S601 to S604 correspond to steps S401 to S404 in the aforementioned embodiment. The specific discussion of steps S601 to S604 in this embodiment can refer to the specific discussion of steps S401 to S404 in the aforementioned embodiment, which will not be repeated in this embodiment.

[0151] S601, respectively determine a reference timestamp of a previous frame of a target video and a target timestamp of a current frame of the target video. The previous frame and the current frame are any two adjacently captured video frames, and the previous frame is captured before the current frame. The reference timestamp and the target timestamp both refer to the capture timestamp of the video frame.

[0152] S602: When the target timestamp is less than or equal to the reference timestamp, determine a first time increment based on the frame rate of the target video. The first time increment is a positive number, and the first time increment is negatively correlated with the frame rate.

[0153] S603: Correct the target timestamp by the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp.

[0154] S604: When the target timestamp is greater than the reference timestamp, determine the time interval formed by the target timestamp and the reference timestamp.

[0155] S605: When the time interval is greater than the maximum boundary value of the preset interval, the product of the inverse of the frame rate of the target video and the third coefficient is used as the absolute value of the second time increment, and the second time increment is a negative number. The third coefficient is a positive number less than 1, and the absolute value of the second time increment is negatively correlated with the frame rate. The preset interval is a standard for determining whether the duration of the time interval is reasonable.

[0156] Alternatively, the second time increment may be expressed as: The absolute value of the second increment can be expressed as:

[0157] Among them, fps is the frame rate of the target video; D is the third coefficient, 0 <D<1。

[0158] S606: Iteratively correct the target timestamp using the sum of the target timestamp and the second time increment, so that the time interval formed by the target timestamp and the reference timestamp belongs to a preset interval.

[0159] The ideal time interval between any two adjacent captured video frames of the target video is 1 / frame rate. In this embodiment, the absolute value of the second time increment is a positive number less than the ideal time interval, that is, the absolute value of the second time increment is a small positive number. The second time increment is a negative number. In order to ensure that when the target timestamp is corrected by the sum of the target timestamp and the second time increment, that is, when the target timestamp is corrected by the difference between the absolute values ​​of the target timestamp and the second time increment, the time interval formed by the corrected target timestamp and the reference timestamp can be reduced, and the time interval can be less than the maximum boundary value of the preset interval interval, and belongs to the preset interval interval, an iterative correction method can be adopted. That is, the second time increment is added on the basis of the initial target timestamp, that is, the absolute value of the second time increment is subtracted to obtain the target timestamp after the first correction; continue to judge whether the time interval formed by the target timestamp after the first correction and the reference timestamp belongs to the preset interval interval. If the time interval is still greater than the maximum boundary value of the preset interval interval, the second time increment is added on the basis of the target timestamp after the first correction to obtain the target timestamp after the second correction; continue to judge whether the time interval formed by the target timestamp after the second correction and the reference timestamp belongs to the preset interval interval. The above process is continued until the time interval formed by the corrected target timestamp and the reference timestamp is less than the maximum boundary value of the preset interval and belongs to the preset interval, so that the target timestamp of the current frame is more reasonable and accurate, and the acquisition timestamp of the video frame is more accurate.

[0160] In this embodiment, the absolute value of the second time increment is set to a small positive number, and the second time increment is a negative number. The target timestamp is iteratively corrected by the sum of the target timestamp and the second time increment. This can not only make the time interval formed by the target timestamp and the reference time smaller than the maximum boundary value of the preset interval, but also make the time interval belong to the preset interval, so that the target timestamp of the current frame is more reasonable and more accurate, and the acquisition timestamp of the video frame is more accurate. At the same time, through micro-multiple fine corrections, the time interval formed by the target timestamp and the reference timestamp can be effectively controlled to be reasonable, so that the time interval is smaller than the maximum boundary value of the preset interval and greater than or equal to the minimum boundary value of the preset interval. It can avoid the situation where the time interval formed by the corrected target timestamp and the reference timestamp is too small and smaller than the minimum boundary value of the preset interval due to the excessive absolute value of the second time increment, and avoid excessive correction resulting in the time interval still not belonging to the preset interval.

[0161] In one embodiment, if Figure 7As shown, the timestamp correction method further includes the following steps S701 to S708. Among them, steps S701 to S706 correspond to steps S401 to S406 in the aforementioned embodiment. The specific discussion of steps S701 to S708 in this embodiment can refer to the specific discussion of steps S401 to S406 in the aforementioned embodiment, which will not be repeated in this embodiment.

[0162] S701, respectively determine a reference timestamp of a previous frame of a target video and a target timestamp of a current frame of the target video. The previous frame and the current frame are any two adjacently captured video frames, and the previous frame is captured before the current frame. The reference timestamp and the target timestamp both refer to the capture timestamp of the video frame.

[0163] S702: When the target timestamp is less than or equal to the reference timestamp, determine a first time increment based on the frame rate of the target video. The first time increment is a positive number, and the first time increment is negatively correlated with the frame rate.

[0164] S703: Correct the target timestamp by the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp.

[0165] S704: When the target timestamp is greater than the reference timestamp, determine a time interval formed by the target timestamp and the reference timestamp.

[0166] S705: If the time interval does not belong to the preset interval, determine a second time increment based on the frame rate of the target video. The preset interval is a standard for determining whether the duration of the time interval is reasonable.

[0167] S706: Correct the target timestamp by using the sum of the target timestamp and the second time increment, so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval.

[0168] S707, determining the timestamp to be corrected of the target frame of the target video. The target frame is any video frame captured after the current frame, and the timestamp to be corrected is the capture timestamp of the target frame;

[0169] S708, the time stamp to be corrected is corrected by the sum of the time stamp to be corrected and the target time increment. The target time increment is the second time increment or the sum of the first time increment and the second time increment.

[0170] Since the acquisition timestamp of the video frame collected later is incremented based on the acquisition timestamp of the video frame collected earlier, theoretically, the time stamp to be corrected of the target frame is determined based on the target timestamp before the current frame is corrected. For example, when the theoretical time interval is 10 milliseconds, the reference timestamp, the target timestamp, and the time stamp to be corrected are 40 milliseconds, 39 milliseconds, and 49 milliseconds, respectively. In the case where the target timestamp is wrong, the time stamp to be corrected will also be wrong. Based on this, after correcting the target timestamp of the current frame, it is necessary to postpone the correction of the time stamp to be corrected of the target frame collected after the current frame based on the correction amplitude of the target timestamp. For example, when the correction amplitude is 10 milliseconds, the target timestamp is corrected from 39 milliseconds to 49 milliseconds, and the time stamp to be corrected is corrected by the correction amplitude, so that the time stamp to be corrected is corrected from 49 milliseconds to 59 milliseconds by adding the correction amplitude. It should be understood that the target frame is any video frame collected after the current frame, even if the target frame and the current frame are not two adjacent video frames collected, the correction method for the time stamp to be corrected of the target frame is still the same as above.

[0171] It can be clearly known from the above examples that based on the correction amplitude of the target timestamp, the correction of the target frame to be corrected after the current frame is postponed, that is, the timestamp to be corrected is corrected by the sum of the timestamp to be corrected and the target time increment, which can make the timestamp to be corrected of the target frame more reasonable, wherein the target time increment is used to characterize the correction amplitude of the target timestamp. Furthermore, when determining whether the timestamp to be corrected of the target frame is greater than the acquisition timestamp of the adjacent video frame, it is possible to determine with a higher probability that the timestamp to be corrected is greater than the acquisition timestamp of the adjacent video frame, and this adjacent video frame refers to the video frame collected before the target frame and collected adjacent to the target frame. If the correction of the timestamp to be corrected of the target frame is not postponed, in the above example, when determining whether the timestamp to be corrected of the target frame is greater than the acquisition timestamp of the adjacent video frame, the adjacent video frame of the target frame is the current frame, and the acquisition timestamp of the adjacent video frame has been corrected to 49 milliseconds, but the timestamp to be corrected of the target frame is still 49 milliseconds. Obviously, in this case, it is more likely to determine that the timestamp to be corrected is less than or equal to the acquisition timestamp of the adjacent video frame. At this time, the timestamp to be corrected needs to be further corrected to make it more accurate.

[0172] It should be understood that the target timestamp in step S704 "when the target timestamp is greater than the reference timestamp" may be the first type of target timestamp, that is, the target timestamp that is greater than the reference timestamp without the first time increment correction; or it may be the second type of target timestamp, that is, the target timestamp that is initially less than or equal to the reference timestamp and is greater than the reference timestamp after the first time increment correction. If it is the first type of target timestamp, the target timestamp in step S706 "so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval" is the target timestamp obtained only after the second time increment correction. At this time, the correction amplitude of the target timestamp is to increase the second time increment. Based on this, when the time stamp to be corrected is corrected by the sum of the time stamp to be corrected and the target time increment, the target time increment is the second time increment. If it is the second type of target timestamp, the target timestamp in step S706 "so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval" is the target timestamp obtained after the first time increment correction and then the second time increment correction. At this time, the correction range of the target timestamp is the sum of the first time increment and the second time increment. Based on this, when the time stamp to be corrected is corrected by the sum of the time stamp to be corrected and the target time increment, the target time increment is the sum of the first time increment and the second time increment.

[0173] Exemplarily, the purpose is to make the target timestamp to be corrected have the same correction amplitude as the target timestamp of the current frame. When the target timestamp is iteratively corrected with the first time increment, the timestamp to be corrected also needs to be iteratively corrected with the first time increment. When the target timestamp is iteratively corrected with the second time increment, the timestamp to be corrected also needs to be iteratively corrected with the second time increment. The number of times the timestamp to be corrected is iteratively corrected can be the same as the number of times the target timestamp is iteratively corrected. Specifically, for example, when the target time increment is the sum of the first time increment and the second time increment, and the target timestamp is iteratively corrected with the sum of the target timestamp and the first time increment to obtain the corrected target timestamp, and then the target timestamp is iteratively corrected with the sum of the corrected target timestamp and the second time increment to obtain the final corrected target timestamp, then the timestamp to be corrected also needs to be iteratively corrected with the sum of the timestamp to be corrected and the first time increment to obtain the corrected timestamp to be corrected, and then the timestamp to be corrected is iteratively corrected with the sum of the corrected timestamp to be corrected and the second time increment to obtain the final corrected timestamp to be corrected. The timestamp to be corrected undergoes the same number of first time increment iteration corrections as the target timestamp, and the timestamp to be corrected undergoes the same number of second time increment iteration corrections as the target timestamp.

[0174] In this embodiment, while making the target timestamp of the current frame more accurate, the timestamp of the target frame to be corrected collected after the current frame is corrected in a delayed manner based on the correction amplitude of the target timestamp, so that the timestamp of the target frame to be corrected can be made more reasonable. This can reduce the probability of determining that the timestamp to be corrected is less than or equal to the acquisition timestamp of the adjacent video frame when determining whether the timestamp to be corrected of the target frame is greater than the acquisition timestamp of the adjacent video frame, and can reduce the frequency of correction of the acquisition timestamp of the video frame, thereby saving resources.

[0175] In one of the embodiments, in any of the aforementioned embodiments, before respectively determining the reference timestamp of the previous frame of the target video and the target timestamp of the current frame of the target video in step S101, the timestamp correction method further includes the following steps: performing target processing on the original video to obtain the target video, the target processing including any one of acquisition processing and pre-processing.

[0176] It can be understood that when the target video is a video obtained after target processing is performed, the previous frame, current frame, and target frame described in the aforementioned embodiments are all video frames in the target video obtained after target processing is performed.

[0177] In the process of collecting and processing the video and marking the acquisition timestamp of the video frame, the reference clock may be unstable, resulting in errors in the acquired acquisition timestamp, errors in marking, and other factors, resulting in errors in the acquisition timestamp of the video frame. In the process of pre-processing the video, when the pixel level of the video frame is adjusted, the acquisition timestamp originally stored in the data information may be modified or overwritten with new timestamp information, resulting in errors in the acquisition timestamp of the video frame. Based on this, when the original video is collected or pre-processed to obtain the target video, it is necessary to promptly judge whether the acquisition timestamp of the video frame of the target video is wrong. And when it is found that the acquisition timestamp of the video frame is wrong, such as the acquisition timestamps of the two adjacent video frames collected are not increasing, or the time interval when increasing is unreasonable, the acquisition timestamp of the video frame is promptly corrected to make the acquisition timestamp of the video frame more accurate. It can be understood that this timestamp correction method can correct the acquisition timestamp of the video frame in time after the original video is captured and processed, and can also correct the acquisition timestamp of the video frame in time after the original video is pre-processed, so as to discover and solve problems more promptly.

[0178] In this embodiment, when the original video is collected or pre-processed to obtain the target video, it is timely judged whether the acquisition timestamp of the target video is wrong, and the acquisition timestamp of the video frame is corrected in time to make the acquisition timestamp of the video frame more accurate, which is conducive to preventing the error of the acquisition timestamp from being further expanded in the subsequent processing link, and is conducive to preventing more errors in the acquisition timestamp. After the original video is collected and pre-processed, the original video is encoded. Based on this, before the original video is encoded, the acquisition timestamp is corrected in time to make the acquisition timestamp of the video frame more accurate, which is conducive to adaptively determining a more accurate decoding timestamp and display timestamp based on the more accurate acquisition timestamp of the video frame during the encoding process. The video frame can be decoded and displayed in an orderly and stable manner according to the more accurate decoding timestamp and display timestamp, which is conducive to reducing abnormal situations during video playback.

[0179] It is worth noting that, since the decoding timestamp and display timestamp of the video frame have been determined and marked during the encoding process, after the original video is encoded, it is judged whether the acquisition timestamp of the video frame is wrong, and if the acquisition timestamp is wrong, the acquisition timestamp is corrected. At this time, it is difficult to adaptively determine a more accurate decoding timestamp and display timestamp based on a more accurate acquisition timestamp. Based on this, in this embodiment, the target processing is set as the acquisition processing or pre-processing before the encoding processing, so as to adaptively determine a more accurate decoding timestamp and display timestamp based on the more accurate acquisition timestamp after correction. It is feasible to timely judge whether the acquisition timestamp of the video frame is wrong after the original video is encoded, and if it is determined that there is an error, the marked decoding timestamp and display timestamp are corrected in time based on the specific circumstances of the error, so as to obtain a more accurate decoding timestamp and display timestamp.

[0180] It is also worth noting that this timestamp correction method can not only be applied to the terminal, but also be used as the execution subject to correct the acquisition timestamp of the video frame of the target video obtained after acquisition processing or pre-processing, so as to make the acquisition timestamp of the video frame more accurate, thereby reducing abnormal situations such as black screen, freeze, acceleration, and screen distortion during video playback. This timestamp correction method can also be applied to the corresponding module in the terminal that performs target processing on the original video to obtain the target video. For example, when the target video is a video obtained by acquisition processing of the original video, the execution subject of this timestamp correction method can be the corresponding module in the terminal that performs acquisition processing; when the target video is a video obtained by pre-processing the original video, the execution subject of this timestamp correction method can be the corresponding module in the terminal that performs pre-processing. After obtaining the target video, the corresponding module that performs target processing corrects the acquisition timestamp of the video frame of the target video to ensure that the acquisition timestamp of the video frame of the output target video is more accurate.

[0181] The present application also provides a video frame display method applied to a live broadcast terminal. Figure 8 As shown, the video frame display method includes the following steps S801 to S805.

[0182] S801, respectively determine a reference timestamp of a previous frame of a target video and a target timestamp of a current frame of the target video. The previous frame and the current frame are any two adjacently captured video frames, and the previous frame is captured before the current frame. The reference timestamp and the target timestamp both refer to the capture timestamp of the video frame.

[0183] S802: When the target timestamp is less than or equal to the reference timestamp, determine a first time increment based on the frame rate of the target video. The first time increment is a positive number, and the first time increment is negatively correlated with the frame rate.

[0184] S803: Correct the target timestamp by the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp.

[0185] The live broadcast terminal executes the video frame display method in this embodiment, and first corrects the target timestamp of the current frame by the timestamp correction method described in the aforementioned embodiment to make the target timestamp of the current frame more accurate.

[0186] It should be noted that for the specific description of steps S801 to S803 in the embodiment of the present application, reference may be made to the specific description of steps S101 to S103 in the aforementioned timestamp correction method embodiment, wherein steps S801 to S803 correspond to steps S101 to S103 one by one, and will not be repeated in this embodiment.

[0187] S804: When the target timestamp is greater than the reference timestamp, determine a decoding timestamp and a display timestamp of the current frame based on the target timestamp.

[0188] When the target timestamp is greater than the reference timestamp, the target timestamp of the current frame is more accurate. It can be understood that the target timestamp that is greater than the reference timestamp at this time may be a target timestamp that is not corrected and is already greater than the reference timestamp, or it may be a target timestamp that is initially less than or equal to the reference timestamp and is greater than the reference timestamp after correction. When the target timestamp is more accurate, the live broadcast terminal determines the decoding timestamp and display timestamp of the current frame based on the more accurate target timestamp. The decoding timestamp and display timestamp obtained at this time are also more accurate. Since the decoding timestamp refers to the time information for decoding the current frame, the display timestamp refers to the time information for displaying the decoded current frame. Therefore, accurate decoding timestamps and display timestamps are conducive to ensuring that the current frame and other video frames are decoded and played normally, and are conducive to reducing abnormal situations during video playback. For example, if the decoding timestamp is too small, the current frame needs to be decoded earlier. At this time, the current frame may not be decoded because the reference frame that the current frame needs to refer to for decoding has not been decoded, which may affect the normal decoding of other video frames. When an error occurs in video frame decoding, the decoded video frame is difficult to play normally, and the video playback may experience abnormalities such as freeze, black screen, acceleration, and screen distortion. For example, if the display timestamp is too large, the decoded current frame will be displayed later, and the video playback may also experience abnormalities such as freeze.

[0189] It should be noted that the video frame display method in this embodiment first improves the accuracy of the target timestamp of the current frame by the method described in any of the above-mentioned timestamp correction method embodiments, and then determines the more accurate decoding timestamp and display timestamp of the current frame based on the more accurate target timestamp. Based on this, it should be understood that the video frame display method in this embodiment can also determine the time interval formed by the target timestamp and the reference timestamp before determining the decoding timestamp and display timestamp of the current frame based on the target timestamp when the target timestamp is greater than the reference timestamp; when the time interval does not belong to the preset interval interval, determine the second time increment based on the frame rate of the target video; and correct the target timestamp with the sum of the target timestamp and the second time increment so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval interval. That is, before determining the decoding timestamp and display timestamp of the current frame based on the target timestamp, steps S404 to S406 in the above-mentioned timestamp correction method embodiment can also be performed. Of course, the specific discussion on steps S404 to S406 in the above-mentioned timestamp correction method embodiment can also be applied to this embodiment, and will not be repeated in this embodiment. It should be emphasized that when the target timestamp is greater than the reference timestamp, the live broadcast terminal determines the time interval formed by the target timestamp and the reference timestamp to further correct the target timestamp. When the time interval formed by the target timestamp and the reference timestamp belongs to a preset interval, the live broadcast terminal determines the decoding timestamp and display timestamp of the current frame based on the target timestamp.

[0190] S805: Send the current frame and the decoding timestamp and display timestamp of the current frame to the service server in the live link. The decoding timestamp refers to the time information of decoding the current frame, and the display timestamp refers to the time information of displaying the decoded current frame.

[0191] The business server in the live broadcast link is mainly responsible for receiving the live broadcast data pushed by the live broadcast push side, and performing a series of processing to meet various needs of the live broadcast. Specifically, the functions of the business server include transcoding the live broadcast data into data streams with different bit rates for user terminals to choose, transmitting the data streams to user terminals in accordance with the streaming media transmission protocol, recording and storing the live broadcast content, and performing load balancing. The business server is one of the servers in the live broadcast link. The servers in the live broadcast link also include database servers, message servers, and streaming media servers. The collaborative work of multiple servers facilitates the construction and operation of the live broadcast link.

[0192] The video frame display method in this embodiment can be applied to Fig. 9 The video frame shown in the figure shows the live broadcast terminal in the system. Fig. 9As shown in (a), the live broadcast terminal on the live streaming side sends the current frame and a more accurate decoding timestamp and display timestamp of the current frame to the business server, so that the business server can send the current frame and a more accurate decoding timestamp and display timestamp to the user terminal. After the user terminal obtains the current frame and a more accurate decoding timestamp and display timestamp from the business server, it decodes the current frame in an orderly and stable manner based on the more accurate decoding timestamp, and renders and displays the decoded current frame in an orderly and stable manner based on the more accurate display timestamp. It is beneficial to reduce abnormal situations during video frame display and video playback, and is beneficial to improving the live broadcast effect. It should be noted that in this embodiment, in order to highlight the role of the business server in the transmission process of live broadcast data such as the current frame from the live broadcast terminal to the user terminal, only the business server is shown in the video frame display system. But in fact, in addition to the business server, the video frame display system also includes other types of servers such as database servers and streaming media servers.

[0193] Optionally, the process of the video frame display system from capturing live video to playing the video can be as follows: Fig. 9 As shown in (b), the video frame display system includes a live broadcast terminal, a service server and a user terminal. The live broadcast terminal, the service server and the user terminal respectively process the collected live broadcast video as follows:

[0194] From the start of the live broadcast by the anchor, the live broadcast terminal performs video acquisition and pre-processes the acquired live video. Pre-processing includes beauty, adding filters, noise reduction, etc. Before encoding the video, the live broadcast terminal can determine at any time whether the acquisition timestamp of the video frame of the video is wrong, and the video at the time of judgment is the aforementioned target video. If the acquisition timestamp of the video frame is wrong, the acquisition timestamp of the video frame is corrected to improve the accuracy of the acquisition timestamp of the video frame. Optionally, the live broadcast video is used as the aforementioned original video, and the acquired live broadcast video is used as the aforementioned target video. After the video is acquired, the live broadcast terminal determines whether the acquisition timestamp of the video frame of the target video is wrong, and corrects the acquisition timestamp in time if there is an error. Optionally, the live broadcast video is used as the aforementioned original video, and the pre-processed live broadcast video is used as the aforementioned target video. After the pre-processing, the live broadcast terminal determines whether the acquisition timestamp of the video frame of the target video is wrong, and corrects the acquisition timestamp in time if there is an error. By correcting the acquisition timestamp in time after the corresponding target is processed, the error of the acquisition timestamp can be prevented from being further expanded in the subsequent processing link, which is conducive to preventing more errors in the acquisition timestamp. The live broadcast terminal performs video encoding on the pre-processed video to generate code stream data. During the encoding process, a more accurate decoding timestamp and display timestamp of the video frame are determined and marked based on the more accurate acquisition timestamp. Then, the live broadcast terminal packages and transmits the encoded code stream data according to the relevant protocol and sends it to the service server. Among them, the code stream data includes video frames and related data such as decoding timestamps and display timestamps. During the network distribution process, the service server distributes the packet to the user terminal after receiving the packet receiving request from the user terminal. In this process, the server may also perform business processing such as transcoding on the packet according to business needs. The user terminal performs packet receiving, framing and post-processing. Post-processing such as super-resolution and other video content enhancement processing improves the resolution. Then, the user terminal performs video decoding on the data divided according to the video frame after post-processing, and decodes the corresponding video frame in an orderly and stable manner according to the decoding timestamp. Finally, the user terminal performs video rendering, and renders and displays the decoded corresponding video frames in an orderly and stable manner according to the display timestamp, so as to achieve stable display of video frames and smooth video playback for viewing by the user corresponding to the user terminal. Reducing abnormal situations during video playback can provide users with a better viewing experience.

[0195] In this embodiment, the live broadcast terminal determines the decoding timestamp and display timestamp of the current frame based on the more accurate target timestamp while making the target timestamp of the current frame more accurate. Furthermore, the current frame and the accurate decoding timestamp and display timestamp are sent to the service server in the live broadcast link, so that the current frame is decoded based on the more accurate decoding timestamp, and the decoded current frame is displayed based on the more accurate display timestamp. There are many reasons for abnormal problems such as freeze, black screen, acceleration, and screen distortion in live broadcast. This video frame display method focuses on the live broadcast streaming side to solve the problem that live broadcast is prone to abnormalities. This video frame display method is conducive to reducing the possibility of abnormal situations in video playback caused by live broadcast streaming, so that video frames can be decoded and displayed in an orderly and stable manner according to accurate decoding timestamps and display timestamps, which is conducive to reducing abnormal situations during video playback and improving the live broadcast effect.

[0196] The present application also provides a timestamp correction device 1000. Fig.10 As shown, the timestamp correction device 1000 includes a timestamp determination module 1001, a time increment determination module 1002 and a timestamp correction module 1003. The timestamp determination module 1001 is used to respectively determine the reference timestamp of the previous frame of the target video and the target timestamp of the current frame of the target video, wherein the previous frame and the current frame are any two adjacently captured video frames, and the previous frame is captured before the current frame, and the reference timestamp and the target timestamp both refer to the capture timestamp of the video frame. The time increment determination module 1002 is used to determine a first time increment based on the frame rate of the target video when the target timestamp is less than or equal to the reference timestamp, wherein the first time increment is a positive number and is negatively correlated with the frame rate. The timestamp correction module 1003 is used to correct the target timestamp by the sum of the target timestamp and the first time increment so that the target timestamp is greater than the reference timestamp.

[0197] In one embodiment, the time increment determination module 1002 is further configured to, when the target timestamp is less than or equal to the reference timestamp, use the product of the inverse of the frame rate of the target video and the first coefficient as the first time increment, and the first coefficient is a positive number less than 1. The timestamp correction module 1003 is further configured to iteratively correct the target timestamp using the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp.

[0198] In one embodiment, the timestamp correction device 1000 further includes a target frame determination module and a target frame correction module. The target frame determination module is used to determine the timestamp to be corrected of the target frame of the target video, where the target frame is any video frame captured after the current frame, and the timestamp to be corrected is the capture timestamp of the target frame. The target frame correction module is used to correct the timestamp to be corrected by the sum of the timestamp to be corrected and the first time increment.

[0199] In one embodiment, the timestamp correction device 1000 further includes a time interval determination module, which is used to determine the time interval formed by the target timestamp and the reference timestamp when the target timestamp is greater than the reference timestamp. The time increment determination module 1002 is also used to determine the second time increment based on the frame rate of the target video when the time interval does not belong to the preset interval. The preset interval is a standard for determining whether the duration of the time interval is reasonable. The timestamp correction module 1003 is also used to correct the target timestamp with the sum of the target timestamp and the second time increment so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval.

[0200] In one embodiment, the time increment determination module 1002 is further used to determine a second time increment based on a frame rate of the target video when the time interval is less than a minimum boundary value of a preset interval, wherein the second time increment is a positive number and is negatively correlated with the frame rate.

[0201] In one embodiment, the time increment determination module 1002 is further configured to, when the time interval is less than the minimum boundary value of the preset interval, use the product of the inverse of the frame rate of the target video and the second coefficient as the second time increment, and the second coefficient is a positive number less than 1. The timestamp correction module 1003 is further configured to iteratively correct the target timestamp using the sum of the target timestamp and the second time increment, so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval.

[0202] In one embodiment, the time increment determination module 1002 is further used to determine a second time increment based on a frame rate of a target video when the time interval is greater than a maximum boundary value of a preset interval, wherein the second time increment is a negative number and an absolute value of the second time increment is negatively correlated with the frame rate.

[0203] In one embodiment, the time increment determination module 1002 is further configured to, when the time interval is greater than the maximum boundary value of the preset interval, use the product of the inverse of the frame rate of the target video and the third coefficient as the absolute value of the second time increment, the third coefficient is a positive number less than 1, and the second time increment is a negative number. The timestamp correction module 1003 is further configured to iteratively correct the target timestamp using the sum of the target timestamp and the second time increment, so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval.

[0204] In one embodiment, the target frame determination module is further used to determine the target frame of the target video, the target frame is any video frame captured after the current frame, and the target frame is the capture timestamp of the target frame. The target frame correction module is further used to correct the target frame by the sum of the target frame and the target time increment, and the target time increment is the second time increment or the sum of the first time increment and the second time increment.

[0205] In one embodiment, the timestamp correction device 1000 further includes a target processing module for performing target processing on the original video to obtain a target video, wherein the target processing includes any one of acquisition processing and pre-processing.

[0206] For the explanation of each term, please refer to the relevant description in the aforementioned timestamp correction method embodiment, which will not be elaborated here.

[0207] It should be noted that the specific execution process of the above-mentioned time stamp correction device 1000 can be found in Figures 1 to 7 The specific description of the illustrated embodiment will not be repeated here.

[0208] Each module in the above-mentioned timestamp correction device 1000 can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0209] The embodiment of the present application also provides a video frame display device applied to a live broadcast terminal. Fig.11As shown, the video frame display device 1100 includes a first determination module 1101, a second determination module 1102, a correction module 1103, a third determination module 1104 and a sending module 1105. The first determination module 1101 is used to respectively determine the reference timestamp of the previous frame of the target video and the target timestamp of the current frame of the target video. The previous frame and the current frame are any two adjacent video frames that are captured, and the previous frame is captured before the current frame. The reference timestamp and the target timestamp both refer to the capture timestamp of the video frame. The second determination module 1102 is used to determine the first time increment based on the frame rate of the target video when the target timestamp is less than or equal to the reference timestamp. The first time increment is a positive number and is negatively correlated with the frame rate. The correction module 1103 is used to correct the target timestamp with the sum of the target timestamp and the first time increment so that the target timestamp is greater than the reference timestamp. The third determination module 1104 is used to determine the decoding timestamp and display timestamp of the current frame based on the target timestamp when the target timestamp is greater than the reference timestamp. The sending module 1105 is used to send the current frame and the decoding timestamp and display timestamp of the current frame to the business server in the live link. The decoding timestamp refers to the time information for decoding the current frame, and the display timestamp refers to the time information for displaying the decoded current frame.

[0210] For the explanation of each term, please refer to the relevant description in the aforementioned video frame display method embodiment, which will not be elaborated here.

[0211] It should be noted that the specific execution process of the above-mentioned video frame display device 1100 can be found in Figures 8 to 9 The specific description of the illustrated embodiment will not be repeated here.

[0212] Each module in the video frame display device 1100 can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0213] like Fig.12 As shown, the embodiment of the present application further provides a computer device 1200. Exemplarily, the computer device 1200 may include a processor 1201, a communication interface 1202, a communication bus 1203, and a memory 1204. Specifically, the computer device 1200 may include:

[0214] At least one processor 1201, such as a CPU, at least one communication interface 1202, a memory 1204, and at least one communication bus 1203. The communication bus 1203 is used to realize the connection and communication between these components. The communication interface 1202 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface or a Bluetooth interface, etc.). The memory 1204 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1204 may optionally be at least one storage device located away from the aforementioned processor 1201. Fig.12 As shown, the memory 1204 as a computer storage medium may include an operating system and program instructions.

[0215] Exemplarily, the processor 1201 may be used to implement the above Fig.10 The steps or methods executed by the timestamp determination module 1001, the time increment determination module 1002 and the timestamp correction module 1003 in the embodiment.

[0216] It is understandable that the above method is only an example, and the processor 1201 and other modules in the above computer device 1200 may also cooperate to perform the above method. Fig.10 The steps or methods executed by the timestamp determination module 1001, the time increment determination module 1002 and the timestamp correction module 1003 are not limited in this document.

[0217] exist Fig.12 In the computer device 1200 shown, the processor 1201 may be used to load program instructions stored in the memory 1204 and specifically perform the following operations:

[0218] Determine respectively a reference timestamp of a previous frame of the target video and a target timestamp of a current frame of the target video, the previous frame and the current frame are any two adjacently captured video frames, and the previous frame is captured before the current frame, and the reference timestamp and the target timestamp both refer to the capture timestamp of the video frame;

[0219] When the target timestamp is less than or equal to the reference timestamp, a first time increment is determined based on the frame rate of the target video, the first time increment is a positive number, and the first time increment is negatively correlated with the frame rate;

[0220] The target timestamp is corrected by the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp.

[0221] For the explanation of each term, please refer to the relevant description in the aforementioned timestamp correction method embodiment, which will not be elaborated here.

[0222] It should be noted that the specific implementation process can be found in Figures 1 to 7 The specific description of the illustrated embodiment will not be repeated here.

[0223] Exemplarily, the processor 1201 may be used to implement the above Fig.11 The steps or methods executed by the first determination module 1101, the second determination module 1102, the correction module 1103, the third determination module 1104 and the sending module 1105.

[0224] It is understandable that the above method is only an example, and the processor 1201 and other modules in the above computer device 1200 may also cooperate to perform the above method. Fig.11 The steps or methods executed by the first determination module 1101, the second determination module 1102, the correction module 1103, the third determination module 1104 and the sending module 1105 are not limited in this document.

[0225] exist Fig.12 In the computer device 1200 shown, the processor 1201 may be used to load program instructions stored in the memory 1204 and specifically perform the following operations:

[0226] Determine respectively a reference timestamp of a previous frame of the target video and a target timestamp of a current frame of the target video, the previous frame and the current frame are any two adjacently captured video frames, and the previous frame is captured before the current frame, and the reference timestamp and the target timestamp both refer to the capture timestamp of the video frame;

[0227] When the target timestamp is less than or equal to the reference timestamp, a first time increment is determined based on the frame rate of the target video, the first time increment is a positive number, and the first time increment is negatively correlated with the frame rate;

[0228] Correcting the target timestamp by the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp;

[0229] When the target timestamp is greater than the reference timestamp, determining a decoding timestamp and a display timestamp of the current frame based on the target timestamp;

[0230] The current frame and its decoding timestamp and display timestamp are sent to the service server in the live broadcast link. The decoding timestamp refers to the time information of decoding the current frame, and the display timestamp refers to the time information of displaying the decoded current frame.

[0231] For the explanation of each term, please refer to the relevant description in the aforementioned video frame display method embodiment, which will not be elaborated here.

[0232] It should be noted that the specific implementation process can be found in Figures 8 to 9The specific description of the illustrated embodiment will not be repeated here.

[0233] The present application also provides a computer-readable storage medium, which can store a plurality of instructions, and the instructions are suitable for being loaded and executed by a processor as described above. Figures 1 to 9 The method steps of the embodiment shown in the figure can be found in the specific execution process. Figures 1 to 9 The specific description of the illustrated embodiment will not be repeated here.

[0234] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0235] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any fusion thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0236] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0237] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0238] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A timestamp correction method, characterized in that: The method comprises: Determine respectively a reference timestamp of a previous frame of a target video and a target timestamp of a current frame of the target video, wherein the previous frame and the current frame are any two adjacently captured video frames, and the previous frame is captured before the current frame, and both the reference timestamp and the target timestamp refer to the capture timestamp of the video frame; When the target timestamp is less than or equal to the reference timestamp, determining a first time increment based on the frame rate of the target video, the first time increment is a positive number, and the first time increment is negatively correlated with the frame rate; The target timestamp is corrected by the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp.

2. The method according to claim 1, characterized in that The determining the first time increment based on the frame rate of the target video includes: The product of the inverse of the frame rate of the target video and a first coefficient is used as the first time increment, where the first coefficient is a positive number less than 1; The step of correcting the target timestamp by using the sum of the target timestamp and the first time increment so that the target timestamp is greater than the reference timestamp includes: The target timestamp is iteratively corrected by using the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp.

3. The method according to claim 1, characterized in that After correcting the target timestamp by the sum of the target timestamp and the first time increment so that the target timestamp is greater than the reference timestamp, the method further includes: Determine a timestamp to be corrected of a target frame of the target video, where the target frame is any video frame captured after the current frame, and the timestamp to be corrected is a capture timestamp of the target frame; The time stamp to be modified is modified by the sum of the time stamp to be modified and the first time increment.

4. The method according to claim 1, characterized in that The method further comprises: In a case where the target timestamp is greater than the reference timestamp, determining a time interval formed by the target timestamp and the reference timestamp; In the case where the time interval does not belong to a preset interval, determining a second time increment based on the frame rate of the target video, the preset interval being a standard for determining whether the duration of the time interval is reasonable; The target timestamp is corrected by the sum of the target timestamp and the second time increment, so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval.

5. The method according to claim 4, characterized in that When the time interval does not belong to the preset interval, determining the second time increment based on the frame rate of the target video includes: When the time interval is less than the minimum boundary value of the preset interval, the second time increment is determined based on the frame rate of the target video, the second time increment is a positive number, and the second time increment is negatively correlated with the frame rate.

6. The method according to claim 5, characterized in that The determining the second time increment based on the frame rate of the target video includes: The product of the reciprocal of the frame rate of the target video and a second coefficient is used as the second time increment, where the second coefficient is a positive number less than 1; The step of correcting the target timestamp by using the sum of the target timestamp and the second time increment so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval includes: The target timestamp is iteratively corrected by using the sum of the target timestamp and the second time increment, so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval.

7. The method according to claim 4, characterized in that When the time interval does not belong to the preset interval, determining the second time increment based on the frame rate of the target video includes: When the time interval is greater than the maximum boundary value of the preset interval, the second time increment is determined based on the frame rate of the target video, the second time increment is a negative number, and the absolute value of the second time increment is negatively correlated with the frame rate.

8. The method according to claim 7, characterized in that The determining the second time increment based on the frame rate of the target video, wherein the second time increment is a negative number, comprises: The product of the reciprocal of the frame rate of the target video and the third coefficient is used as the absolute value of the second time increment, the third coefficient is a positive number less than 1, and the second time increment is a negative number; The step of correcting the target timestamp by using the sum of the target timestamp and the second time increment so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval includes: The target timestamp is iteratively corrected by using the sum of the target timestamp and the second time increment, so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval.

9. The method according to claim 4, characterized in that After the target timestamp is corrected by the sum of the target timestamp and the second time increment so that the time interval formed by the target timestamp and the reference timestamp belongs to the preset interval, the method further includes: Determine a timestamp to be corrected of a target frame of the target video, where the target frame is any video frame captured after the current frame, and the timestamp to be corrected is a capture timestamp of the target frame; The time stamp to be corrected is corrected by the sum of the time stamp to be corrected and a target time increment, wherein the target time increment is the second time increment or the sum of the first time increment and the second time increment.

10. The method according to any one of claims 1 to 9, characterized in that: Before respectively determining the reference timestamp of the previous frame of the target video and the target timestamp of the current frame of the target video, the method further includes: The original video is subjected to target processing to obtain the target video, wherein the target processing includes any one of acquisition processing and pre-processing.

11. A video frame display method, characterized in that: The method is applied to a live broadcast terminal, and the method comprises: Determine respectively a reference timestamp of a previous frame of a target video and a target timestamp of a current frame of the target video, wherein the previous frame and the current frame are any two adjacently captured video frames, and the previous frame is captured before the current frame, and both the reference timestamp and the target timestamp refer to the capture timestamp of the video frame; When the target timestamp is less than or equal to the reference timestamp, determining a first time increment based on the frame rate of the target video, the first time increment is a positive number, and the first time increment is negatively correlated with the frame rate; Correcting the target timestamp by the sum of the target timestamp and the first time increment so that the target timestamp is greater than the reference timestamp; In a case where the target timestamp is greater than the reference timestamp, determining a decoding timestamp and a display timestamp of the current frame based on the target timestamp; The current frame and the decoding timestamp and the display timestamp of the current frame are sent to the service server in the live broadcast link, where the decoding timestamp refers to the time information for decoding the current frame, and the display timestamp refers to the time information for displaying the decoded current frame.

12. A timestamp correction device, characterized in that: The device comprises: A timestamp determination module, used to respectively determine a reference timestamp of a previous frame of a target video and a target timestamp of a current frame of the target video, wherein the previous frame and the current frame are any two adjacently captured video frames, and the previous frame is captured before the current frame, and the reference timestamp and the target timestamp both refer to the capture timestamp of the video frame; a time increment determination module, configured to determine a first time increment based on a frame rate of the target video when the target timestamp is less than or equal to the reference timestamp, wherein the first time increment is a positive number and is negatively correlated with the frame rate; The timestamp correction module is used to correct the target timestamp by using the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp.

13. A video frame display device, characterized in that: The device is applied to a live broadcast terminal, and the device includes: A first determination module is used to respectively determine a reference timestamp of a previous frame of a target video and a target timestamp of a current frame of the target video, wherein the previous frame and the current frame are any two adjacently captured video frames, and the previous frame is captured before the current frame, and the reference timestamp and the target timestamp both refer to the capture timestamp of the video frame; A second determining module is used to determine a first time increment based on the frame rate of the target video when the target timestamp is less than or equal to the reference timestamp, wherein the first time increment is a positive number and is negatively correlated with the frame rate; A correction module, configured to correct the target timestamp by using the sum of the target timestamp and the first time increment, so that the target timestamp is greater than the reference timestamp; A third determining module, configured to determine a decoding timestamp and a display timestamp of the current frame based on the target timestamp when the target timestamp is greater than the reference timestamp; The sending module is used to send the current frame and the decoding timestamp and the display timestamp of the current frame to the business server in the live broadcast link, wherein the decoding timestamp refers to the time information for decoding the current frame, and the display timestamp refers to the time information for displaying the decoded current frame.

14. A computer device, characterized in that: include: A memory and a processor, wherein the memory stores program instructions; when the program instructions are executed by the processor, the processor executes the method according to any one of claims 1 to 10 or the method according to claim 11.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program; when the computer program runs on one or more processors, the method according to any one of claims 1 to 10 or the method according to claim 11 is executed.

16. A computer program product, characterized in that The computer program product comprises a computer program or instructions; when the computer program or instructions are run on a computer, the computer is caused to execute the method according to any one of claims 1 to 10 or the method according to claim 11.