A method, apparatus, terminal device, and storage medium for processing video data.

By calculating the actual time and frame rate of video data frames, the number of dropped frames is determined, thus solving the latency problem of high-resolution video monitoring on browsers and enabling smooth playback of video data.

CN115348409BActive Publication Date: 2026-11-10VISIONVERA INFORMATION TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210816355.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-07-12
Publication Date
2026-11-10
Estimated Expiration
2042-07-12

AI Technical Summary

Technical Problem

H.265 encoded video surveillance, especially high-resolution video surveillance, cannot be decoded and played in browsers, resulting in problems such as screen delay and slow playback, which cannot be effectively solved by existing technologies.

Method used

By acquiring the actual time consumed by video data frames, calculating the time difference and frame rate, determining the number of dropped frames, discarding some P-frame video frames to reduce decoding time, and using frame drop processing within a preset time period to ensure smooth playback.

Benefits of technology

It effectively reduces video playback delay, avoids slow playback, and improves the smoothness of video data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115348409B_ABST
    Figure CN115348409B_ABST
Patent Text Reader

Abstract

This invention provides a video data processing method, apparatus, terminal device, and storage medium, including: acquiring the actual consumption time of the video data, determining the time difference within a preset time period based on the actual consumption time and a preset consumption time, and then calculating the number of P-frames to be discarded within the preset time period based on the time difference. In this way, several P-frames are discarded within the current preset time period, reducing decoding time. Decoding is then performed again in the next preset time period, reducing the accumulation of decoding time and thus reducing the latency when playing video data, preventing slow playback.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, terminal device, and storage medium for processing video data. Background Technology

[0002] While H.265 (video encoding standard) encoding of surveillance footage typically allows for playback on web pages, some browsers do not support the H.265 encoded video format and cannot decode and play H.265 encoded surveillance videos. Therefore, to enable real-time playback of H.265 encoded surveillance videos in browsers without plugins, WASM (WebAssembly) decoding is required. The video data decoded by WASM is in YUV format, which needs color space conversion for rendering. This conversion can be performed using FFmpeg's libswscale module.

[0003] For lower-resolution video surveillance, if the total video decoding and rendering time is less than or equal to the specified video duration, the video surveillance can be displayed normally. However, for high-resolution video surveillance such as 4K or 8K, if the total video decoding and rendering time exceeds the specified video duration, and the decoding device's decoding capability is insufficient, there will be a delay in decoding each frame of the video surveillance. If the delay in a single frame is small, it is imperceptible to the human eye, but if it accumulates, it will cause a significant increase in delay, resulting in cumulative latency and thus a slow playback. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a video data processing method, apparatus, terminal device, and storage medium that overcomes or at least partially solves the above problems.

[0005] In a first aspect, embodiments of the present invention provide a method for processing video data, the method comprising:

[0006] Get the video data to be played;

[0007] Each frame in the video data is parsed to obtain the actual time consumed within a preset time period; wherein the preset time period includes at least the time interval between two I-frame video frames.

[0008] Based on the actual consumption time and the preset consumption time, determine the first time difference within the preset time period;

[0009] Based on the first time difference and the video frame rate, determine the number of dropped frames within the preset time period;

[0010] If the first time difference is less than or equal to the preset perception time, the P-frame video frames corresponding to the number of dropped frames within the preset time period in the video data are processed to drop the dropped frames, and the dropped frame video data is played. The P-frame video frames are located before the next I-frame video frame.

[0011] Optionally, the preset time consumption is obtained based on the video frame rate of the video data.

[0012] Optionally, determining the first time difference within the preset time period based on the actual consumption time and the preset consumption time includes:

[0013] Calculate the second time difference of parsing a video frame. The second time difference is obtained based on the difference between the actual time consumed and the preset time consumed in a video frame.

[0014] Get the number of video frames between two I-frame video frames;

[0015] Based on the number of video frames and the second time difference, a first time difference within a preset time period is determined.

[0016] Optionally, the method further includes:

[0017] If the first time difference is greater than the preset perception time, then the P-frame video frame corresponding to the preset perception time that is located before the next I-frame within the current preset time period is dropped to obtain the dropped video data.

[0018] In the next preset time period, continue to drop P-frame video frames corresponding to the preset perception time that are located before the next I-frame, until the cumulative number of data packets that need to be dropped is the same as the number of video frames between the I-frames of the two frames, and then discard all data packets that need to be dropped.

[0019] Optionally, calculating the number of dropped frames within a preset time period based on the first time difference and the video frame rate includes:

[0020] Calculate the time required to parse one video frame based on the video frame rate;

[0021] The number of dropped frames within a preset time period is calculated based on the first time difference and the time required to parse one video frame.

[0022] Secondly, embodiments of the present invention provide a video data processing apparatus, the apparatus comprising:

[0023] The receiving module is used to acquire the video data to be played.

[0024] The parsing module is used to parse each frame in the video data to obtain the actual time consumed within a preset time period; wherein the preset time period includes at least the time interval between two I-frame video frames.

[0025] The determining module is used to determine a first time difference within a preset time period based on the actual consumption time and the preset consumption time.

[0026] The calculation module is used to determine the number of dropped frames within a preset time period based on the first time difference and the video frame rate.

[0027] The frame dropping module is used to drop P-frames in the video data within a preset time period corresponding to the number of dropped frames if the first time difference is less than or equal to a preset sensing time, to obtain video data after frame dropping, and to play the video data after frame dropping, wherein the P-frames are located before the next I-frame.

[0028] Optionally, the preset time consumption is obtained based on the video frame rate of the video data.

[0029] Optionally, the determining module is used to:

[0030] Calculate the second time difference of parsing a video frame. The second time difference is obtained based on the difference between the actual time consumed and the preset time consumed in a video frame.

[0031] Get the number of video frames between two I-frame video frames;

[0032] Based on the number of video frames and the second time difference, a first time difference within a preset time period is determined.

[0033] Optionally, the frame dropping module is used for:

[0034] If the first time difference is greater than the preset perception time, then the P-frame video frame corresponding to the preset perception time that is located before the next I-frame within the current preset time period is dropped to obtain the dropped video data.

[0035] In the next preset time period, continue to drop P-frame video frames corresponding to the preset perception time that are located before the next I-frame, until the cumulative number of data packets that need to be dropped is the same as the number of video frames between the I-frames of the two frames, and then discard all data packets that need to be dropped.

[0036] Optionally, the computing module is used for:

[0037] Calculate the time required to parse one video frame based on the video frame rate;

[0038] The number of dropped frames within a preset time period is calculated based on the first time difference and the time required to parse one video frame.

[0039] Thirdly, embodiments of the present invention provide a terminal device, including: at least one processor and a memory;

[0040] The memory stores a computer program; the at least one processor executes the computer program stored in the memory to implement the video data processing method provided in the first aspect.

[0041] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed, implements the video data processing method provided in the first aspect.

[0042] The embodiments of the present invention have the following advantages:

[0043] The video data processing method, apparatus, terminal device, and storage medium provided in this invention involve: acquiring video data to be played; parsing each frame of the video data to obtain the actual time consumed within a preset time period; wherein the preset time period includes at least the time interval between two I-frame video frames; determining a first time difference within the preset time period based on the actual time consumed and the preset time consumed; determining the number of dropped frames within the preset time period based on the first time difference and the video frame rate; and if the first time difference is less than or equal to a preset sensing time, performing frame dropping processing on the P-frame video frames corresponding to the number of dropped frames within the preset time period of the video data. The video data after frame dropping is obtained and played. P-frames are located before the next I-frame. The actual time taken to consume the video data is obtained, and the time difference within a preset time period is determined based on the actual time and a preset time. Then, the number of P-frames to be dropped within the preset time period is calculated based on this time difference. In this way, several P-frames are dropped within the current preset time period, reducing decoding time. Decoding is then performed again in the next preset time period, reducing the accumulation of decoding time and thus reducing the latency when playing the video data, preventing slow playback. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating the steps of an embodiment of a video data processing method according to the present invention;

[0045] Figure 2 This is a flowchart illustrating the steps of another embodiment of the video data processing method of the present invention;

[0046] Figure 3 This is a schematic diagram of another embodiment of the video data processing method of the present invention;

[0047] Figure 4 This is a structural block diagram of an embodiment of a video data processing device according to the present invention;

[0048] Figure 5 This is a schematic diagram of the structure of a terminal device according to the present invention. Detailed Implementation

[0049] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] The video network is a significant milestone in network development. It is a real-time network that enables real-time transmission of high-definition video, pushing many internet applications towards high-definition video and enabling high-definition face-to-face interaction.

[0051] The video network adopts real-time high-definition video switching technology, which can integrate dozens of services such as high-definition video conferencing, video surveillance, intelligent monitoring and analysis, emergency command, digital broadcasting, time-delayed television, online teaching, live broadcasting, VOD, TV mail, personalized recording (PVR), intranet (self-operated) channels, intelligent video broadcast control, information release, etc., into a single system platform, enabling high-definition video playback through televisions or computers.

[0052] Definitions:

[0053] Wasm:WebAssembly is a binary instruction format based on a stack-based virtual machine. Designed for portability, Wasm can be used to compile high-level languages ​​such as C / C++ / Rust, enabling client and server applications to be deployed on the web. WASM has various implementations, including browsers and standalone systems, and can be used for applications such as video and audio codecs, graphics and 3D, multimedia and games, cryptographic computation, or portable language implementations.

[0054] FFmpeg is an open-source computer program that can be used to record, convert, and stream digital audio and video.

[0055] Generally, for lower-resolution video data, the video data can be displayed normally if the total video decoding and rendering time is less than or equal to the specified video duration. However, for high-resolution video data, such as 4K or 8K video data, if the total video decoding and rendering time exceeds the specified video duration, the video will play slowly. The time delay of a single frame is imperceptible to the human eye, but it accumulates and increases the time delay, resulting in cumulative delay and slow video playback.

[0056] For example, if the surveillance video resolution is 3840*2160, the frame rate is 25 fps, and the I-frame interval is 50 frames (25 frames per second, 40ms per frame, and one I-frame every 2 seconds) to display one image, assuming the terminal device has limited decoding capabilities, the fastest time to decode one image is 42ms (rendering is ignored here). The first image should be displayed at 40ms, but because the decoding time is 42ms, it is only displayed at 42ms, delayed by (42-40) = 2ms from the expected time (40ms). When 50 video frames are decoded consecutively, this delay becomes 50 * 2ms = 100ms, and the next I-frame decoding time is 100ms slower than expected. This delay is cumulative and only increases with time.

[0057] To address the aforementioned issues, and given the inability to modify the hardware or optimize FFmpeg's internal decoding mechanism, this invention provides a video data processing method for processing video data. The execution entity in this embodiment is a video data processing device, pre-installed on a terminal device, which can be a computer, tablet computer, or mobile phone, etc.

[0058] Reference Figure 1 The diagram illustrates a flowchart of an embodiment of a video data processing method according to the present invention. The method may specifically include the following steps:

[0059] S101. Obtain the video data to be played;

[0060] Specifically, the terminal device can receive video data sent by the sending device, obtain locally stored video data, or download video data from the network. No specific limitations are made in this embodiment.

[0061] S102. Analyze each frame in the video data to obtain the actual time consumed within a preset time period; wherein the preset time period includes at least the time interval between two I-frame video frames.

[0062] Specifically, when the terminal device obtains the video data to be played, it extracts the video data within a preset time period and parses each frame of the video data within the preset time period. It needs to decode each video frame to obtain the decoding time and render each video frame to obtain the rendering time. In this embodiment, since the rendering time is small, the rendering time is ignored. Therefore, after parsing each video frame, it is only necessary to obtain the decoding time. Then, based on multiple video frames within the preset time period, the decoding time of parsing multiple video frames is obtained to obtain the actual time consumed within the preset time period.

[0063] The preset time period includes a time interval of at least two I-frame video frames. For example, the preset time period may include two or four I-frame video frames; no specific limitation is made here. To minimize decoding time delay, this embodiment uses a preset time period that includes two I-frame video frames.

[0064] S103. Determine the first time difference within the preset time period based on the actual consumption time and the preset consumption time.

[0065] Specifically, the video data to be played requires a certain amount of time to be encoded, and this time is set as the preset consumption time.

[0066] If the decoding capability of the terminal device is strong enough, it should also consume a preset time to decode and play the video data to be played. However, due to the insufficient decoding capability of the terminal device, the actual time for decoding the video frames may be even longer. Therefore, in this embodiment, after decoding the video frames within the preset time period, the actual time consumed is obtained, and the actual time consumed is compared with the preset time consumed to determine whether the actual time consumed is greater than the preset time consumed, thereby determining whether the video data needs to be processed, whether a time delay has occurred, and calculating the first time difference between the actual time consumed and the preset time consumed.

[0067] S104. Determine the number of dropped frames within the preset time period based on the first time difference and the video frame rate;

[0068] Specifically, the terminal device judges the actual consumption time and the preset consumption time. If the actual consumption time is less than or equal to the preset consumption time, the video data is not processed, indicating that the terminal device can decode the video data normally.

[0069] If the actual time consumed is greater than the preset time consumed, the difference between the actual time consumed and the preset time consumed is calculated, which is the first time difference.

[0070] The terminal device obtains the video frame rate of the video data to be played, calculates the time required to play one video frame based on the video frame rate, and then calculates how many video frames correspond to the total delay time within the preset time period, i.e., the first time difference, and the time required to play one video frame, based on the total delay time within the preset time period. In other words, the number of frames that need to be dropped.

[0071] S105. If the first time difference is less than or equal to the preset perception time, the P-frame video frames corresponding to the number of dropped frames within the preset time period in the video data are processed to drop the dropped frames, and the dropped frame video data is played. The P-frame video frames are located before the next I-frame video frames.

[0072] Specifically, the preset perception time is the maximum time during which the human eye cannot perceive a delay in the video, such as 300ms;

[0073] When the terminal device calculates the first time difference, and the first time difference is less than the preset sensing time, and the preset time period includes at least two I-frame video frames, the P-frame video frames corresponding to the number of dropped frames preceding the next I-frame video frame are discarded. In the next preset time period, the above operation is still performed to perform frame dropping processing, obtain the discarded video data, and play the video data after the frame dropping. In this way, by performing frame dropping processing on the P-frame data, the decoding time of the video frames is reduced, the cumulative time is reduced, and the slow video playback situation is avoided.

[0074] Specifically, if the preset time period includes 4 I-frame video frames, then the P-frames preceding the 4th I-frame video frame are discarded; if the preset time period includes 5 I-frame video frames, then the P-frames preceding the 5th I-frame video frame are discarded. No specific limitation is made here.

[0075] The video data processing method provided in this embodiment of the invention involves: acquiring video data to be played; parsing each frame in the video data to obtain the actual time consumed within a preset time period; wherein the preset time period includes at least the time interval between two I-frame video frames; determining a first time difference within the preset time period based on the actual time consumed and the preset time consumed; determining the number of dropped frames within the preset time period based on the first time difference and the video frame rate; if the first time difference is less than or equal to a preset perception time, dropping the P-frame video frames corresponding to the number of dropped frames within the preset time period in the video data to obtain the dropped video data, and playing the dropped video data, wherein the P-frame video frames are located before the next I-frame video frame. By acquiring the actual time consumed of the video data and determining the time difference within the preset time period based on the actual time consumed and the preset time consumed, and then calculating the number of P-frames to be dropped within the preset time period based on the time difference, several P-frames are dropped within the current preset time period, reducing decoding time. Decoding is then performed again in the next preset time period, reducing the accumulation of decoding time, thereby reducing the delay time when playing video data and preventing slow playback.

[0076] Another embodiment of the present invention further supplements the description of the video data processing method provided in the above embodiments.

[0077] like Figure 2 The diagram illustrates a flowchart of another embodiment of a video data processing method according to the present invention, which includes:

[0078] S201. Obtain the video data to be played;

[0079] Specifically, the terminal device can receive video data sent by the sending device, obtain locally stored video data, or download video data from the network. No specific limitations are made in this embodiment.

[0080] S202. Analyze each frame in the video data to obtain the actual time consumed within a preset time period; wherein the preset time period includes at least the time interval between two I-frame video frames.

[0081] Specifically, when the terminal device obtains the video data to be played, it extracts the video data within a preset time period and parses each frame of the video data within the preset time period. It needs to decode each video frame to obtain the decoding time and render each video frame to obtain the rendering time. In this embodiment, since the rendering time is small, the rendering time is ignored. Therefore, after parsing each video frame, it is only necessary to obtain the decoding time. Then, based on multiple video frames within the preset time period, the decoding time of parsing multiple video frames is obtained to obtain the actual time consumed within the preset time period.

[0082] The preset time period includes a time interval of at least two I-frame video frames. For example, the preset time period may include two or four I-frame video frames; no specific limitation is made here. To minimize decoding time delay, this embodiment uses a preset time period that includes two I-frame video frames.

[0083] S203. Calculate and parse the second time difference of a video frame. The second time difference is obtained based on the difference between the actual time consumed and the preset time consumed of a video frame.

[0084] Specifically, the terminal device decodes each frame within a preset time period to obtain the actual time consumed in decoding one video frame, and calculates the delay time for decoding one frame as the second time difference based on the preset time consumed by the actual time consumed by one video frame.

[0085] S204. Obtain the number of video frames between two I-frame video frames;

[0086] S205. Determine the first time difference within the preset time period based on the number of video frames and the second time difference.

[0087] Specifically, the terminal device calculates the first time difference within a preset time period based on the second time difference of a video frame and the number of video frames.

[0088] S206. Calculate the time required to parse one video frame based on the video frame rate;

[0089] For example, if the video frame rate is 25fps, the time required to parse one video frame is 1000 / 25 = 40ms.

[0090] S207. Calculate the number of dropped frames within a preset time period based on the first time difference and the time required to parse one video frame.

[0091] The terminal device obtains the video frame rate of the video data to be played, calculates the time required to play one video frame based on the video frame rate, and then calculates how many video frames correspond to the total delay time within the preset time period, i.e., the first time difference, and the time required to play one video frame, based on the total delay time within the preset time period. In other words, the number of frames that need to be dropped.

[0092] S208. If the first time difference is less than or equal to the preset perception time, the P-frames corresponding to the number of dropped frames within the preset time period in the video data are dropped to obtain the dropped video data, and the dropped video data is played. The P-frames are located before the next I-frame. When the terminal device calculates the first time difference, and the first time difference is less than the preset perception time, and the preset time period includes at least two I-frames, the P-frames corresponding to the number of dropped frames before the next I-frame are discarded. The above operation is repeated in the next preset time period to perform the same dropping process, obtaining the dropped video data, and then playing it. This reduces the decoding time of the video frames by dropping the P-frames, decreasing the accumulated time and preventing slow video playback.

[0093] S209. If the first time difference is greater than the preset perception time, then the P-frame video frame corresponding to the preset perception time that is located before the next I-frame in the current preset time period is dropped to obtain the dropped video data.

[0094] S210. In the next preset time period, continue to drop the P-frame video frames corresponding to the preset perception time that are located before the next I-frame, until the cumulative number of data packets that need to be dropped is the same as the number of video frames between the two I-frame video frames, and then discard all the data packets that need to be dropped.

[0095] Specifically, when watching video data on a browser webpage on a terminal device, if the decoding time in the receiving queue exceeds a certain value, the queue size will accumulate and increase. Frame dropping can only delay the situation, but it will inevitably still increase latency and memory usage. To a certain extent, it is necessary to actively drop frames according to the GOP (Getting Over-the-Counter) algorithm to ensure that there is no cumulative latency.

[0096] When the terminal device calculates the first time difference, if this first time difference is greater than the preset sensing time (e.g., 500ms) and the preset sensing time is 300ms, then the 500ms video frame needs to be segmented and discarded. Within the current preset time period, the P-frames corresponding to the preset sensing time before the next I-frame are dropped. In the next preset time period, P-frames continue to be dropped, but new delays are still generated during this process. This continues until the number of video frames to be dropped reaches a certain value within a certain preset time period (e.g., the number of video frames between I-frames). At this point, all data packets requiring frame dropping can be dropped; for example, the entire GOP (Group of Pictures) corresponding to the I-frame can be dropped.

[0097] In this embodiment, decoding time and rendering time are calculated separately. Rendering time can be ignored during the calculation. When frames are dropped, I-frames cannot be dropped, and some P-frames are actively dropped. However, P-frames supplement the preceding I-frames; dropping intermediate P-frames would cause screen tearing. Therefore, several P-frames before the next I-frame should be dropped. The number of dropped P-frames is dynamically adjusted based on the difference between decoding time and actual display time. However, monitoring typically uses a 2-second I-frame interval; dropping too many P-frames will result in noticeable interface jumps. Actively dropping too many frames is also not advisable.

[0098] In this embodiment of the invention, the current monitoring resolution, frame rate and other information and decoding capability are first obtained based on the video information of the video data, that is, the average time T2 consumed to decode a frame under the current resolution, bit rate and playback device hardware conditions and the rendering time (the difference in playback device can be ignored).

[0099] The difference between the decoding time T1 (which can be understood as 1000 / frame rate) and T2 for a normal frame is the second time difference.

[0100] Within the I-frame interval, determine how many frames need to be dropped to compensate for the time difference. For example, if the frame rate is 25, decoding one frame takes 40ms;

[0101] 1) The current decoding time is 45ms and the I-frame interval is 50 frames. The total extra time in a GOP, i.e. the first time difference, is 50*(45-40)=250ms. Therefore, 250 / 40 frames need to be dropped to make up for the delay caused by the slow decoding in this GOP.

[0102] 2) Decoding takes 50ms, and the I-frame interval is 50 frames. The total extra time in a GOP is initially 50*(50-40) = 500ms. If this 500ms time loss is perceptible to a human, then dropping 200 / 40 frames within the current preset time period is sufficient to compensate for the 500ms delay caused by slow decoding in this GOP. The remaining 300ms is the cumulative delay. In the next GOP, time delays continue to occur, along with the remaining cumulative delay from the previous GOP. At a certain moment, the queue contains 2250 data packets. When the next I-frame is detected, all data in the queue is cleared, causing the time on the screen to jump to the latest time, thus eliminating the cumulative delay.

[0103] In other words, if the total extra time in a Gop is less than 300ms (the preset perception time), the last p-frame in the Gop corresponding to that time will be discarded.

[0104] If the total extra time in a Gop is greater than 300ms (the preset perception time), discard the last p-frame in the Gop corresponding to the 300ms of time.

[0105] If the length of the detection queue exceeds 2000 (which could be the length of a GOP), the entire GOP is discarded when the next I-frame arrives, and decoding begins from the new I-frame.

[0106] Figure 3 This is a schematic diagram of another embodiment of the video data processing method of the present invention, as shown below. Figure 3 As shown, a video frame normally includes I-frames and P-frames. If played normally, the I-frame should be playing at point B. However, due to the slow decoding speed of the terminal device, point A is only decoded when point B is reached. Therefore, it is necessary to drop the P-frames between point A and point B. That is, directly jump from the P-frame at point A to the I-frame at point B to continue decoding. By dynamically dropping the P-frames, the decoding time is reduced. Decoding is then restarted in the next preset time period, reducing the accumulation of decoding time and thus reducing the delay time when playing video data, preventing the slow playback of the picture.

[0107] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0108] The video data processing method provided in this embodiment of the invention involves: acquiring video data to be played; parsing each frame in the video data to obtain the actual time consumed within a preset time period; wherein the preset time period includes at least the time interval between two I-frame video frames; determining a first time difference within the preset time period based on the actual time consumed and the preset time consumed; determining the number of dropped frames within the preset time period based on the first time difference and the video frame rate; if the first time difference is less than or equal to a preset perception time, dropping the P-frame video frames corresponding to the number of dropped frames within the preset time period in the video data to obtain the dropped video data, and playing the dropped video data, wherein the P-frame video frames are located before the next I-frame video frame. By acquiring the actual time consumed of the video data and determining the time difference within the preset time period based on the actual time consumed and the preset time consumed, and then calculating the number of P-frames to be dropped within the preset time period based on the time difference, several P-frames are dropped within the current preset time period, reducing decoding time. Decoding is then performed again in the next preset time period, reducing the accumulation of decoding time, thereby reducing the delay time when playing video data and preventing slow playback.

[0109] Another embodiment of the present invention provides a video data processing apparatus for executing the video data processing method provided in the above embodiments.

[0110] Reference Figure 4 The diagram illustrates a structural block diagram of an embodiment of a video data processing apparatus according to the present invention. The apparatus may specifically include the following modules: a receiving module 401, a parsing module 402, a determining module 403, a calculation module 404, and a frame dropping module 405, wherein:

[0111] The receiving module 401 is used to acquire the video data to be played;

[0112] The parsing module 402 is used to parse each frame in the video data to obtain the actual time consumed within a preset time period; wherein the preset time period includes at least the time interval between two I-frame video frames;

[0113] The determining module 403 is used to determine the first time difference within the preset time period based on the actual consumption time and the preset consumption time;

[0114] The calculation module 404 is used to determine the number of dropped frames within a preset time period based on the first time difference and the video frame rate;

[0115] The frame dropping module 405 is used to drop P-frame video frames corresponding to the number of dropped frames within a preset time period in the video data if the first time difference is less than or equal to the preset perception time, to obtain the video data after frame dropping, and to play the video data after frame dropping, wherein the P-frame video frame is located before the next I-frame video frame.

[0116] The video data processing apparatus provided in this embodiment of the invention acquires video data to be played; parses each frame in the video data to obtain the actual time consumed within a preset time period; wherein the preset time period includes at least the time interval between two I-frame video frames; determines a first time difference within the preset time period based on the actual time consumed and the preset time consumed; determines the number of dropped frames within the preset time period based on the first time difference and the video frame rate; if the first time difference is less than or equal to a preset sensing time, performs frame dropping processing on the P-frame video frames in the video data corresponding to the number of dropped frames within the preset time period to obtain the video data after frame dropping, and plays the video data after frame dropping, wherein the P-frame video frames are located before the next I-frame video frames. By acquiring the actual time consumed of the video data and determining the time difference within the preset time period based on the actual time consumed and the preset time consumed, and then calculating the number of P-frames to be dropped within the preset time period based on the time difference, several P-frames are dropped within the current preset time period, reducing decoding time. Decoding is then performed again in the next preset time period, reducing the accumulation of decoding time, thereby reducing the delay time when playing video data and preventing slow playback.

[0117] Another embodiment of the present invention further provides a description of the video data processing apparatus provided in the above embodiments.

[0118] Optionally, the preset time consumption is obtained based on the video frame rate of the video data.

[0119] Optionally, the determination module is used for:

[0120] The second time difference of parsing a video frame is calculated. The second time difference is obtained based on the difference between the actual time consumed and the preset time consumed in a video frame.

[0121] Get the number of video frames between two I-frame video frames;

[0122] The first time difference within the preset time period is determined based on the number of video frames and the second time difference.

[0123] Optionally, the frame dropping module is used for:

[0124] If the first time difference is greater than the preset perception time, then the P-frame video frame corresponding to the preset perception time that is located before the next I-frame within the current preset time period will be dropped to obtain the dropped video data.

[0125] In the next preset time period, continue to drop P-frame video frames that are located before the next I-frame and correspond to the preset perception time, until the cumulative number of data packets that need to be dropped is the same as the number of video frames between the two I-frames, at which point all data packets that need to be dropped are discarded.

[0126] Optionally, the calculation module is used for:

[0127] Calculate the time required to parse one video frame based on the video frame rate;

[0128] Calculate the number of dropped frames within a preset time period based on the first time difference and the time required to parse one video frame.

[0129] It should be noted that each of the implementable methods in this embodiment can be implemented individually or in any combination without conflict. This application does not limit this.

[0130] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0131] The video data processing apparatus provided in this embodiment of the invention acquires video data to be played; parses each frame in the video data to obtain the actual time consumed within a preset time period; wherein the preset time period includes at least the time interval between two I-frame video frames; determines a first time difference within the preset time period based on the actual time consumed and the preset time consumed; determines the number of dropped frames within the preset time period based on the first time difference and the video frame rate; if the first time difference is less than or equal to a preset sensing time, performs frame dropping processing on the P-frame video frames in the video data corresponding to the number of dropped frames within the preset time period to obtain the video data after frame dropping, and plays the video data after frame dropping, wherein the P-frame video frames are located before the next I-frame video frames. By acquiring the actual time consumed of the video data and determining the time difference within the preset time period based on the actual time consumed and the preset time consumed, and then calculating the number of P-frames to be dropped within the preset time period based on the time difference, several P-frames are dropped within the current preset time period, reducing decoding time. Decoding is then performed again in the next preset time period, reducing the accumulation of decoding time, thereby reducing the delay time when playing video data and preventing slow playback.

[0132] In another embodiment of the present invention, a terminal device is provided for executing the video data processing method provided in the above embodiments.

[0133] Figure 5 This is a schematic diagram of the structure of a terminal device according to the present invention, such as... Figure 5 As shown, the terminal device includes: at least one processor 501 and a memory 502;

[0134] The memory stores a computer program; at least one processor executes the computer program stored in the memory to implement the video data processing method provided in the above embodiments.

[0135] The terminal device provided in this embodiment acquires video data to be played; parses each frame in the video data to obtain the actual time consumed within a preset time period; wherein the preset time period includes at least the time interval between two I-frame video frames; determines a first time difference within the preset time period based on the actual time consumed and the preset time consumed; determines the number of dropped frames within the preset time period based on the first time difference and the video frame rate; if the first time difference is less than or equal to a preset perception time, performs frame dropping processing on the P-frame video frames in the video data corresponding to the number of dropped frames within the preset time period to obtain the video data after frame dropping, and plays the video data after frame dropping, wherein the P-frame video frame is located before the next I-frame video frame. By acquiring the actual time consumed of the video data and determining the time difference within the preset time period based on the actual time consumed and the preset time consumed, and then calculating the number of P-frames to be dropped within the preset time period based on the time difference, several P-frames are dropped within the current preset time period, reducing decoding time. Decoding is then performed again in the next preset time period, reducing the accumulation of decoding time, thereby reducing the delay time when playing video data and preventing slow playback.

[0136] Another embodiment of this application provides a computer-readable storage medium storing a computer program that, when executed, implements the video data processing method provided in any of the above embodiments.

[0137] According to the computer-readable storage medium of this embodiment, video data to be played is acquired; each frame in the video data is parsed to obtain the actual time consumed within a preset time period; wherein the preset time period includes at least the time interval between two I-frame video frames; a first time difference is determined within the preset time period based on the actual time consumed and the preset time consumed; the number of dropped frames within the preset time period is determined based on the first time difference and the video frame rate; if the first time difference is less than or equal to a preset perception time, the P-frame video frames corresponding to the number of dropped frames within the preset time period in the video data are dropped to obtain the dropped video data, and the dropped video data is played, wherein the P-frame video frames are located before the next I-frame video frames. By acquiring the actual time consumed of the video data and determining the time difference within the preset time period based on the actual time consumed and the preset time consumed, the number of P-frames to be dropped within the preset time period is calculated based on the time difference. In this way, several P-frames are dropped within the current preset time period, reducing the decoding time. Decoding is performed again in the next preset time period, reducing the accumulation of decoding time, thereby reducing the delay time when playing video data and preventing slow playback.

[0138] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0139] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0140] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, electronic devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing electronic device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing electronic device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0141] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing electronic device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0142] These computer program instructions can also be loaded onto a computer or other programmable data processing electronic device to cause a series of operational steps to be performed on the computer or other programmable electronic device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable electronic device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0143] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0144] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or electronic device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or electronic device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or electronic device that includes said element.

[0145] The present invention has provided a detailed description of a video data processing method and a video data processing apparatus. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for processing video data, characterized in that, The method includes: Get the video data to be played; Each frame in the video data is parsed to obtain the actual time consumed within a preset time period; wherein the preset time period includes at least the time interval between two I-frame video frames. Based on the actual consumption time and the preset consumption time, determine the first time difference within the preset time period; Based on the first time difference and the video frame rate, determine the number of dropped frames within the preset time period; If the first time difference is less than or equal to the preset perception time, the P-frame video frames corresponding to the number of dropped frames within the preset time period in the video data are processed to drop the video data after dropping the frames, and the video data after dropping the frames is played, wherein the P-frame video frames are located before the next I-frame video frames. If the first time difference is greater than the preset perception time, then the P-frame video frame corresponding to the preset perception time that is located before the next I-frame within the current preset time period is dropped to obtain the dropped video data. In the next preset time period, continue to drop P-frame video frames corresponding to the preset perception time that are located before the next I-frame, until the cumulative number of video frames that need to be dropped is the same as the number of video frames between the I-frames of the two frames, and then discard all data packets that need to be dropped.

2. The method according to claim 1, characterized in that, The preset time consumption is obtained based on the video frame rate of the video data.

3. The method according to claim 1, characterized in that, The step of determining the first time difference within the preset time period based on the actual consumption time and the preset consumption time includes: Calculate the second time difference of parsing a video frame. The second time difference is obtained based on the difference between the actual time consumed and the preset time consumed in a video frame. Get the number of video frames between two I-frame video frames; Based on the number of video frames and the second time difference, a first time difference within a preset time period is determined.

4. The method according to claim 1, characterized in that, The step of calculating the number of dropped frames within a preset time period based on the first time difference and the video frame rate includes: Calculate the time required to parse one video frame based on the video frame rate; The number of dropped frames within a preset time period is calculated based on the first time difference and the time required to parse one video frame.

5. A video data processing apparatus, characterized in that, The device includes: The receiving module is used to acquire the video data to be played. The parsing module is used to parse each frame in the video data to obtain the actual time consumed within a preset time period; wherein the preset time period includes at least the time interval between two I-frame video frames. The determining module is used to determine a first time difference within a preset time period based on the actual consumption time and the preset consumption time. The calculation module is used to determine the number of dropped frames within a preset time period based on the first time difference and the video frame rate. The frame dropping module is used to drop P-frame video frames corresponding to the number of dropped frames within a preset time period in the video data if the first time difference is less than or equal to a preset perception time, to obtain video data after frame dropping, and to play the video data after frame dropping, wherein the P-frame video frame is located before the next I-frame video frame. The frame dropping module is configured to, if the first time difference is greater than a preset sensing time, drop the P-frame video frames corresponding to the preset sensing time that are located before the next I-frame within the current preset time period to obtain dropped video data; and continue to drop the P-frame video frames corresponding to the preset sensing time that are located before the next I-frame within the next preset time period until the cumulative number of data packets that need to be dropped is the same as the number of video frames between the I-frame video frames of the two frames, and then discard all data packets that need to be dropped.

6. The apparatus according to claim 5, characterized in that, The preset time consumption is obtained based on the video frame rate of the video data.

7. The apparatus according to claim 6, characterized in that, The determining module is used for: Calculate the second time difference of parsing a video frame. The second time difference is obtained based on the difference between the actual time consumed and the preset time consumed in a video frame. Get the number of video frames between two I-frame video frames; Based on the number of video frames and the second time difference, a first time difference within a preset time period is determined.

8. A terminal device, characterized in that, include: At least one processor and memory; The memory stores computer programs; The at least one processor executes the computer program stored in the memory to implement the video data processing method according to any one of claims 1-4.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the method for processing video data according to any one of claims 1-4.

Citation Information

Patent Citations

  • Video data processing method and device, terminal and computer readable storage medium

    CN109391585A

  • Digital broadcast receiving apparatus and method for processing synchronization therein

    US20060220930A1