Frame loss method in web player and web player

Abstract

The invention discloses a frame loss method in a web player and the web player, and relates to the technical field of monitoring video processing.The method comprises the steps that firstly, video source parameters are obtained, and whether the I-frame interval is fixed or not is judged; detecting a delay time difference when the decoding timestamp in the web player is less than the current display time, if the delay time difference reaches a preset threshold value, indicating that frame loss is needed, otherwise, not needing frame loss; carrying out frame loss processing according to a judgment result whether the I frame interval is fixed or not; and after frame loss is completed or frame loss processing is not needed, transmitting the data to a video label for rendering, and continuing to judge the next data frame. According to the invention, resource occupation can be reduced, and the playing effect is effectively ensured not to be delayed.

Description

Technical Field

[0001] The present invention relates to the technical field of monitoring video processing, and more particularly to a frame dropping method in a web player and the web player. Background Art

[0002] With the continuous advancement of science and technology and the rapid development of surveillance technology, high-definition and even ultra-high-definition cameras have been widely used in many fields such as security, traffic monitoring, and public place management. These high-definition cameras not only significantly improve the ability to capture details of surveillance images, making the surveillance content richer and more accurate, but also bring unprecedented data processing challenges. Especially when it is necessary to play back surveillance videos at high speed to quickly locate key events or details, the massive amount of video data places extremely high demands on storage systems, processing chips, and network bandwidth.

[0003] Traditional video processing processes often face tremendous pressure because they need to decode each frame of an image in real time. As the frame rate increases dramatically, the amount of pixel data that the decoder needs to process increases exponentially, making it difficult for the decoding speed to keep up with the playback speed, resulting in serious playback delays. This delay not only affects monitoring efficiency, but may also miss the timely capture and processing of important information, reducing the overall efficiency of the monitoring system.

[0004] Therefore, how to solve the problem that the decoding speed cannot keep up and the delay becomes larger and larger is an issue that technical personnel in this field need to solve urgently. Summary of the invention

[0005] In view of this, the present invention provides a frame dropping method in a web player and a web player, which can effectively ensure that the playback effect is not delayed.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A frame dropping method in a web player comprises the following steps:

[0008] Step 1: Get the video source parameters and determine whether the I-frame interval is fixed;

[0009] Step 2: Detect the delay time difference between the decoding timestamp in the web player and the current display time. If the delay time difference reaches a preset threshold, it means that frame dropping is required, and the process goes to step 3. Otherwise, frame dropping is not required, and the process goes directly to step 4.

[0010] Step 3: Perform frame loss processing according to the result of determining whether the I frame interval is fixed in step 1;

[0011] Step 4: After the frame is dropped in step 3 or no frame drop processing is required in step 2, the data is transmitted to the video tag for rendering, and the condition judgment of step 2 is repeated for the next data frame.

[0012] Optionally, in step 1, the obtained I frame interval is passed to the web player in the form of parameter transmission.

[0013] Optionally, in step 2, the preset threshold refers to the delay time difference when the current image and the actual monitoring image are out of sync as observed by naked eyes.

[0014] Optionally, in step 3, the method for performing frame loss processing is:

[0015] When the I-frame interval is fixed, the frame loss is counted backwards according to the frame interval parameter until the next I-frame.

[0016] When the I-frame interval is not fixed, a memory buffer is established, and frames are dropped in this buffer in sequence until the next frame with an I-frame mark.

[0017] Optionally, after the frame loss processing in step three, if the timestamp of the I frame is still less than the current display time, step three is repeated.

[0018] A web player using any of the frame dropping methods, comprising:

[0019] The video source parameter acquisition module is used to obtain the video source parameters and determine whether the I frame interval is fixed;

[0020] The delay time difference calculation module is used to detect the delay time difference of the decoding timestamp in the web player that is less than the current display time. If the delay time difference reaches a preset threshold, it means that frame dropping is required, otherwise frame dropping is not required;

[0021] A frame loss processing module, used to perform frame loss processing according to the judgment result of whether the I frame interval is fixed;

[0022] The rendering module is used to transmit data to the video tag for rendering after the frame is lost or no frame loss processing is required, and continue to determine the next data frame.

[0023] It can be seen from the above technical solutions that the present invention provides a frame dropping method in a web player and a web player, which have the following beneficial effects compared with the prior art:

[0024] Compared with existing web players that generally do not have a frame dropping mechanism, or use automatic adjustment of the playback speed to offset such delays, the present invention can reduce CPU and GPU performance usage, especially in the field of monitoring, in the case of synchronous playback of multiple screens (especially 4K high-definition video), for example, at 2x speed, 1 / 2 of the video frames can be dropped, saving half of the decoding resources. The higher the speed, the more decoding resources are saved, reducing resource usage and effectively ensuring that the playback effect is not delayed. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0026] Figure 1 It is a flowchart of the frame loss method of the present invention;

[0027] Figure 2 This is a principle block diagram of the frame dropping method of the present invention. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] The method principle of the present invention:

[0030] Assume that the video is an I frame (IntraPicture, key frame) with a fixed interval and is independent of other frames. The regulation is to display a picture every 40ms (25 frames per second, and the first frame per second is an I frame), and the decoding capability of the device is limited. The fastest time to decode a picture is 42ms. In this way, the first picture should be displayed in 40ms, but because the decoding time takes 42ms, this picture is not displayed until 42ms, which is delayed by (42ms-40ms) 2ms than the stipulated time (40ms). When 24 pictures are decoded continuously, the delay reaches 24x2ms=48ms. Assuming that this 48ms delay is already very large, increasing the delay will cause the naked eye to obviously feel that the current picture is not synchronized with the actual monitoring picture. Therefore, it is necessary to discard the undecoded (25-24) I frames and not display them. Instead, start decoding and displaying the data of the next second frame. In this way, one frame of data is successfully discarded, and the picture is synchronized with the actual monitoring picture as much as possible.

[0031] If the interval is not fixed (the interval between two I frames is not of fixed length, for example, the first I frame and the second I frame are directly separated by 24 other frames, and the third I frame and the second I frame differ by 35 other frames), you cannot decode and play 24 frames and then drop the 25th frame, because it is possible that the next frame after dropping 25 frames is still not an I frame. In this way, the decoding will not be able to decode the complete picture, and the displayed picture will have a distorted screen, affecting the experience. Therefore, it is necessary to define a memory buffer area, and try to include 2 or more I frames in this area. For example: the player starts decoding and playing from the first key frame. Due to limited decoding capabilities, when the theoretical time should be immediately decoded and displayed for the second key frame, the player is still decoding the 5th frame before this key frame. That is to say, the player has to decode 5 more frames to reach this key frame. In this way, these 5 frames can be discarded and not decoded, and decoded directly from this key frame. The playback time is almost consistent with the theoretical playback time, making it imperceptible to be out of sync and causing no distorted screen.

[0032] Therefore, the embodiment of the present invention discloses a frame dropping method in a web player, see Figure 1-Figure 2 , including the following steps:

[0033] Step 1: Get the video source parameters and determine whether the I-frame interval is fixed.

[0034] The obtained I frame interval is passed to the web player as a parameter.

[0035] Step 2: Detect the delay time difference of the decoding timestamp in the web player that is less than the current display time. If the delay time difference reaches a preset threshold, it means that frame dropping is required, and proceed to step 3. Otherwise, frame dropping is not required, and proceed directly to step 4.

[0036] The preset threshold refers to the delay time difference when the current image and the actual monitoring image are out of sync as observed by the naked eye.

[0037] Step 3: Based on the result of determining whether the I frame interval is fixed in step 1, frame loss processing is performed:

[0038] When the I-frame interval is fixed, the dropped frames are counted backwards according to the frame interval parameter until the next I-frame. If the timestamp of this I-frame is still less than the current display time, this action is repeated without creating a buffer.

[0039] When the I-frame interval is not fixed, a memory buffer is established to store video frames and mark I frames. When frames need to be dropped, frames are dropped in this buffer one by one until the next frame marked with an I frame. If the timestamp of this I frame is still less than the current display time, this action is repeated.

[0040] Step 4: After the frame is dropped in step 3 or no frame drop processing is required in step 2, the data is transmitted to the video tag for rendering, and the condition judgment of step 2 is repeated for the next data frame until the video ends.

[0041] Another embodiment of the present invention discloses a web player using the above frame dropping method, comprising:

[0042] The video source parameter acquisition module is used to obtain the video source parameters and determine whether the I frame interval is fixed;

[0043] The delay time difference calculation module is used to detect the delay time difference of the decoding timestamp in the web player that is less than the current display time. If the delay time difference reaches a preset threshold, it means that frame dropping is required, otherwise frame dropping is not required;

[0044] A frame loss processing module, used to perform frame loss processing according to the judgment result of whether the I frame interval is fixed;

[0045] The rendering module is used to transmit data to the video tag for rendering after the frame is lost or no frame loss processing is required, and continue to determine the next data frame.

[0046] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0047] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A frame dropping method in a web player, characterized in that: The following steps are involved: Step 1: Get the video source parameters and determine whether the I-frame interval is fixed; Step 2: Detect the delay time difference between the decoding timestamp in the web player and the current display time. If the delay time difference reaches a preset threshold, it means that frame dropping is required, and the process goes to step 3. Otherwise, frame dropping is not required, and the process goes directly to step 4. Step 3: Perform frame loss processing according to the result of determining whether the I frame interval is fixed in step 1; Step 4: After the frame is dropped in step 3 or no frame drop processing is required in step 2, the data is transmitted to the video tag for rendering, and the condition judgment of step 2 is repeated for the next data frame.

2. A frame dropping method in a web player according to claim 1, characterized in that: In the step 1, the obtained I frame interval is passed to the web player in the form of parameter transmission.

3. The frame dropping method in a web player according to claim 1, characterized in that: In the step 2, the preset threshold refers to the delay time difference when the current picture and the actual monitoring picture are out of sync as observed by naked eyes.

4. The frame dropping method in a web player according to claim 1, characterized in that: In step 3, the method for performing frame loss processing is: When the I-frame interval is fixed, the frame loss is counted backwards according to the frame interval parameter until the next I-frame. When the I-frame interval is not fixed, a memory buffer is established, and frames are dropped in this buffer in sequence until the next frame with an I-frame mark.

5. The frame dropping method in a web player according to claim 1, characterized in that: After the frame loss processing in step three, if the timestamp of the I frame is still less than the current display time, step three is repeated.

6. A web player using the frame dropping method according to any one of claims 1 to 5, characterized in that: include: The video source parameter acquisition module is used to obtain the video source parameters and determine whether the I frame interval is fixed; The delay time difference calculation module is used to detect the delay time difference of the decoding timestamp in the web player that is less than the current display time. If the delay time difference reaches a preset threshold, it means that frame dropping is required, otherwise frame dropping is not required; A frame loss processing module, used to perform frame loss processing according to the judgment result of whether the I frame interval is fixed; The rendering module is used to transmit data to the video tag for rendering after the frame is lost or no frame loss processing is required, and continue to determine the next data frame.

Images