Live broadcast stream cache control method and device, storage medium and electronic equipment

By resetting the frame header of the live stream and implementing an intelligent frame dropping strategy, the stuttering problem of the live stream during network jitter was solved, achieving stable live stream caching and low-latency playback, thus improving the user experience.

CN118945391APending Publication Date: 2024-11-12JD DIGITS HAIYI INFORMATION TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410931408.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-11
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In existing technologies, network jitter causes stuttering in the live stream during playback, resulting in a poor viewing experience for users, especially with long wait times and high stuttering rates on the first screen.

Method used

By acquiring the current frame of the target live stream, the sequence duration of the Group of Pictures (GOP) is determined. When the number of keyframes reaches a preset number, the frame header of the video or audio/video stream is reset, and the frame number and frame tail are determined. Based on the reset frame header, frame number, and frame tail, the live stream is output, realizing caching according to duration and GOP number, and optimizing caching with intelligent frame dropping strategy.

Benefits of technology

It effectively reduces live streaming buffering, shortens the initial screen waiting time, improves the stability of the live stream and user experience, and supports low-latency playback.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118945391A_ABST
    Figure CN118945391A_ABST
Patent Text Reader

Abstract

The invention provides a live broadcast stream cache control method and device, a storage medium and electronic equipment, and relates to the technical field of telecommunication. The method comprises the following steps: acquiring a current frame of a target live broadcast stream; when the current frame is the key frame of the video stream or the audio / video stream, determining the sequence duration of a GOP (Group of Pictures); when the number of the key frames in the sequence duration is not less than the preset GOP number, resetting the frame header of the video stream or the audio / video stream, and determining the frame number and the frame end; and outputting the target live stream based on the reset frame header, frame number and frame tail. The live broadcast stream is cached according to the duration, the duration of the live broadcast stream can be guaranteed, the caching duration can be guaranteed under any GOP condition, and the problems that the live broadcast lagging rate is high and the first screen waiting time is long are solved.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0002] When the network environment is very unstable, as long as the network jitters slightly, it will cause the live stream distributed by subscribers (users) to be stuck when the player buffer is empty, which has a great impact on the user's viewing experience.

[0003] In the prior art, the native live broadcast protocol RTMP module does not cache the live broadcast streams published by publishers, resulting in a high freeze rate and a long first screen time when watching live broadcasts, causing users to wait a long time for the first screen of the video and freezes during the viewing process.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides a live streaming cache control method and device, a storage medium and an electronic device, which at least to a certain extent overcome the problems of high live streaming freeze rate and long first screen waiting time in related technologies.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a live stream cache control method is provided, including: obtaining a current frame of a target live stream; when the current frame is a key frame of a video stream or an audio and video stream, determining a sequence duration of a group of pictures GOP; when the number of key frames in the sequence duration is not less than a preset number of GOPs, resetting a frame header of the video stream or the audio and video stream, and determining a frame number and a frame tail; and outputting the target live stream based on the reset frame header, frame number and frame tail.

[0008] In some embodiments, after obtaining the current frame of the target live stream, the method further includes: when the current frame is a non-key frame, the target live stream is an audio stream and meets preset conditions, resetting the frame header of the audio stream, and determining the number of frames and the end of the frame; based on the reset frame header, number of frames and end of the frame, outputting the target live stream.

[0009] In some embodiments, resetting the frame header of the video stream or audio and video stream includes: obtaining the cached duration of the video stream or audio and video stream and the remaining cache duration minus a target GOP; when the cached duration and the remaining cache duration of the video stream or audio and video stream meet a first condition, cyclically clearing the GOP in the cache until a second condition is met and then resetting the frame header of the video stream or audio and video stream.

[0010] In some embodiments, satisfying the first condition includes: the cached duration of the video stream or audio and video stream is greater than the preset cache duration, or the remaining cache duration is not less than the preset cache duration; satisfying the second condition includes: the number of remaining key frames is less than the preset number of key frames or the remaining cache duration is less than the preset cache duration.

[0011] In some embodiments, resetting the frame header of the audio stream includes: obtaining the cache duration and the number of cached frames of the audio stream; when the cache duration and the number of cached frames of the audio stream meet a third condition, by cleaning up frame data and calculating the remaining duration, looping to clean up the frame sequence until a fourth condition is met, and then resetting the frame header of the audio stream.

[0012] In some embodiments, the third condition is satisfied when: the cache duration of the audio stream is not less than the preset cache duration and the number of cache frames is not less than the preset number of cache frames; the fourth condition is satisfied when: the number of remaining key frames is less than the preset number of key frames or the remaining cache duration is less than the preset cache duration.

[0013] In some embodiments, outputting the target live stream based on the reset frame header, frame number and frame tail also includes: based on the reset frame header, frame number and frame tail, determining whether frame loss occurs according to preset conditions, and then outputting the target live stream.

[0014] In some embodiments, determining whether to drop frames according to preset conditions includes: when the target live stream is an audio and video stream or a video stream and the cache duration is greater than or equal to a preset threshold, cyclically releasing a GOP until the remaining duration is greater than or equal to the duration.

[0015] In some embodiments, determining whether to drop frames according to preset conditions includes: when the target live stream is an audio stream and the cache duration is greater than or equal to a preset threshold, cyclically releasing an audio frame until the remaining duration is greater than or equal to the duration.

[0016] In some embodiments, determining whether to drop frames based on preset conditions includes: when the number of cached frames is greater than or equal to the configured maximum number of cached frames; if the target live stream is an audio and video stream or a video stream, retaining the last GOP; if the target live stream is an audio stream, dropping all cached audio frames.

[0017] In some embodiments, outputting the target live stream based on the reset frame header, frame number and frame tail also includes: when the total duration of the cached data exceeds the delay configuration, processing the timestamp of the cached data according to preset rules, and outputting the target live stream to the downstream or stream pulling end.

[0018] According to another aspect of the present disclosure, a live stream cache control device is also provided, including: a current frame acquisition module, used to acquire the current frame of the target live stream; a sequence duration acquisition module, used to determine the sequence duration of the image group GOP when the current frame is a key frame of the video stream or the audio and video stream; a frame header reset module, used to reset the frame header of the video stream or the audio and video stream and determine the number of frames and the frame tail when the number of key frames in the sequence duration is not less than the preset number of GOPs; an output module, used to output the target live stream based on the reset frame header, number of frames and frame tail.

[0019] According to another aspect of the present disclosure, an electronic device is also provided, which includes: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the live stream cache control methods described above by executing the executable instructions.

[0020] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the live stream cache control method described in any one of the above is implemented.

[0021] According to another aspect of the present disclosure, a computer program product is also provided, including a computer program, wherein when the computer program is executed by a processor, the live stream cache control method described above is implemented.

[0022] The live stream cache control method and device, storage medium and electronic device provided in the embodiments of the present disclosure obtain the current frame of the target live stream; when the current frame is a key frame of the video stream or the audio and video stream, determine the sequence duration of the image group GOP; when the number of key frames in the sequence duration is not less than the preset number of GOPs, reset the frame header of the video stream or the audio and video stream, and determine the number of frames and the frame tail; based on the reset frame header, number of frames and frame tail, output the target live stream. The present disclosure caches the live stream according to the duration, which can guarantee the duration of the live stream, so that the cache duration can be guaranteed regardless of the GOP situation, solving the problems of high live broadcast freeze rate and long first screen waiting time.

[0023] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0025] Figure 1 A schematic diagram showing a structure of a live stream cache control system in an embodiment of the present disclosure;

[0026] Figure 2 A flow chart of a live stream cache control method according to an embodiment of the present disclosure is shown;

[0027] Figure 3 A flowchart showing a specific example of a live stream cache control method according to an embodiment of the present disclosure;

[0028] Figure 4 A flowchart showing another specific example of a live stream cache control method in an embodiment of the present disclosure;

[0029] Figure 5 A flowchart showing another specific example of a live stream cache control method according to an embodiment of the present disclosure;

[0030] Figure 6 A flowchart of resetting the frame header of a video stream or an audio and video stream in a live stream cache control method according to an embodiment of the present disclosure is shown;

[0031] Figure 7 A flowchart showing a specific example of resetting the frame header of a video stream or an audio and video stream in a live stream cache control method according to an embodiment of the present disclosure;

[0032] Figure 8 A flowchart of resetting the frame header of an audio stream in a live stream cache control method according to an embodiment of the present disclosure is shown;

[0033] Fig. 9 A flowchart showing a specific example of resetting the frame header of an audio stream in a live stream buffer control method according to an embodiment of the present disclosure;

[0034] Fig.10 A flowchart showing whether to drop frames according to preset conditions in a live stream cache control method in an embodiment of the present disclosure is shown;

[0035] Fig.11 A flow chart showing an intelligent frame loss strategy in a live stream cache control method in an embodiment of the present disclosure is shown;

[0036] Fig.12 A flowchart showing a pure audio stream frame loss strategy in a live stream cache control method in an embodiment of the present disclosure is shown;

[0037] Fig.13 A flowchart showing a violent frame loss strategy in a live stream cache control method in an embodiment of the present disclosure is shown;

[0038] Fig.14A flowchart showing low latency in a live stream cache control method in an embodiment of the present disclosure is shown;

[0039] Fig.15 A schematic diagram showing a frame in a live stream cache control method according to an embodiment of the present disclosure;

[0040] Fig.16 A schematic diagram of a live stream cache control device according to an embodiment of the present disclosure is shown;

[0041] Fig.17 A structural block diagram of a computer device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0042] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0043] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0044] For ease of understanding, before introducing the embodiments of the present disclosure, several terms involved in the embodiments of the present disclosure are first explained as follows:

[0045] GOP: Group of Pictures. In video coding, GOP is a sequence of multiple frames, including one or more I frames (key frames), P frames (predicted frames) and B frames (bidirectionally predicted frames). GOP is a group of consecutive picture frames that are processed as a unit during compression.

[0046] RTMP: Real Time Messaging Protocol, live real-time message media transmission protocol;

[0047] CDN: Content Delivery Network, a distributed network built on and covered by the bearer network, consisting of edge node server clusters distributed in different regions;

[0048] NGX: Nginx, a high-performance HTTP and reverse proxy global WAN server.

[0049] The specific implementation of the embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0050] Figure 1 FIG. 1 shows an exemplary application system architecture diagram to which the live stream cache control method in the embodiment of the present disclosure can be applied. Figure 1 As shown, the system architecture may include a terminal device 101 , a network 102 and a server 103 .

[0051] The network 102 is a medium for providing a communication link between the terminal device 101 and the server 103, and can be a wired network or a wireless network.

[0052] Optionally, the wireless network or wired network described above uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, the data exchanged through the network is represented by technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPSec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.

[0053] The terminal device 101 can be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, smart speakers, smart watches, wearable devices, augmented reality devices, virtual reality devices, etc.

[0054] Optionally, the client of the application installed in different terminal devices 101 is the same, or the client of the same type of application based on different operating systems. Based on the different terminal platforms, the specific form of the client of the application can also be different, for example, the application client can be a mobile client, a PC client, etc.

[0055] The server 103 may be a server that provides various services, such as a background management server that provides support for the device operated by the user using the terminal device 101. The background management server may analyze and process the received request and other data, and feed back the processing results to the terminal device.

[0056] Optionally, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0057] In one example of the present disclosure, the server is used to obtain the current frame of the target live stream; when the current frame is a key frame of the video stream or the audio and video stream, the sequence duration of the image group GOP is determined; when the number of key frames in the sequence duration is not less than the preset number of GOPs, the frame header of the video stream or the audio and video stream is reset, and the number of frames and the frame tail are determined; based on the reset frame header, number of frames and frame tail, the target live stream is output.

[0058] Those skilled in the art will know that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration, and any number of terminal devices, networks and servers may be provided according to actual needs, and the embodiments of the present disclosure do not limit this.

[0059] Figure 2 A flow chart of a live stream cache control method according to an embodiment of the present disclosure is shown as follows: Figure 2 As shown, the live streaming cache control method provided in the embodiment of the present disclosure includes the following steps:

[0060] S202, obtaining the current frame of the target live stream.

[0061] It should be noted that the above live streaming can be the process from production to consumption of live streaming content, which mainly involves three key links: the push stream end, CDN, and the pull stream end. The push stream end is responsible for the collection, processing, encoding, and packaging of audio and video data, and pushes the data to the CDN edge node. CDN includes source stations and edge nodes. After receiving the push stream data, the edge node will push the stream to the superior until the source station, and the source station will perform transcoding, recording, etc. The pull stream end pulls the audio and video data from the CDN edge node according to the stream address.

[0062] S204: When the current frame is a key frame of a video stream or an audio / video stream, determine the sequence duration of a group of pictures GOP.

[0063] It should be noted that the sequence duration of GOP refers to the length of time from one I frame to the next I frame. This duration determines the interval between key frames in video encoding, and also affects the quality of the video stream, compression efficiency, and playback delay. When the GOP setting is small, the interval between I frames is shortened, which can reduce the first frame loading time and improve the real-time performance of the video, but it may increase the proportion of I frames and reduce the compression ratio and video quality. When the GOP setting is large, the interval between I frames increases, which can improve the compression efficiency, but it may increase the first frame loading time and cause delays in video playback. The sequence duration of GOP is a technical parameter. By reasonably setting the sequence duration of GOP, you can achieve a low-latency, high-compression-efficiency video playback effect while ensuring video quality.

[0064] For example, when the current frame is a key frame, the duration of the previous GOP sequence is calculated and updated.

[0065] S206, when the number of key frames in the sequence duration is not less than the preset number of GOPs, the frame header of the video stream or the audio and video stream is reset, and the number of frames and the frame tail are determined.

[0066] It should be noted that the above-mentioned preset number of GOPs is the expected number of frames in one GOP.

[0067] For example, determine whether the number of key frames is greater than or equal to the preset number of GOPs. If so, reset the frame header of the buffer cache video stream or audio and video stream, release the old frame sequence, record the frame number, and update the frame tail pointer.

[0068] S208, output the target live stream based on the reset frame header, frame number and frame tail.

[0069] The present invention caches the live stream according to the duration, which can ensure the duration of the live stream, so that the cache duration can be guaranteed regardless of the GOP situation, solving the problems of high live broadcast freeze rate and long first screen waiting time.

[0070] Figure 3A flowchart of a specific example of a live stream buffer control method according to an embodiment of the present disclosure is shown. Figure 3 As shown, the live streaming buffering control method provided in the embodiment of the present disclosure includes the following steps, which can buffer the pure audio stream according to the number of frames:

[0071] S302, when the current frame is a non-key frame, the target live stream is an audio stream and meets the preset conditions, reset the frame header of the audio stream, and determine the number of frames and the frame tail.

[0072] For example, when the current frame is not a key frame, the target live stream is a (pure) audio stream and the cached frame meets the configuration (equivalent to the above-mentioned preset conditions), the frame header of the Cache audio stream is reset, the old frame sequence is released, the number of frames is recorded, and the frame tail pointer is updated, where the cached frame meets the configuration, indicating that the number of cached frames is less than the preset number of cached frames.

[0073] S304: Output the target live stream based on the reset frame header, frame number and frame tail.

[0074] Figure 4 A flowchart showing another specific example of a live stream cache control method according to an embodiment of the present disclosure is shown as follows: Figure 4 As shown, in the live stream cache control method provided in the embodiment of the present disclosure, based on the reset frame header, frame number and frame tail, outputting the target live stream includes the following steps:

[0075] S402, when the total duration of the cached data exceeds the delay configuration, the timestamp of the cached data is processed according to a preset rule, and the target live stream is output to the downstream or pull stream end.

[0076] It should be noted that the above delay configuration may be a preset threshold for delay duration configuration.

[0077] Figure 5 A flowchart showing another specific example of a live stream cache control method according to an embodiment of the present disclosure is shown as follows: Figure 5 As shown, the live streaming cache control method provided in the embodiment of the present disclosure includes the following steps:

[0078] S501, determine whether it is a key frame, if so, jump to S502, if not, jump to S505;

[0079] S502, calculating and updating the duration of the previous GOP sequence;

[0080] S503, determine whether the number of key frames is greater than or equal to the set number of GOPs, if so, jump to S504, if not, jump to S507;

[0081] S504, reset the cache video stream (V / AV) frame header and release the old frame sequence, ngx_rtmp_live_reset_gop_header;

[0082] S505, determine whether it is a pure audio stream and the number of cached frames meets the configuration, if so, jump to S506, if not, jump to S507;

[0083] S506, reset the frame header of the cached audio stream (A) and release the old frame sequence, ngx rtmp_live_reset_pure_audio_header;

[0084] S507, record the frame number and update the frame end pointer.

[0085] The above S501-S507 is the logic for implementing cache frames, which can be understood as entering the cache Buffer after receiving the audio and video frames, and determining whether the current cache is full. If it exceeds the threshold, a reset operation will be triggered to delete the old frame and add the new frame.

[0086] In one embodiment of the present disclosure, Figure 6 As shown, the live stream cache control method provided in the embodiment of the present disclosure can reset the frame header of the video stream or the audio and video stream through the following steps, and can realize the cache GOP to support cache by GOP number and cache by duration, so as to ensure the duration of the live stream, so that the cache duration can be guaranteed regardless of the GOP situation, thereby effectively avoiding the jamming caused by insufficient cache and reducing the jamming rate:

[0087] S602, obtaining a cached duration and a remaining cached duration of a video stream or an audio and video stream, wherein the remaining cached duration indicates a remaining duration after subtracting a target GOP;

[0088] S604, when the cached duration and the remaining cached duration of the video stream or the audio and video stream meet the first condition, the GOP in the cache is cleared cyclically until the second condition is met and then the frame header of the video stream or the audio and video stream is reset.

[0089] It should be noted that the above target GOP may be the earliest GOP in time sequence.

[0090] For example, when the player needs three slices to achieve the goal, you can configure caching by GOP number and caching by duration at the same time, so that the cached live stream meets both the GOP number and the cache duration. In this way, when edge viewing triggers back-to-source streaming, you can instantly pull the cached stream from the transit (aggregation core or secondary parent) to cut out three slices, thereby achieving a fast viewing effect.

[0091] In one embodiment of the present disclosure, the first condition mentioned above includes: the cached duration of the video stream or the audio and video stream is greater than the preset cache duration, or the remaining cache duration is not less than the preset cache duration; the second condition mentioned above includes: the number of remaining key frames is less than the preset number of key frames or the remaining cache duration is less than the preset cache duration. By configuring the first condition and the second condition, the cached live stream satisfies both the number of GOPs and the cache duration.

[0092] In another embodiment of the present disclosure, the first condition is satisfied as follows: the cached time of the video stream or the audio and video stream is greater than the preset cache time and the remaining cache time is not less than the preset cache time.

[0093] In another embodiment of the present disclosure, the second condition is satisfied including: the number of remaining key frames is less than the preset number of key frames and the remaining cache duration is less than the preset cache duration.

[0094] In one embodiment of the present disclosure, Figure 7 As shown, the above live stream cache control method for resetting the frame header of the video stream or the audio and video stream specifically includes the following steps:

[0095] S701, calculate the cached time, and calculate the remaining cache time minus 1 gop;

[0096] S702, determine whether the cache_gop_duration (duration) is configured and whether the number of cached frames is less than or equal to the set cache duration or whether the remaining duration is less than the set cache duration. If so, end the process. If not, jump to S703;

[0097] S703, while loop clears cached GOP;

[0098] S704, releasing a group of GOP frame sequences;

[0099] S705, determine whether the number of remaining key frames is less than the number of set key frames, if so, jump to S707, if not, jump to S706;

[0100] S706, determine whether eache_gop_duration is configured and whether the remaining time is less than the set cache time. If so, jump to S707, if not, jump to S703;

[0101] S707, reset the cache frame sequence header and end the process.

[0102] In one embodiment of the present disclosure, Figure 8As shown, the live stream cache control method provided in the embodiment of the present disclosure can reset the frame header of the audio stream through the following steps, and can also support cache by frame number for pure audio stream, so as to ensure the duration of the live stream, so that the cache duration can be guaranteed regardless of the GOP situation, thereby effectively avoiding the jamming caused by insufficient cache and reducing the jamming rate:

[0103] S802, obtaining the buffering duration and buffering frame number of the audio stream;

[0104] S804: When the cache duration and the number of cache frames of the audio stream meet the third condition, the frame data is cleaned up and the remaining duration is calculated, and the frame sequence is cleaned up in a loop until the fourth condition is met, and then the frame header of the audio stream is reset.

[0105] In one embodiment of the present disclosure, the third condition is satisfied when: the cache duration of the audio stream is not less than the preset cache duration and the number of cache frames is not less than the preset number of cache frames; the fourth condition is satisfied when: the number of remaining key frames is less than the preset number of key frames or the remaining cache duration is less than the preset cache duration. By configuring the third and fourth conditions, the cached live stream also supports caching by frame number for pure audio streams.

[0106] In one embodiment of the present disclosure, Fig. 9 As shown, the frame header of the audio stream is reset in the live stream cache control method, which specifically includes the following steps:

[0107] S901, configure cache_gop_duration (duration) and determine whether the cache duration is less than the set cache duration. If so, end the process. If not, jump to S902;

[0108] S902, configure ceche_pure_audio_frame (pure audio architecture) and determine whether the number of cached frames is less than the set number of cached frames. If so, end the process. If not, jump to S903;

[0109] S903, while loop clears the frame sequence;

[0110] S904, clearing frame data and calculating remaining time;

[0111] S905, whether to configure cache_gop_duration, if yes, jump to S906, if not, jump to S908;

[0112] S906, whether the remaining time is less than or equal to the set cache time, if so, jump to S909, if not, jump to S907;

[0113] S907, whether cache_pure_audio_frame is configured and the number of remaining frames is less than or equal to the number of cache pure audio frames set, if yes, jump to S909, if not, jump to S903;

[0114] S908, whether cache_pure_audio_frame is configured and the number of remaining frames is less than or equal to the number of cache pure audio frames set;

[0115] S909, reset the cache frame sequence header and end the process.

[0116] In one example of the present disclosure, based on the reset frame header, frame number and frame tail, outputting the target live stream further includes: based on the reset frame header, frame number and frame tail, judging whether to drop frames according to preset conditions, and then outputting the target live stream. By deleting unnecessary frames of certain scenes, the freeze rate and the first opening time are solved.

[0117] For example, drop_gop (drop GOP mode), drop GOP supports three modes: intelligent drop GOP, pure audio stream frame drop and violent frame drop. It takes effect after the cache GOP mode is turned on. After the cache is turned on, Publishers will continuously distribute their cached live streams to the cache of Subscribers. The network conditions of different Subscribers are also different, which may cause too many data frames to accumulate in the buffer of Subscribers with poor network and insufficient bandwidth. If it reaches a certain threshold and is not cleared, it will cause the Subscribers cache data frame to overflow or abnormal, and will occupy more memory. According to different situations, drop_gop will use three strategies to drop frames: drop_gop_intelligently (intelligent drop GOP), drop_gop_rudely (violent frame drop), and drop_pure_audio (pure audio stream frame drop).

[0118] In one embodiment of the present disclosure, Fig.10 As shown, the live stream buffer control method described above specifically comprises the following steps of determining whether to drop frames according to preset conditions:

[0119] S1001, determine whether the configured drop_gop_times and the number of drops is greater than or equal to the configured number of drops, if so, jump to S1008, if not, jump to S1002;

[0120] S1002, calculate drop_delta;

[0121] S1003, determine whether the accumulated number of frames is greater than or equal to the maximum cache frame number out_queue, if so, jump to S1009, if not, jump to S1004;

[0122] S1004, determine whether drop_gop_threshold is equal to 0, if so, end the process, if not, jump to S1005;

[0123] S1005, determine whether drop delta is less than drop_gop_threshold, if so, end the process, if not, jump to S1006;

[0124] S1006, determine whether it is a pure audio stream, if so, jump to S1010, if not, jump to S1007;

[0125] S1007, determine whether the number of cached key frames is greater than 1, if so, jump to S1011, if not, end the process;

[0126] S1008, end the RTMP session;

[0127] S1009, using violent frame drop strategy ngx_rtmp_drop_gop_rudely;

[0128] S1010, adopt pure audio stream frame drop strategy ngx_rtmp_drop_pure_audio;

[0129] S1011, adopts the intelligent frame drop strategy ngx_rtmp_drop__gop_intelligently.

[0130] The present disclosure determines whether to drop frames according to preset conditions when distributing, that is, after sending a frame, this conditional judgment will be triggered, and attempts to determine whether to drop frames by calculating the difference in timestamps, checking the number and type of messages, and considering whether it is a pure audio or video stream. The frame buffer in the cache is discarded according to the GOP rule or violent frame dropping.

[0131] In an example of the present disclosure, judging whether to drop frames according to preset conditions includes: when the target live stream is an audio and video stream or a video stream and the cache duration is greater than or equal to a preset threshold, releasing a GOP in a loop until the remaining duration is greater than or equal to the duration, which is equivalent to the above-mentioned intelligent frame dropping strategy.

[0132] For example, the intelligent GOP drop strategy rop_gop_intelligently will drop the older GOPS in the Subscribers buffer and ensure that the remaining duration is greater than or equal to cache_gop_duration (duration). Each time a GOP is dropped, the intelligent GOP drop strategy calculates whether the remaining duration is greater than cache_gop_duration. If it is less than or equal to, the frame drop is completed; if it is greater, it will continue to detect the duration of the next GOP and calculate again whether the remaining duration after deletion is greater than cache_gop_duration. If the duration after detection is less than, the frame drop is completed, otherwise the next intelligent frame drop cycle will continue. Intelligent GOP drop strategy trigger conditions: live audio and video stream or pure video stream, Subscribers cache duration is greater than or equal to the duration set by drop_gop_threshold (threshold).

[0133] In an example of the present disclosure, judging whether to drop frames according to preset conditions includes: when the target live stream is an audio stream and the cache duration is greater than or equal to a preset threshold, cyclically releasing an audio frame until the remaining duration is greater than or equal to the duration. This is equivalent to the above-mentioned pure audio stream frame dropping strategy. In addition, the number and duration of cyclically released audio frames can be determined according to actual needs.

[0134] In one example of the present disclosure, determining whether to drop frames according to preset conditions includes: when the target live stream is an audio stream and the cache duration is greater than or equal to a preset threshold, randomly releasing at least one audio frame so that the remaining duration is greater than or equal to the duration.

[0135] For example, the pure audio stream frame drop policy drop_pure_audio: when cache_pure_aud io_frame is configured, the latest cache_pure_audio_frame pure audio frames in the subscribers buffer will be saved, and when cache_gop_duration is configured, frames of the corresponding duration will be saved. Triggering conditions for the pure audio stream frame drop policy: live pure audio stream, subscribers cache duration is greater than or equal to drop_gop_threshold (threshold).

[0136] In an example of the present disclosure, judging whether to drop frames according to preset conditions includes: when the number of cached frames is greater than or equal to the configured maximum number of cached frames; if the target live stream is an audio and video stream or a video stream, retaining the last GOP; if the target live stream is an audio stream, dropping all cached audio frames. This is equivalent to the above-mentioned violent frame dropping strategy.

[0137] For example, the violent frame drop strategy drop_gop_rudely will ignore the cache duration. In the case of live audio and video streams or pure video streams, only the last GOP in the Subscribers buffer (if there is a GOP) will be retained. In the case of pure audio streams, all cached pure audio frames will be dropped. Triggering conditions for the violent frame drop strategy: the number of cached frames is greater than or equal to the configured maximum number of cached frames (out_queue).

[0138] In one embodiment of the present disclosure, it also includes a playback timestamp clearing mode zero_play. Usually, the flash player of Subscribers has the function of counting the viewing time. If it is not turned on, the timestamp will be displayed as the broadcast time of Publishers, and it usually does not start from zero; after turning it on, the timestamp will be displayed as the viewing time of Subscribers, and the time will start from zero. In the CDN link, the roles such as source station, transit, and forwarding generally do not need to be turned on, and the edge nodes directly connected to the audience generally need to be turned on. The purpose of fast playback is achieved by adjusting the inter-frame timestamp.

[0139] In one embodiment of the present disclosure, a low-latency play function low_latency_play is also enabled. This configuration item belongs to a low-latency switch and is used in conjunction with the Cache function to maintain a specified buffer length and ensure that each playback end plays data at the same time to achieve synchronization of data playback.

[0140] In one embodiment of the present disclosure, it also includes setting a specified duration of low latency, low_latency_duration: the unit is seconds. This configuration is used to specify the delay requirement of low latency. For example, a low latency of 3s means that the total Duration of the last frame data timestamp received in the server plus the data frame buffered by the server plus the data frame buffered by the client is the set low latency duration. It can be understood as the playback delay duration of this link. If the playback end plays strictly according to the Duration, this delay time can be basically maintained stably.

[0141] In one embodiment of the present disclosure, Fig.11 As shown, the intelligent frame loss strategy in the above live stream cache control method specifically includes the following steps:

[0142] S1101, finding the first key frame position;

[0143] S1102, determine whether the key frame is still not found after searching the end of the cache sequence, if so, end the process, if not, jump to S1103;

[0144] S1103, counting drop_gop_times to record the number of frames k (start) before the first key frame;

[0145] S1104, for loop drop gop;

[0146] S1105, while loop releases a group of GOPs and counts the number of released frames I (new);

[0147] S1106, calculating the remaining time;

[0148] S1107, determine whether the remaining duration is less than or equal to the set cache duration or the number of remaining key frames is less than or equal to the set number of cache key frames, if so, jump to S1110, if not, jump to S1108;

[0149] S1108, detecting in advance and calculating the remaining time after releasing the next GOP sequence;

[0150] S1109, determine whether the key frame is not found or the remaining detection time is less than the set cache time, if so, jump to S1110, if not, jump to S1108;

[0151] S1110, determine whether I is greater than or equal to k, if so, jump to S1111, if not, jump to S1112;

[0152] S1111, update the timestamp, copy the start area frame to the new area, reset out_pos, and end the process;

[0153] S1112, release the Stand area frame;

[0154] S1113, update the timestamp, reset out_pos, and end the process.

[0155] It should be noted that the intelligent frame drop strategy is to intelligently discard GOPs (Group of Pictures) to keep the frame duration in the cache longer than a specific threshold. When the duration of the distributed frame exceeds the configured discarded GOP cache threshold, start discarding the GOP. Keep one or more key frames to ensure that the frame duration in the cache is greater than the cache interval and the configured GOP. If the frame duration of the start area is less than or equal to the frame duration to be discarded, the frames in the start area will be copied to the new area. If the frame duration of the start area is greater than the frame duration to be discarded, the frames in the start area will be discarded (memmove is not used. Memmove is used to copy bytes. If the target area and the source area overlap, memmove can ensure that the source string copies the bytes of the overlapping area to the target area before being overwritten, but the source content will be changed after the copy).

[0156] In one embodiment of the present disclosure, Fig.12As shown, the pure audio stream frame loss strategy in the above live stream cache control method specifically includes the following steps:

[0157] S1201, determine whether cache_pure_audio_frame is configured and whether the number of cache frames is less than 2xcache_pure_audio_frame, if yes, end the process, if no, jump to S1202;

[0158] S1202, determine whether caehe_gop_duration is configured and determine whether drop_delta <= cache_gop_duration, if yes, end the process, if not, jump to S1203;

[0159] S1203, count drop_gop_times;

[0160] S1204, while loop releases pure audio frames;

[0161] S1205, determine whether cache_gop_duration is configured, if yes, jump to S1206, if no, jump to S1209;

[0162] S1206, determine whether the remaining time is less than or equal to the set cache time, if so, jump to S1208, if not, jump to S1207;

[0163] S1207, determine whether cache_pure_audio_frame is configured and whether the remaining number of frames is less than or equal to the set number of cache pure audio frames, if so, jump to S1208, if not, jump to S1204;

[0164] S1208, update the timestamp, reset out_pos, and end the process;

[0165] S1209, determine whether cache_pure_audio_frame is configured and whether the remaining number of frames is less than or equal to the set cache pure audio frame number, if so, jump to S1208, if not, jump to S1204.

[0166] In one embodiment of the present disclosure, Fig.13 As shown, the violent frame loss strategy in the above live stream cache control method specifically includes the following steps:

[0167] S1301, record j=out_last, count drop_gop_times;

[0168] S1302, determine whether the number of cached key frames is greater than or equal to 1, if so, jump to S1306, if not, jump to S1303;

[0169] S1303, while loop releases the frame sequence from the beginning;

[0170] S1304, determine if i!=j?, if yes, jump to S1303, if no, jump to S1305;

[0171] S1305, update time, reset out_pos, and end the process;

[0172] S1306, search for the first key frame position from the end of the cache.

[0173] It should be noted that the violent frame drop strategy is the frame drop logic of GOP in the video stream when the network condition is poor. If the number of frames is greater than or equal to the value of out_queue, GOPs (frame groups) will be dropped. If the number of distributed key frames (I frames) is greater than 1, one key frame will be retained; otherwise, all frames will be violently dropped. For example, when the network condition of the downstream subscription is very bad and may be blocked; or, if the configured out_queue value is greater than twice 2*cache_pure_audio_frame, the pure audio stream should be processed separately in the ngx_rtmp_drop_pure_audio function. Otherwise, all audio frames will be dropped here. The default values ​​of out_queue and cache_pure_audio_frame are 256 and 200 respectively. However, the online out_queue is usually configured to 4096, so pure audio frames will not reach this function.

[0174] In one embodiment of the present disclosure, based on the reset frame header, frame number and frame tail, outputting the target live stream also includes: when the total duration of the cached data exceeds the delay configuration, processing the timestamp of the cached data according to a preset rule, and outputting the target live stream to the downstream or stream pulling end. Through the low-latency setting, it is possible to ensure that the user can achieve the precise delay specified by the user on the basis of opening in seconds.

[0175] In one embodiment of the present disclosure, Fig.14 As shown in the low-latency flow chart, in the above live stream cache control method, when the total duration of cached data exceeds the delay configuration, the timestamp of the cached data is processed according to the preset rules, and the target live stream is output to the downstream or pull end, which specifically includes the following steps:

[0176] S1401, obtaining Video / audio (equivalent to the above-mentioned target live stream);

[0177] S1402, determine whether to enable low latency, if yes, jump to S1404, if not, jump to S1403;

[0178] S1403, normal sending process;

[0179] S1404, determine whether it has been processed, if so, jump to S1403, if not, jump to S1405;

[0180] S1405, setting the basic pointer position;

[0181] S1406, determine whether it is single audio / no key frame, if yes, jump to S1407, if no, jump to S1408;

[0182] S1407, set Absolute=1;

[0183] S1408, calculating the remaining cache duration;

[0184] S1409, determine whether it is a Key, if so, jump to S1410, if not, jump to S1411;

[0185] S1410, update latency Start, jump to S1411;

[0186] S1411, determine whether the cache is less than or equal to the low latency duration, if so, jump to S1412, if not, jump to S1408;

[0187] S1412, check if Absolute=1? If so, jump to S1412, if not, jump to S1414;

[0188] S1413, update latency Start, jump to S1414;

[0189] S1414, obtaining the range in the cache;

[0190] S1415, set A / V header time stamp;

[0191] S1416, Send Cache;

[0192] S1417, determine whether Frame is between Start-pos, if so, jump to S1418, if not, jump to S1420;

[0193] S1418, determine whether it is audio data, if so, jump to S1419, if not, jump to S1422;

[0194] S1419, discard Continue and end the process;

[0195] S1420, directly send cache data;

[0196] S1421, determine whether the cache is sent, if so, end the process, if not, jump to S1419;

[0197] S1422, restore the original data of the chain;

[0198] S1423, re-encapsulate into an RTMP data packet with delta=0;

[0199] S1424, send data, jump to S1421.

[0200] The present disclosure implements a low-latency process, which is a low-latency logic implemented based on the cache mode. If the total duration of the cached data exceeds the delay configuration, the timestamp of the cached data will be processed according to certain rules and distributed to the downstream or streaming end at the same time.

[0201] In a specific example, the low-latency implementation logic includes: in the live broadcast server, adjusting the inter-frame timestamps to achieve fast playback. Fig.15 (Frame diagram) as shown: A represents Audio, that is, audio frame; V i Vp stands for Video, an I frame in a video frame; Vb stands for Video, a P frame in a video frame; I frame is an intra-coded frame (also called a key frame), P frame is a forward predicted frame (forward reference frame), and B frame is a bidirectional interpolated frame (bidirectional reference frame). Simply put, an I frame is a complete picture, while P frame and B frame record the changes relative to the I frame. Without I frame, P frame and B frame cannot be decoded. If the number of key frames (I frames) sent is greater than 1, one key frame is retained; otherwise, all frames are rudely discarded.

[0202] Fig.15 In the first step, this frame sequence is the frame sequence in the cache. The frame sequence in the cache is strictly arranged in the order of reception and the ascending sequence of the timestamps of the same type of frames. It is a circular queue with right in and left out. Whenever the frame sequence cached in the cache exceeds the maximum chache duration, it will be cleared backward from position 0 to clear the duration of a GOP sequence. This GOP sequence is the duration of the Video sequence, but the interlaced Audio is cleared synchronously to maintain the cache size.

[0203] The second step is to play the request, locate the starting position of the frame sequence that needs to be sent to the user, and set two pointers: start pointer: the starting position of the video data to be sent. pos: the actual position where the low-latency data starts.

[0204] The third step is to set the initial start = pos and set them to the head position of the cache, that is, position 0. The duration is calculated according to the frame type, that is, Video / Audio uses which frame type according to their respective timestamp sequences, based on the frame type of the latest data frame.

[0205] Step 4: Get the frame information at the pos position and determine whether the current frame is an IDR frame of Video. If it is an IDR frame, update the start pointer position to point to IDR. Execute downward.

[0206] The fifth step is to determine whether the duration of the pos position is greater than the low latency duration. If it is greater than the low latency duration, pos moves to the next of pos, that is, moves one position to the right, and returns to the fourth step; if they are equal, select the pos position and execute downward.

[0207] Step 6. So far, two pointer positions have been selected, namely 1 is the start position and 2 is the pos position. The characteristic is that start is the most recent IDR frame that pos closely follows. That is, pos is in the GOP sequence starting with the IDR of start or pos is the starting IDR of this GOP sequence (start = pos).

[0208] Step 7. Take out the timestamp of the frame information at pos as the starting absolute timestamp. For the data frames between start and pos, keep the video frames, set the frame interval duration to 0, and discard the audio frames.

[0209] Step 8: Insert all data from start to the end of the cache into the send queue of Play. Follow the description of step 7.

[0210] In the ninth step, the new audio and video data is directly inserted into the sending queue. At this point, the low-latency playback process is completed.

[0211] The present disclosure is applied to CDN live streaming to support cache caching, and the main problems to be overcome are the freeze rate and the first screen time. With the cache, the ability to resist network jitter is enhanced, the freeze rate is naturally low, and the first screen time can be achieved in seconds. And it supports low latency function. The purpose of this low latency is to ensure that the user can achieve the precise delay specified by the user on the basis of opening in seconds. The delay currently refers to the live broadcast delay distributed from the Edge to the client. If the low latency function is configured for the entire link, when the network is stable and normal, the overall delay of the link is basically equal to the dealy time specified by the low latency. Low latency is achieved on the basis of cache.

[0212] Based on the same inventive concept, the embodiment of the present disclosure also provides a live stream cache control device, as described in the following embodiment. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0213] Fig.16 A schematic diagram of a live stream cache control device according to an embodiment of the present disclosure is shown. Fig.16 As shown, the device includes: a current frame acquisition module 161, a sequence duration acquisition module 162, a frame header resetting module 163 and an output module 164.

[0214] Among them, the current frame acquisition module 161 is used to obtain the current frame of the target live stream;

[0215] The sequence duration acquisition module 162 is used to determine the sequence duration of the group of pictures GOP when the current frame is a key frame of the video stream or the audio and video stream;

[0216] The frame header resetting module 163 is used to reset the frame header of the video stream or the audio and video stream and determine the number of frames and the frame tail when the number of key frames in the sequence duration is not less than the preset number of GOPs;

[0217] The output module 164 is used to output the target live stream based on the reset frame header, frame number and frame tail.

[0218] It should be noted that the above current frame acquisition module 161, sequence duration acquisition module 162, frame header reset module 163 and output module 164 correspond to S202 to S208 in the method embodiment, and the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above method embodiment. It should be noted that the above modules as part of the device can be executed in a computer system such as a set of computer executable instructions.

[0219] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0220] Refer to the following Fig.17 1700 according to this embodiment of the present disclosure is described. Fig.17 The electronic device 1700 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0221] like Fig.17 As shown, the electronic device 1700 is in the form of a general computing device. The components of the electronic device 1700 may include but are not limited to: at least one processing unit 1710, at least one storage unit 1720, and a bus 1730 connecting different system components (including the storage unit 1720 and the processing unit 1710).

[0222] The storage unit stores program codes, which can be executed by the processing unit 1710, so that the processing unit 1710 executes the steps described in the above “exemplary method” section of this specification according to various exemplary embodiments of the present disclosure.

[0223] For example, the processing unit 1710 can execute the following steps of the above method embodiment: obtain the current frame of the target live stream; when the current frame is a key frame of the video stream or the audio and video stream, determine the sequence length of the image group GOP; when the number of key frames in the sequence length is not less than the preset number of GOPs, reset the frame header of the video stream or the audio and video stream, and determine the number of frames and the frame tail; based on the reset frame header, number of frames and frame tail, output the target live stream.

[0224] The storage unit 1720 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 17201 and / or a cache storage unit 17202 , and may further include a read-only storage unit (ROM) 17203 .

[0225] The storage unit 1720 may also include a program / utility 17204 having a set (at least one) of program modules 17205, such program modules 17205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0226] Bus 1730 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0227] The electronic device 1700 may also communicate with one or more external devices 1740 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1700, and / or communicate with any device that enables the electronic device 1700 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1750. Furthermore, the electronic device 1700 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1760. As shown, the network adapter 1760 communicates with other modules of the electronic device 1700 via a bus 1730. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0228] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0229] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer program product, which includes: a computer program, which implements the above-mentioned live stream cache control method when executed by a processor.

[0230] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. A program product capable of implementing the above method of the present disclosure is stored thereon. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above “Exemplary Method” section of this specification.

[0231] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0232] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0233] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0234] In a specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0235] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0236] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0237] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0238] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A live stream cache control method, characterized in that: include: Get the current frame of the target live stream; When the current frame is a key frame of a video stream or an audio / video stream, determining a sequence duration of a group of pictures GOP; When the number of key frames in the sequence duration is not less than the preset number of GOPs, the frame header of the video stream or the audio and video stream is reset, and the number of frames and the frame tail are determined; Based on the reset frame header, frame number and frame tail, the target live stream is output.

2. The live stream cache control method according to claim 1, characterized in that: After obtaining the current frame of the target live stream, the method further includes: When the current frame is a non-key frame, the target live stream is an audio stream and meets a preset condition, reset the frame header of the audio stream, and determine the number of frames and the frame tail; Based on the reset frame header, frame number and frame tail, the target live stream is output.

3. The live stream cache control method according to claim 1, characterized in that: The resetting of the frame header of the video stream or the audio and video stream includes: Obtaining the cached duration of the video stream or the audio and video stream and the remaining cached duration minus one target GOP; When the cached duration and the remaining cached duration of the video stream or the audio and video stream meet the first condition, the GOP in the cache is cleared cyclically until the second condition is met and then the frame header of the video stream or the audio and video stream is reset.

4. The live stream cache control method according to claim 3, characterized in that: The first condition is satisfied when: the cached duration of the video stream or audio / video stream is greater than the preset cache duration, or the remaining cache duration is not less than the preset cache duration; the second condition is satisfied when: the number of remaining key frames is less than the preset number of key frames or the remaining cache duration is less than the preset cache duration.

5. The live stream cache control method according to claim 2, characterized in that: The resetting the frame header of the audio stream comprises: Obtaining the cache duration and cache frame number of the audio stream; When the cache duration of the audio stream and the number of cache frames meet the third condition, the frame header of the audio stream is reset by cleaning up the frame data and calculating the remaining duration, and cyclically cleaning up the frame sequence until the fourth condition is met.

6. The live stream cache control method according to claim 5, characterized in that: The third condition is satisfied when: the cache duration of the audio stream is not less than the preset cache duration and the number of cache frames is not less than the preset number of cache frames; the fourth condition is satisfied when: the number of remaining key frames is less than the preset number of key frames or the remaining cache duration is less than the preset cache duration.

7. The live streaming buffer control method according to any one of claims 1 or 2, characterized in that: The outputting the target live stream based on the reset frame header, frame number and frame tail further comprises: Based on the reset frame header, frame number and frame tail, determine whether to drop frames according to preset conditions, and then output the target live stream.

8. The live stream cache control method according to claim 7, characterized in that: The determining whether to drop frames according to a preset condition includes: When the target live stream is an audio and video stream or a video stream and the cache duration is greater than or equal to a preset threshold, a GOP is released in a loop until the remaining duration is greater than or equal to the duration.

9. The live streaming buffer control method according to claim 7, characterized in that: The determining whether to drop frames according to a preset condition includes: When the target live stream is an audio stream and the cache duration is greater than or equal to a preset threshold, an audio frame is released in a loop until the remaining duration is greater than or equal to the duration.

10. The live streaming buffer control method according to claim 7, characterized in that: The determining whether to drop frames according to a preset condition includes: When the number of cached frames is greater than or equal to the configured maximum number of cached frames; If the target live stream is an audio / video stream or a video stream, retain the last GOP; If the target live stream is an audio stream, all cached audio frames are discarded.

11. The live streaming buffer control method according to any one of claims 1 or 2, characterized in that: The outputting the target live stream based on the reset frame header, frame number and frame tail further comprises: When the total duration of the cached data exceeds the delay configuration, the timestamp of the cached data is processed according to the preset rules, and the target live stream is output to the downstream or pull stream end.

12. A live stream cache control device, characterized in that: include: The current frame acquisition module is used to obtain the current frame of the target live stream; A sequence duration acquisition module, used for determining the sequence duration of a group of pictures GOP when the current frame is a key frame of a video stream or an audio and video stream; A frame header resetting module, used for resetting the frame header of the video stream or the audio and video stream and determining the number of frames and the frame tail when the number of key frames in the sequence duration is not less than the preset number of GOPs; The output module is used to output the target live stream based on the reset frame header, frame number and frame tail.

13. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the live stream cache control method according to any one of claims 1 to 11 by executing the executable instructions.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the live stream cache control method according to any one of claims 1 to 11 is implemented.