Picture replaying method, system and equipment of real-time video and storage medium

By detecting video playback information, combining the network cache time to calculate the replay time, extracting and playing pre-built video frames, the problem of video playback lag in weak network environments is solved, and user experience and video fluency are improved.

CN119071580APending Publication Date: 2024-12-03BIGO TECH PTE LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411114793.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-14
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

In a weak network environment, the existing technology is difficult to effectively solve the problem of lag in video playback, especially when the network cache is empty and no new data arrives for a long time, the user experience is affected.

Method used

By detecting data broadcast information and video stuttering information, we judge whether to replay the picture, and calculate the replay time based on the network cache time, extract the video screen of the corresponding number of frames from the pre-constructed replay queue, and add it to the play queue frame by frame, to achieve Replay the screen to reduce lag and improve smoothness.

Benefits of technology

When the network cache is empty and no new data arrives for a long time, screen replay is adaptive to ensure that the replayed screen is connected with real-time video, reduce playback lag, and improve user viewing experience and video playback fluency and stability .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119071580A_ABST
    Figure CN119071580A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses a picture replaying method, system and equipment of a real-time video and a storage medium. According to the technical scheme provided by the embodiment of the invention, the image replay judgment of the current real-time video is carried out based on the data play-free information and the video lag information by detecting the data play-free information and the video lag information of the current real-time video; under the condition that it is determined that the picture replay of the current real-time video is carried out, the replay duration is calculated based on the network cache duration of the current real-time video; determining the number of replay frames according to the replay duration, and extracting video pictures corresponding to the number of replay frames from a pre-constructed replay queue in a descending order according to the playing position of the current real-time video as replay pictures; and adding the replay pictures to the front end of a playing queue of the current real-time video frame by frame according to the sequence number sequence, and playing the replay pictures based on the playing queue. By adopting the technical means, the playing lagging can be reduced, the video playing fluency and stability are improved, and the watching experience of a user is further improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of video technology, and in particular to a method, system, device and storage medium for replaying real-time video images. Background Art

[0002] At present, in real-time video calls, live video broadcasts and other scenarios, video freezes have a great impact on users' subjective feelings and are an important indicator for evaluating real-time video quality. In order to maintain low freezes in video playback under weak network conditions, a certain length of network cache is usually set to reduce freezes by increasing video delay.

[0003] However, simply setting up a network cache is difficult to cover all video freeze situations. Since the network cache length is limited, when the network cache is empty and no new data arrives for a long time, freeze problems will still occur, affecting the user's viewing experience. Summary of the invention

[0004] The embodiments of the present application provide a method, system, device and storage medium for replaying real-time video images, which can adaptively replay real-time video images and solve the problem of playback freezes in real-time video.

[0005] In a first aspect, an embodiment of the present application provides a method for replaying a real-time video, comprising:

[0006] Detecting data blanking information and video freeze information of the current real-time video, and making a picture replay judgment of the current real-time video based on the data blanking information and video freeze information;

[0007] When it is determined to replay the current real-time video, the replay duration is calculated based on the network cache duration of the current real-time video;

[0008] Determine the number of replay frames according to the replay duration, and extract the video images corresponding to the number of replay frames from the pre-built replay queue in descending order according to the playback position of the current real-time video as the replay images. The replay queue is used to store the completed video images in the order of the serial numbers.

[0009] The replay images are added frame by frame to the front end of the play queue of the current real-time video according to the sequence number order, and the replay images are played based on the play queue.

[0010] In a second aspect, an embodiment of the present application provides a real-time video screen replay system, including:

[0011] A judgment module is configured to detect data blanking information and video freeze information of the current real-time video, and perform a picture replay judgment of the current real-time video based on the data blanking information and the video freeze information;

[0012] A duration calculation module, configured to calculate the replay duration based on the network cache duration of the current real-time video when it is determined to replay the current real-time video;

[0013] The picture extraction module is configured to determine the number of replay frames according to the replay duration, and extract the video pictures corresponding to the number of replay frames from the pre-built replay queue in descending order according to the playback position of the current real-time video as the replay pictures. The replay queue is used to store the video pictures that have been played in order of sequence numbers;

[0014] The replay module is configured to add the replay picture frame by frame to the front end of the play queue of the current real-time video according to the sequence number order, and play the replay picture based on the play queue.

[0015] In a third aspect, an embodiment of the present application provides a real-time video screen replay device, including:

[0016] memory and one or more processors;

[0017] The memory is configured to store one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the real-time video screen replay method as described in the first aspect.

[0019] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a computer processor, they are configured to execute the real-time video screen replay method as described in the first aspect.

[0020] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed on a computer or a processor, the computer or the processor executes the real-time video screen replay method as described in the first aspect.

[0021] The embodiment of the present application detects the data empty information and video freeze information of the current real-time video, and makes a judgment on the screen replay of the current real-time video based on the data empty information and the video freeze information; when it is determined that the screen replay of the current real-time video is to be performed, the replay duration is calculated based on the network cache duration of the current real-time video; the number of replay frames is determined according to the replay duration, and the video screens corresponding to the number of replay frames are extracted in descending order from the pre-built replay queue according to the playback position of the current real-time video as the replay screens, and the replay queue is used to store the video screens that have been played according to the sequence number sequence; the replay screens are added frame by frame to the front end of the playback queue of the current real-time video according to the sequence number sequence, and the replay screens are played based on the playback queue. The above-mentioned technical means are adopted, and when it is determined that the screen replay is to be performed, the replay duration is accurately calculated in combination with the network cache duration, and the appropriate replay screen is selected for replay according to the replay duration, so that the screen can be adaptively replayed when the network cache is empty and no new data arrives for a long time, and the replay screen can be accurately connected with the real-time video screen, reducing the playback freeze, improving the smoothness and stability of the video playback, and thus improving the user's viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a flow chart of a method for replaying a real-time video provided by an embodiment of the present application;

[0023] Figure 2 is a schematic diagram of caching of played video images in an embodiment of the present application;

[0024] Figure 3 This is a flow chart of circular caching of played video images in an embodiment of the present application;

[0025] Figure 4 is a calculation flow chart of the replay duration in an embodiment of the present application;

[0026] Figure 5 is a playback flow chart of the replay screen in the embodiment of the present application;

[0027] Figure 6 It is a structural diagram of a real-time video screen replay system provided in an embodiment of the present application;

[0028] Figure 7 It is a structural diagram of a real-time video screen replay device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0030] The real-time video screen replay method provided in this application aims to determine the situation of screen replay, accurately calculate the replay duration in combination with the network cache duration, and select the appropriate replay screen for replay according to the replay duration, so as to reduce playback jams and improve the user's viewing experience.

[0031] In real-time video scenarios, in order to meet the viewing needs of users in different network environments, it is necessary to set corresponding countermeasures for weak network environments to improve the quality of real-time video. Among them, stuttering is an important indicator for evaluating the quality of real-time video, which has a great impact on the subjective feelings of users. How to maintain a low stuttering rate of playback in a weak network environment is very important for the user's QoE (Quality of Experience). Playback stuttering is mainly caused by a weak network environment. Generally speaking, before decoding and playing the video, there will be a series of operations to reduce stuttering as much as possible, such as setting network cache and reducing stuttering by increasing delay. However, this strategy cannot be perfect. In a heterogeneous network environment, there are still many stutterings that cannot be covered by the network cache. These stutterings will also seriously affect the user's perception and should also be taken seriously. For example, in some extremely weak network situations, if the network cache has not received new data for a long time and is in an empty state, there will still be video stuttering problems, resulting in poor user experience.

[0032] Based on this, a method for replaying a real-time video in an embodiment of the present application is provided to solve the problem of playback freezes in real-time video. By caching the played video images, when it is determined that the network cache is empty and no new data has arrived for a long time, a suitable video image is dynamically selected for replay as a transitional segment before the arrival of new data, thereby coping with freezes that cannot be covered by the network cache, reducing freezes from the user's visual sense, and improving the user's viewing experience.

[0033] Example:

[0034] Figure 1 A flowchart of a real-time video screen replay method provided in an embodiment of the present application is given. The real-time video screen replay method provided in this embodiment can be executed by a real-time video screen replay device, which can be implemented by software and / or hardware. The real-time video screen replay device can be composed of two or more physical entities, or it can be composed of one physical entity. Generally speaking, the real-time video screen replay device can be a terminal device such as a computer, a mobile phone, a tablet, etc. for implementing video calls and real-time video live broadcasts.

[0035] The following is a description of the method for replaying a real-time video by taking the problem of a real-time video playback being stuck as an example. Figure 1 The real-time video screen replay method specifically includes:

[0036] S110: Detect data blanking information and video freeze information of the current real-time video, and make a picture replay judgment of the current real-time video based on the data blanking information and the video freeze information.

[0037] When replaying the screen, this application sets strategies such as data cache of the played video screen, replay trigger conditions, replay duration decision, and screen replay method, so as to deal with some stuttering problems that cannot be covered by network cache, reduce stuttering from the visual sense, and improve the user's viewing experience.

[0038] Before replaying the screen, a data cache is pre-built to store the played video screen, so that the screen can be replayed when the screen replay is triggered later. Screen replay is to replay the video screen that has been played by the current real-time video. The storage of the video screen that has been played is particularly important. It is necessary to consider the information value of the played video screen and the burden of data cache on the device overhead. Therefore, this application selects the original screen of the real-time video played for the first time as the screen that needs to be cached, and reduces the overhead problems caused by data storage and memory development through a limited-size circular cache.

[0039] Among them, a replay queue is pre-built to store the media data of the played video screen. The media data is required to be the original screen, so that the screen can be directly selected for replay later. The replay queue is a circular cache queue of fixed length, which is used to cyclically cover and store the video screen that has been completed. According to the video screen that has been completed by the current real-time video (i.e., the historical video screen), it is extracted in real time and placed in the corresponding cache position in order. If the cache position has stored the video screen, the new video screen will overwrite the original cached video screen. It can be understood that in order to reduce cache occupancy and save storage overhead, the historical video screen that is longer from the current time is overwritten and replaced with the newly cached historical video screen.

[0040] Optionally, in order to optimize the replay effect and save the picture information related to the historical video screen, the present application also constructs an additional picture information queue to cache the relevant picture information of the historical video screen, such as video format, timestamp, picture size, etc. The picture information queue is mapped one-to-one with the replay queue of the storage media data queue, and the current cache position is recorded through the index. When there is a played video screen that needs to be cached, the media data corresponding to the played video screen is copied to the index position of the playback queue, and the corresponding picture information is copied to the index position of the picture information queue. The index indicates the current cache position of the played video screen. Then, the index is moved backward, and when the index reaches the last position, it returns to the first position to perform cyclic overwriting storage of the played video screen.

[0041] For example, refer to Figure 3 During the playback of real-time video, each time a frame of video is played in real time, the caching operation of the played video is triggered. By judging whether the played video is the original picture played for the first time, the played video is cached only when it is determined that the video is the original picture played for the first time. It is understandable that playing the same historical picture multiple times will affect the user's viewing experience, so only one replay is performed for the played video. When the caching of the played video is triggered, the media data is cached to the current cache position of the playback queue, and the picture information is cached to the current cache position of the picture information queue according to the current cache position indicated by the index of the current replay queue and the picture information queue, and a one-to-one corresponding index relationship is established between the two to facilitate the subsequent call of media data and picture information.

[0042] After completing the data cache of the played video screen, the set replay judgment conditions can be used to determine whether to replay the current real-time video screen, so that the corresponding video screen can be extracted from the data cache for screen replay when the screen replay is triggered.

[0043] Among them, considering the network jitter in the conventional weak network environment, the network cache can be used to deal with it to ensure the smoothness of video playback. However, when the network cache is empty and no new data arrives for a long time, the network cache cannot cope with this stuttering problem. Therefore, this application determines whether to replay the picture by combining the data empty information and video stuttering information of the current real-time video. Among them, the data empty information indicates the empty situation of the network cache, and the video stuttering information indicates the stuttering situation of the next frame of the current real-time video. Through the above-mentioned data empty information and video stuttering information, it can be determined whether the current real-time video is in a situation where the network cache is empty and no new data arrives for a long time, and then the trigger judgment of the picture replay can be accurately performed.

[0044] Optionally, judging the current real-time video screen replay based on the data blanking information and the video freeze information includes:

[0045] When it is determined based on the data empty information that the decoding queue and the playback queue of the current real-time video are both empty, and when it is determined based on the video freeze information that the playback time of the current real-time video from the previous frame of video reaches the set threshold information, it is determined to replay the current real-time video.

[0046] Considering that although screen replay can reduce visual freezes to a certain extent, too frequent replays can also bring a bad sensory experience, this application designs a reasonable replay trigger condition to make screen replay judgments and trigger screen replay when necessary. The conditions for triggering screen replay are expressed as follows:

[0047] Screen replay = data empty && about to freeze

[0048] The judgment condition of data broadcast empty is expressed as:

[0049] DecodeQueue.empty()&&DisplayQueue.empty()

[0050] DecodeQueue.empty() indicates that the decoding queue of the current real-time video is empty, and DisplayQueue.empty() indicates that the playback queue of the current real-time video is empty. Optionally, in the actual process of determining that the data is empty, the length of the decoding queue and the playback queue may be compared with a corresponding length threshold, and the data may be determined to be empty when the length is lower than the set length threshold.

[0051] The condition for judging whether the video is about to freeze is determined by comparing the playback time of the current real-time video from the previous frame of the video with the corresponding threshold information, and determining that the current real-time video is about to freeze when the playback time reaches the threshold information. According to actual needs, the threshold information can be one or more. When there are multiple threshold information, the judgment method can be that only one of them is met, that is, it is determined that the current real-time video is about to freeze, or it can be that when multiple threshold information is met at the same time, it is determined that the current real-time video is about to freeze. This application does not impose fixed restrictions on the setting of threshold information, and will not be elaborated here.

[0052] Optionally, determining, based on the video freeze information, whether the playback time of the current real-time video from the previous frame of the video reaches a set threshold value includes:

[0053] When the playback time of the current real-time video from the last frame of the video reaches the set first freeze threshold, determining that the playback time of the current real-time video from the last frame of the video reaches the set threshold; or

[0054] When the playback time of the current real-time video from the previous frame of video reaches the set second freeze threshold, and the playback time of the current real-time video from the previous frame of video reaches the set picture playback time interval information, it is determined that the playback time of the current real-time video from the previous frame of video reaches the set threshold information.

[0055] The judgment condition of the jam is expressed as:

[0056] TimePassed>DefiniteCaton||(TimePassed>PotentialCaton&&TimePassed>DisplayIntervalPer80*2)

[0057] Among them, TimePassed is the playback time of the current real-time video from the previous frame of video, DisplayIntervalPer80 is the playback time of the current real-time video from the previous frame of video to the set picture playback time interval information. Here, the 80th percentile of the picture playback time interval is selected. DefiniteCaton is the set first jamming threshold, which is set to 500ms, indicating a more serious jamming. PotentialCaton is the set second jamming threshold, which is set to 200ms, indicating a slight jamming situation.

[0058] Based on the triggering condition for the screen replay, when it is determined that the current real-time video meets the above judgment condition, the screen replay of the current real-time video is triggered.

[0059] S120: When it is determined to replay the current real-time video, calculate the replay duration based on the network cache duration of the current real-time video.

[0060] Furthermore, when determining to trigger the replay of the current real-time video, the replay duration is first accurately determined. It is understandable that when it is determined that the current screen needs to be replayed, a reasonable replay duration needs to be designed. Since the replay duration is too short and too few screens are inserted, the replay effect will be affected; if the replay duration is too long, the user will noticeably perceive it, which will cause a bad sensory experience. Therefore, this application can better enhance the user's sensory experience through reasonable replay duration decisions.

[0061] The corresponding replay duration is determined by the network cache duration. For example, the corresponding play duration value is set according to different network cache durations, and then according to the mapping relationship between the two, the corresponding value is selected as the replay duration based on the network cache duration of the current real-time video.

[0062] Optionally, refer to Figure 4 , the replay duration is calculated based on the network cache duration of the current real-time video, including:

[0063] S1201, determining the time difference between the network cache time of the current real-time video and the playback time of the previous frame of the current real-time video, and calculating the target duration according to the time difference and the set basic duration;

[0064] S1202: Select a minimum value from the set maximum duration and target duration as the replay duration of the current real-time video.

[0065] The calculation formula for the replay duration is expressed as:

[0066] Replay duration = min(DurationMax, DurationBase+max(0, Jitter-FrameReadyPassed))

[0067] Among them, DurationBase represents the basic duration of the screen replay setting, which is set to 500ms in this application, DurationMax is the maximum duration of the screen replay setting, which is set to 1000ms, Jitter is the network cache duration estimated by the current system, and FrameReadyPassed is the playback time of the current real-time video from the previous frame of video.

[0068] Based on the determined playback time, the screen can be replayed according to the playback time. It is understandable that the playback time is set according to the network cache time, so that after the playback time, new data can be received for real-time video playback, achieving accurate screen playback connection, avoiding frame skipping, and improving the smoothness of screen playback.

[0069] S130, determining the number of replay frames according to the replay duration, extracting video pictures corresponding to the number of replay frames from the pre-built replay queue in descending order according to the playback position of the current real-time video as replay pictures, and the replay queue is used to store the video pictures that have been played in sequence according to the sequence number.

[0070] Furthermore, based on the determined replay duration, the present application replays the replay screen by selecting an appropriate played video screen as the replay screen.

[0071] The number of replay frames is determined according to the replay duration, including:

[0072] The average time interval of video picture playback within the set time period is determined, and the number of replay frames of the current real-time video is calculated based on the replay duration and the average time interval of video picture playback.

[0073] Based on the replay duration, the corresponding replay frame number can be calculated by dividing it by the average time interval of the video frames that have been played in the past set period. Then, from the playback position of the current real-time video, the played video frames of the corresponding replay frame number are extracted backward in the data cache as the replay frames.

[0074] Specifically, after deciding the replay duration, it is necessary to convert the replay duration into the corresponding replay frame number, and select the appropriate historical video screen according to the replay frame number, and replay it at a reasonable frame rate. At this time, it is necessary to pay attention to the coordination of the previous and next frame rates and the coherence of the replay screen and the subsequent screens. The calculation of the replay frame number is expressed as:

[0075] ReplayNum=ReplayDuration / DisplayIntervalAvg

[0076] ReplayNum indicates the number of replay frames, ReplayDuration indicates the replay duration, and DisplayIntervalAvg indicates the average time interval between video screen playback.

[0077] S140, adding the replay picture frame by frame to the front end of the play queue of the current real-time video according to the sequence number order, and playing the replay picture based on the play queue.

[0078] Based on the replay frame number determined above, a corresponding number of played video frames can be extracted from the data cache according to the replay frame number for screen replay. In the data cache, the played video frames corresponding to the replay frame number are extracted from the position of the latest cached video frame from the back to the front. In this process, if the extracted frame has been replayed, the extraction of the frame is not performed and the frame extraction stops.

[0079] Exemplarily, when extracting played video frames, if the data cache has cached the played video frames numbered 5, 6, 7, 8, and 95, and the replay frame number ReplayNum=3, then the three frames numbered 7, 8, and 9 are taken out as replay frames.

[0080] Optionally, playing the replay screen based on the play queue includes:

[0081] The replay images are extracted frame by frame from the play queue for playback according to the average time interval of the video images.

[0082] After the replay screen is taken out, it is sent to the play queue in ascending order according to the sequence number. The play interval of the replay screen is set to the average value of the video screen play time interval DisplayIntervalAvg, so that the replay frame rate can be close to the real frame rate, and the subsequent frames can be well connected with the replay screen. For example, assuming that the sequence number of the blocked late frame is 10, and the replay data is 7, 8, 9, then the late frame can be sent to the play queue and just connected with the replay screen, so as to complete the real-time video screen replay process.

[0083] Reference Figure 5 This application caches the historical playback data of real-time video, triggers the replay strategy when it is determined that the data is empty and about to freeze, and then decides the replay duration based on the current predicted jitter value (network cache duration) and the empty duration, and finally selects the appropriate replay screen and replay frame rate, and puts the replay screen in the front of the replay screen for playback. In this way, the freeze problem in real-time video under weak network conditions is reduced, and the real video screen data that has been played is used to fill the gap in the empty playback at the right time, to deal with some freeze problems that cannot be covered by the network cache, reduce freezes from the visual sense, and improve the user viewing experience.

[0084] In the above, by detecting the data empty information and video freeze information of the current real-time video, the screen replay judgment of the current real-time video is performed based on the data empty information and video freeze information; when it is determined to replay the screen of the current real-time video, the replay duration is calculated based on the network cache duration of the current real-time video; the number of replay frames is determined according to the replay duration, and the video screens of the corresponding number of replay frames are extracted from the pre-built replay queue in descending order according to the playback position of the current real-time video as the replay screen, and the replay queue is used to store the video screens that have been played according to the sequence number sequence; the replay screens are added to the front end of the playback queue of the current real-time video frame by frame according to the sequence number sequence, and the replay screens are played based on the playback queue. The above technical means are adopted, by accurately calculating the replay duration in combination with the network cache duration when it is determined to replay the screen, and selecting the appropriate replay screen for replay according to the replay duration, so that the screen can be adaptively replayed when the network cache is empty and no new data arrives for a long time, and the replay screen can be accurately connected with the real-time video screen, reducing the playback freeze, improving the smoothness and stability of video playback, and thus improving the user's viewing experience.

[0085] Based on the above embodiments, Figure 6 This is a structural diagram of a real-time video screen replay system provided by this application. Figure 6 The real-time video screen replay system provided in this embodiment specifically includes: a judgment module 21, a duration calculation module 22, a screen extraction module 23 and a replay module 24.

[0086] The judgment module 21 is configured to detect data blanking information and video freeze information of the current real-time video, and perform a picture replay judgment of the current real-time video based on the data blanking information and the video freeze information;

[0087] The duration calculation module 22 is configured to calculate the replay duration based on the network cache duration of the current real-time video when it is determined to replay the current real-time video;

[0088] The picture extraction module 23 is configured to determine the number of replay frames according to the replay duration, and extract the video pictures corresponding to the number of replay frames from the pre-built replay queue in descending order according to the playback position of the current real-time video as the replay pictures. The replay queue is used to store the video pictures that have been played in sequence according to the sequence number;

[0089] The replay module 24 is configured to add the replay pictures frame by frame to the front end of the play queue of the current real-time video according to the sequence number order, and play the replay pictures based on the play queue.

[0090] Specifically, the replay queue is a circular buffer queue of fixed length, which is used for cyclically covering and storing video frames that have been played.

[0091] Specifically, based on the data blanking information and the video freeze information, the screen replay judgment of the current real-time video is performed, including:

[0092] When it is determined based on the data empty information that the decoding queue and the playback queue of the current real-time video are both empty, and when it is determined based on the video freeze information that the playback time of the current real-time video from the previous frame of video reaches the set threshold information, it is determined to replay the current real-time video.

[0093] The information of determining whether the playback time of the current real-time video from the previous frame of the video reaches the set threshold based on the video freeze information includes:

[0094] When the playback time of the current real-time video from the last frame of the video reaches the set first freeze threshold, determining that the playback time of the current real-time video from the last frame of the video reaches the set threshold; or

[0095] When the playback time of the current real-time video from the previous frame of video reaches the set second freeze threshold, and the playback time of the current real-time video from the previous frame of video reaches the set picture playback time interval information, it is determined that the playback time of the current real-time video from the previous frame of video reaches the set threshold information.

[0096] Specifically, the replay duration is calculated based on the network cache duration of the current real-time video, including:

[0097] Determine the time difference between the network cache time of the current real-time video and the playback time of the previous frame of the current real-time video, and calculate the target duration based on the time difference and the set basic duration;

[0098] The minimum value is selected from the set maximum duration and target duration as the replay duration of the current real-time video.

[0099] Specifically, the number of replay frames is determined according to the replay duration, including:

[0100] The average time interval of video picture playback within the set time period is determined, and the number of replay frames of the current real-time video is calculated based on the replay duration and the average time interval of video picture playback.

[0101] In addition, playback screens are played based on the playback queue, including:

[0102] The replay images are extracted frame by frame from the play queue for playback according to the average time interval of the video images.

[0103] In the above, by detecting the data empty information and video freeze information of the current real-time video, the screen replay judgment of the current real-time video is performed based on the data empty information and video freeze information; when it is determined to replay the screen of the current real-time video, the replay duration is calculated based on the network cache duration of the current real-time video; the number of replay frames is determined according to the replay duration, and the video screens of the corresponding number of replay frames are extracted from the pre-built replay queue in descending order according to the playback position of the current real-time video as the replay screen, and the replay queue is used to store the video screens that have been played according to the sequence number sequence; the replay screens are added to the front end of the playback queue of the current real-time video frame by frame according to the sequence number sequence, and the replay screens are played based on the playback queue. The above technical means are adopted, by accurately calculating the replay duration in combination with the network cache duration when it is determined to replay the screen, and selecting the appropriate replay screen for replay according to the replay duration, so that the screen can be adaptively replayed when the network cache is empty and no new data arrives for a long time, and the replay screen can be accurately connected with the real-time video screen, reducing the playback freeze, improving the smoothness and stability of video playback, and thus improving the user's viewing experience.

[0104] The real-time video screen replay system provided in the embodiment of the present application can be configured to execute the real-time video screen replay method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0105] Based on the above practical example, the present application embodiment also provides a real-time video screen replay device, referring to Figure 7 , the real-time video screen replay device includes: a processor 31, a memory 32, a communication module 33, an input device 34 and an output device 35. The memory, as a computer-readable storage medium, can be configured to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the real-time video screen replay method described in any embodiment of the present application (for example, the judgment module, the duration calculation module in the real-time video screen replay system. Screen extraction module and replay module). The communication module is configured to perform data transmission. The processor executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory, that is, realizing the above-mentioned real-time video screen replay method. The input device can be configured to receive input digital or character information, and generate key signal input related to the user settings and function control of the device. The output device may include a display device such as a display screen. The above-mentioned real-time video screen replay device can be configured to execute the real-time video screen replay method provided in the above-mentioned embodiment, and has corresponding functions and beneficial effects.

[0106] On the basis of the above embodiments, the embodiments of the present application further provide a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, wherein the computer-executable instructions are configured to execute a method for replaying a real-time video when executed by a computer processor, and the storage medium may be any of various types of memory devices or storage devices. Of course, the computer-executable instructions of the computer-readable storage medium provided in the embodiments of the present application are not limited to the method for replaying a real-time video as described above, and may also execute related operations in the method for replaying a real-time video provided in any embodiment of the present application.

[0107] On the basis of the above embodiments, the embodiments of the present application also provide a computer program product. The technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer program product is stored in a storage medium, and includes a number of instructions for enabling a computer device, a mobile terminal or a processor therein to execute all or part of the steps of the real-time video screen replay method described in each embodiment of the present application.

Claims

1. A method for replaying a real-time video. It is characterized in that include: Detecting data blanking information and video freeze information of the current real-time video, and making a picture replay judgment of the current real-time video based on the data blanking information and video freeze information; When it is determined to replay the current real-time video, the replay duration is calculated based on the network cache duration of the current real-time video; Determine the number of replay frames according to the replay duration, extract the video pictures corresponding to the number of replay frames from the pre-constructed replay queue in descending order according to the playback position of the current real-time video as the replay pictures, and the replay queue is used to store the video pictures that have been played in sequence according to the sequence number; The replay picture is added frame by frame to the front end of the play queue of the current real-time video according to the sequence number order, and the replay picture is played based on the play queue.

2. The method for replaying a real-time video according to claim 1, It is characterized in that The judging of replaying the current real-time video based on the data blanking information and the video freeze information includes: When it is determined based on the data empty information that the decoding queue and the playback queue of the current real-time video are both empty, and when it is determined based on the video freeze information that the playback time of the current real-time video from the previous frame of video reaches the set threshold information, it is determined to replay the current real-time video.

3. The method for replaying a real-time video according to claim 2, It is characterized in that The determining, based on the video freeze information, that the playback time of the current real-time video from the previous frame of video reaches the set threshold information includes: When the playback time of the current real-time video from the last frame of the video reaches the set first freeze threshold, determining that the playback time of the current real-time video from the last frame of the video reaches the set threshold; or When the playback time of the current real-time video from the previous frame of video reaches the set second freeze threshold, and the playback time of the current real-time video from the previous frame of video reaches the set picture playback time interval information, it is determined that the playback time of the current real-time video from the previous frame of video reaches the set threshold information.

4. The method for replaying a real-time video according to any one of claims 1 to 3, It is characterized in that The calculating of the replay duration based on the network cache duration of the current real-time video includes: Determine the time difference between the network cache time of the current real-time video and the playback time of the previous frame of the current real-time video, and calculate the target duration according to the time difference and the set basic duration; A minimum value is selected from the set maximum duration and the target duration as the replay duration of the current real-time video.

5. The method for replaying a real-time video according to any one of claims 1 to 3, It is characterized in that Determining the number of replay frames according to the replay duration includes: An average of the video picture playback time intervals within a set time period is determined, and the number of replay frames of the current real-time video is calculated based on the replay duration and the average of the video picture playback time intervals.

6. The method for replaying a real-time video according to claim 5, It is characterized in that The playing the replay picture based on the play queue includes: The replay pictures are extracted frame by frame from the play queue for play according to the average time interval of the video pictures.

7. The method for replaying a real-time video according to any one of claims 1 to 3, It is characterized in that The replay queue is a circular cache queue of fixed length, which is used for cyclically covering and storing video images that have been played.

8. A real-time video screen replay system, It is characterized in that include: A judgment module is configured to detect data blanking information and video freeze information of the current real-time video, and perform a picture replay judgment of the current real-time video based on the data blanking information and the video freeze information; A duration calculation module, configured to calculate the replay duration based on the network cache duration of the current real-time video when it is determined to replay the current real-time video; A picture extraction module is configured to determine the number of replay frames according to the replay duration, and extract video pictures corresponding to the number of replay frames from a pre-built replay queue in descending order according to the playback position of the current real-time video as replay pictures, wherein the replay queue is used to store the video pictures that have been played in sequence according to the sequence number; The replay module is configured to add the replay picture frame by frame to the front end of the play queue of the current real-time video according to the sequence number order, and play the replay picture based on the play queue.

9. A real-time video playback device. It is characterized in that include: memory and one or more processors; The memory is configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the real-time video screen replay method as described in any one of claims 1-7.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer-executable instructions, which, when executed by a computer processor, are configured to execute the real-time video screen replay method according to any one of claims 1 to 7.

11. A computer program product, It is characterized in that The computer program product includes instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes the real-time video screen replay method as described in any one of claims 1-7.