Video push method and system in live broadcast scenario

Abstract

The present application discloses a video push method in a live broadcast scenario, the method comprising: obtaining the i-th video frame provided by multiple input threads through an output thread; updating the timestamp of the i-th video frame through the output thread to obtain the updated i-th video frame: when i=1, the timestamp of the first video frame is updated to the reference time point; when i≥2, the timestamp of the i-th video frame is updated according to the reference time point, the preset frame rate and the value of i; and according to each updated timestamp, pushing the corresponding updated video frame through the output thread. The embodiment of the present application can avoid: when each input thread in the live broadcast scenario provides video frames with different frame rates, the output video frame causes the player to parse the timestamp error.

Description

Technical Field

[0001] The present application relates to the field of audio and video technology, and in particular to a video push method, system, computer device, and computer-readable storage medium in a live broadcast scenario. Background Art

[0002] Live streaming is one of the hottest projects on the Internet. Currently, people have developed various live streaming apps based on operating systems such as Android and IOS. Broadcasters can use these live streaming apps to perform live streaming operations. In live streaming scenarios, live streaming apps face various influences, such as: (1) receiving various inputs, such as screen recording input collected by the system in real time, custom input, and other types of inputs; (2) jamming and blocking; (3) dynamic situations where the sampling rate of external audio and video inputs fluctuates. The above influences may cause errors in the timestamp parsing of the viewer's player. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a video push method, system, computer device and computer-readable storage medium in a live broadcast scenario, which are used to solve the following problem: when each input thread in the live broadcast scenario provides video frames with different frame rates, the output video frames will cause the player to parse the timestamp error.

[0004] One aspect of an embodiment of the present application provides a video push method in a live broadcast scenario, the method comprising: obtaining the i-th video frame provided by multiple input threads through an output thread; updating the timestamp of the i-th video frame through the output thread to obtain an updated i-th video frame: when i=1, the timestamp of the first video frame is updated to a reference time point; when i≥2, the timestamp of the i-th video frame is updated according to the reference time point, a preset frame rate and the value of i; and according to each updated timestamp, pushing the corresponding updated video frame through the output thread.

[0005] Optionally, when i≥2, obtaining the i-th video frame provided by multiple input threads through the output thread, including: controlling the output thread to perform the i-th frame acquisition task through a timer: when the timer reaches a preset i-th time point, controlling the output thread to obtain one or zero video frames provided by the multiple input threads for the i-th time.

[0006] Optionally, it also includes creating the timer: when the output thread obtains the first video frame, a timer is created, the timer includes multiple time points with the same time interval, and the first time point among the multiple time points is the time zero point of the timer.

[0007] Optionally, the method further includes: temporarily storing the updated i-th video frame until the output thread obtains the next video frame.

[0008] Optionally, it also includes: generating a reference count associated with the updated ith video frame; the reference count is used to indicate the temporary storage state of the updated ith video frame, and the temporary storage state includes a non-released state and a released state.

[0009] Optionally, it also includes: executing the i+1th frame acquisition task operation: controlling the output thread to acquire one or zero video frames provided by the multiple input threads for the i+1th time; if the output thread acquires a video frame based on the i+1th frame acquisition task, then the video frame obtained in the i+1th frame acquisition task is used as the i+1th video frame, and the reference count associated with the updated i-th video frame is released; if the output thread acquires zero video frames based on the i+1th frame acquisition task, then the updated i-th video frame is regarded as the i+1th video frame, and the updated i-th video frame is pushed again.

[0010] Optionally, it also includes: controlling the execution strategy between the multiple input threads and the output thread through a serial queue: when the multiple input threads execute external input tasks, suspending the execution of the frame acquisition task of the output thread; and when the output thread executes the frame acquisition task, suspending the execution of the external input task of the input thread.

[0011] Optionally, updating the timestamp of the i-th video frame through the output thread to obtain the updated i-th video frame includes: redrawing the image of the i-th video frame; if the image redrawing is successful, updating the timestamp of the i-th video frame after the image redrawing to obtain the updated i-th video frame; and if the image redrawing fails, maintaining the original data of the i-th video frame.

[0012] One aspect of an embodiment of the present application provides a video push system in a live broadcast scenario, including: an acquisition module, used to acquire the i-th video frame provided by multiple input threads through an output thread; an update module, used to update the timestamp of the i-th video frame through the output thread to obtain an updated i-th video frame: when i=1, the timestamp of the first video frame is updated to a reference time point; when i≥2, the timestamp of the i-th video frame is updated according to the reference time point, a preset frame rate and the value of i; and a push module, used to push the corresponding updated video frame through the output thread according to each updated timestamp.

[0013] One aspect of an embodiment of the present application further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the video push method in the above-mentioned live broadcast scenario are implemented.

[0014] One aspect of an embodiment of the present application provides a computer-readable storage medium, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the video push method in the above-mentioned live broadcast scenario are implemented.

[0015] The video push method, system, device and computer-readable storage medium in the live broadcast scenario provided by the embodiments of the present application reset the timestamp of each video frame according to the output frame rate and the reference time point before pushing each video frame, and then push each video frame with the updated timestamp to avoid errors in the player parsing the timestamp. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A diagram schematically shows an application environment of a video push method in a live broadcast scenario according to an embodiment of the present application;

[0017] Figure 2 The flowchart of the video push method in the live broadcast scenario according to the first embodiment of the present application is schematically shown;

[0018] Figure 3 Another flow chart of the video push method in the live broadcast scenario according to the second embodiment of the present application is schematically shown;

[0019] Figure 4 Another flow chart of the video push method in the live broadcast scenario according to the second embodiment of the present application is schematically shown;

[0020] Figure 5 for Figure 3 The sub-step diagram of step S306;

[0021] Figure 6 The diagram schematically shows the newly added steps of the video push method in the live broadcast scenario according to the second embodiment of the present application;

[0022] Figure 7 Another newly added step diagram of the video push method in the live broadcast scenario according to the second embodiment of the present application is schematically shown;

[0023] Figure 8 Another newly added step diagram of the video push method in the live broadcast scenario according to the second embodiment of the present application is schematically shown;

[0024] Fig. 9A block diagram of a video push system in a live broadcast scenario according to Embodiment 3 of the present application is schematically shown; and

[0025] Fig.10 The hardware architecture diagram of a computer device suitable for implementing a video push method in a live broadcast scenario according to the fourth embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0027] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0028] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0029] Figure 1 The schematic diagram of the environment application of the video push method in the live broadcast scenario according to the embodiment of the present application is schematically shown. In the live broadcast scenario, the anchor terminal 2 can push the live broadcast data (such as audio data, picture data) to the audience terminal 4 in real time.

[0030] The anchor terminal 2 is used to generate live broadcast data in real time and perform streaming operations on the live broadcast data. The live broadcast data may include audio data or video data. The live broadcast terminal 2 may be a smart phone, tablet computer, etc. based on the IOS system. In other embodiments, the live broadcast terminal 2 may be a live broadcast device based on systems such as Android.

[0031] The audience terminal 4 may be configured to receive the live broadcast data of the anchor terminal 2 in real time. The audience terminal 4 may be any type of computing device, such as a smart phone, a tablet device, a laptop computer, a set-top box, a smart TV, etc. The audience terminal 4 may have a built-in browser or a special program, and obtain the live broadcast data through the browser or the special program to output content to the user. The content may include video, audio, comments, text data and / or the like.

[0032] The host terminal 2 has a built-in live broadcast APP (such as BilibiliLink) for collecting audio and video data, processing the audio and video data (such as encoding, etc.) and pushing it to the audience terminal 4. In the live broadcast scenario, the live broadcast APP faces various influences, such as: (1) receiving various inputs, such as: screen recording input collected by the system in real time, custom input, and other types of inputs; (2) jamming and blocking; (3) dynamic situations where the sampling rate of external audio and video inputs is large or small. The above influences may cause the player of the audience terminal 4 to parse the timestamp incorrectly and the playback to jam.

[0033] The present application aims to provide a video frame processing solution for the anchor terminal 2 in a live broadcast scenario, that is, to achieve output of a fixed frame number under the condition of dynamic frame number input.

[0034] Embodiment 1

[0035] Figure 2 The flowchart of the video push method in the live broadcast scenario according to the first embodiment of the present application is schematically shown.

[0036] like Figure 2 As shown, the video push method in the live broadcast scenario may include steps S200 to S204, wherein:

[0037] Step S200: obtaining, through an output thread, an i-th video frame provided by multiple input threads.

[0038] In some embodiments, the video frames are all picture frames. In other embodiments, the video frames are all audio frames.

[0039] The input thread is used as a data input side to supply video frames for the output thread to obtain. The multiple input threads can correspond to different input interfaces to obtain video frames from different sources, such as:

[0040] The first input thread is used to input video frames obtained by real-time screen recording;

[0041] The second input thread is used to input video frames provided by a third party;

[0042] The third input thread is used to input video frames provided by other interfaces; ….

[0043] The multiple input threads may cache the input video frames in a preset cache area. Each time the multiple input threads input a new video frame, the old video frames may need to be discarded in the preset cache area to ensure that the preset cache area always stores the latest video frame obtained by the multiple input threads.

[0044] Step S202: updating the timestamp of the i-th video frame through the output thread.

[0045] When i=1, the timestamp of the first video frame is updated as the reference time point.

[0046] When i≥2, the timestamp of the i-th video frame is updated according to the reference time point, the preset frame rate and the value of i.

[0047] Step S204: Push corresponding updated video frames through the output thread according to each updated timestamp.

[0048] As an example, the input frame rate of the first input thread is 40 frames per second, and the input frame rate of the second input thread is 100 frames per second. That is, the first input thread obtains a video frame every 0.025 seconds, and the second input thread obtains a video frame every 0.01 seconds. Each video frame obtained by the first input thread will have its own native timestamp, and each video frame obtained by the second input thread will also have its own native timestamp. The above example presents such a situation: on one side of the data input, the input rate of the video frame is dynamically changing. The inventors have found that if some video frames are pushed to the audience end 4, and the input rates of these video frames are different and each video frame uses a native timestamp, it will cause some players in the audience end 4 to parse the timestamp incorrectly, which will lead to confusion of the video playback picture, playback jams, etc.

[0049] To solve the above problem, this embodiment provides a fixed frame number output solution, as an example:

[0050] Assume that the output frame rate of the output thread is 50 frames per second. That is, the output thread outputs frames at a fixed rate of 50 frames per second.

[0051] (1) The output thread obtains the first video frame (i=1) provided by the first input thread and the second input thread.

[0052] (2) The output thread modifies the timestamp of the first video frame to a reference time point (eg, 0.00 seconds) to obtain the first video frame with a timestamp of 0.00 seconds.

[0053] (3) The output thread obtains the second video frame (i=2) provided by the first input thread and the second input thread.

[0054] (4) The output thread modifies the timestamp of the second video frame to 0.02 seconds to obtain a second video frame with a timestamp of 0.02 seconds.

[0055] The output frame rate of the output thread is 50 frames per second, so the time interval between adjacent video frames is 0.02 seconds.

[0056] The timestamp of the second video frame = reference time point + [(i-1)*time interval between adjacent video frames] = 0.00 + 1*0.02 = 0.02 seconds.

[0057] (5) The output thread obtains the third video frame (i=3) provided by the first input thread and the second input thread.

[0058] (6) The output thread modifies the timestamp of the third video frame to 0.04 to obtain a third video frame with a timestamp of 0.04.

[0059] The timestamp of the third video frame=reference time point+[(i-1)*time interval between adjacent video frames]=0.00+2*0.02=0.04 seconds.

[0060] And so on, the timestamp of each video frame obtained is updated in the output thread.

[0061] In this embodiment, the output thread outputs the updated video frames at an output frame rate of 50 frames per second. That is, the output thread pushes in sequence: the first video frame with a timestamp of 0.00 seconds, the second video frame with a timestamp of 0.02 seconds, etc. By pushing each video frame with an updated timestamp, the error of parsing the timestamp by the player is effectively avoided.

[0062] Embodiment 2

[0063] Since the input rate of the video frames on the data input side is dynamically changing, if the output frame rate of the output thread is to be effectively controlled to achieve stable streaming, it is necessary to ensure that the output thread takes the video frames of the multiple input threads according to a predetermined rule.

[0064] Figure 3 The flowchart of the video push method in the live broadcast scenario according to the second embodiment of the present application is schematically shown.

[0065] like Figure 3 As shown, the video push method in the live broadcast scenario may include steps S300 to S308, wherein:

[0066] Step S300: acquiring the first video frame through an output thread.

[0067] Step S302: updating the timestamp of the first video frame to the reference time point through the output thread to obtain an updated first video frame.

[0068] Step S304, controlling the output thread to execute the i-th frame acquisition task through a timer: when the timer reaches the preset i-th time point, the output thread is controlled to acquire one or zero video frames provided by the multiple input threads for the i-th time, where i is an integer ≥2.

[0069] In an exemplary embodiment, the video push method in the live broadcast scenario may include the step of creating the timer. Figure 4 As shown, the step of creating the timer may include: step S400, when the output thread obtains the first video frame, a timer is created, the timer includes multiple time points with the same time interval, and the first time point among the multiple time points is the time zero of the timer, that is, the timer is initialized at a time interval of 1 / FPS (Frames Per Second). Based on the above creation timing, a timer for realizing stable streaming can be dynamically created in any live streaming.

[0070] Step S306: updating the timestamp of the ith video frame through the output thread to obtain an updated ith video frame.

[0071] For example: according to the reference time point, the preset frame rate and the value of i, the timestamp of the i-th video frame is updated.

[0072] It should be noted that, each time the output thread obtains a video frame, it can also perform other processing on the video frame, such as:

[0073] In an exemplary embodiment, Figure 5 As shown, the step S306 may further include steps S500 to S504. Step S500, redrawing the image of the i-th video frame; step S502, if the image redrawing is successful, updating the timestamp of the i-th video frame after the image redrawing to obtain the updated i-th video frame; step S504, if the image redrawing fails, maintaining the original data of the i-th video frame.

[0074] For example: according to the rotation parameter (rotateparameter) of the i-th video frame and the current screen direction of the live broadcast terminal 2, the i-th video frame is redrawn to ensure that the display image of the i-th video frame is consistent with the screen direction. In addition, in order to ensure the effective operation of the timer, the working status of the timer needs to be checked after each video frame is redrawn. That is, after redrawing the i-th video frame, the working status of the timer needs to be checked. If the timer is not in a working state, an initialization operation needs to be performed.

[0075] If the image data obtained after the i-th video frame is redrawn cannot pass the verification operation, it means that the image redrawing fails. Therefore, in order to ensure the correctness of the data, the original data of the i-th video frame can be directly used.

[0076] Step S308: Push corresponding updated video frames through the output thread according to each updated timestamp.

[0077] In order to make this embodiment clearer, an example is provided below.

[0078] In this example, it is assumed that the output frame rate of the output thread is 50 frames per second.

[0079] (1) The output thread obtains the first video frame provided by the first input thread and the second input thread.

[0080] (2) When the output thread obtains the first video frame, a timer is created or initialized. The timer can use the time point when the output thread obtains the first video frame as time zero, and use time zero as the first time point to set a time point every 0.02 seconds. For example, 0.02 seconds is set as the second time point, 0.04 seconds is set as the third time point, and so on.

[0081] (3) When the timer reaches 0.02S, the second frame acquisition task is started to control the output thread to obtain the second video frame from the multiple input threads.

[0082] (4) When the timer reaches 0.04S, the third frame acquisition task is started to control the output thread to obtain the third video frame from the multiple input threads.

[0083] By analogy, the output thread takes video frames from the preset cache areas of the multiple input threads every 0.02 seconds. It can be seen that by taking video frames at a fixed time, when the number of video frames provided by the multiple input threads is excessive, it can be ensured that the output thread takes a fixed number of video frames per second.

[0084] The examples described above are ideal situations. The inventors have found that there are still some problems to be further solved:

[0085] First, how to avoid thread safety of the multiple input threads and the output thread.

[0086] In an exemplary embodiment, if Figure 6 As shown, the video push method in the live broadcast scenario may include: step S600, controlling the execution strategy between the multiple input threads and the output thread through a serial queue: when the multiple input threads execute external input tasks, the frame acquisition task of the output thread is suspended; and when the output thread executes the frame acquisition task, the external input task of the input thread is suspended. In this embodiment, the control of the output thread and the multiple input threads by a serial queue ensures the sequentiality of the data processing flow under multi-threading and avoids crashes caused by thread safety hazards, such as: when the output thread is taking the video frame A1 from the preset cache area, a certain input thread is writing the latest video frame A2 into the preset cache area to replace the video frame A1.

[0087] Second, when the number of video frames provided by the multiple input threads is insufficient, how to ensure video smoothness.

[0088] In an exemplary embodiment, if Figure 7 As shown, the video push method in the live broadcast scenario may include: step S700, temporarily storing the updated i-th video frame until the output thread obtains the next video frame.

[0089] Furthermore, to prevent memory leaks and video frames from being erroneously recycled in advance, the video push method in the live broadcast scenario may include: Step 702, generating a reference count associated with the updated i-th video frame; the reference count is used to indicate the temporary storage state of the updated i-th video frame, and the temporary storage state includes a non-released state and a released state. When the output thread obtains the i-th video frame, a corresponding reference count is generated. This reference count is used to inform the operating system not to recycle the i-th video frame.

[0090] In order to make this embodiment more clear and understandable, another example is provided below.

[0091] Let's continue to take the output frame rate of the output thread as 50 frames per second as an example. If the first input thread is 20 frames per second, and the second input thread and other input threads have no video frame input, the following situation will occur: the supply speed of the video frames on the data input side cannot keep up with the video frame acquisition speed of the output thread. Therefore, the output thread cannot push 50 video frames per second to the viewer terminal 4, resulting in an unsmooth playback screen on the viewer terminal 4. This problem can be solved as follows:

[0092] (1) The output thread obtains the first video frame provided by the first input thread.

[0093] (2) When the output thread obtains the first video frame, a timer is created or initialized. The timer can use the time point when the output thread obtains the first video frame as time zero, and use time zero as the first time point to set a time point every 0.02 seconds. For example, 0.02 seconds is set as the second time point, 0.04 seconds is set as the third time point, and so on.

[0094] (3) When the timer reaches 0.02S, the second frame acquisition task is started. Since the first input thread does not provide any new video frames between the second frame acquisition task and the first frame acquisition task in (1) (i.e., there is no video frame in the preset cache area at this time), the output thread obtains zero video frames.

[0095] (4) When the timer reaches 0.04S, the third frame acquisition task is started, and the output thread obtains zero video frames.

[0096] (5) When the timer reaches 0.06S, the fourth frame acquisition task is started, and the output thread obtains a video frame.

[0097] By analogy, the output thread fetches one or zero video frames from the preset cache areas of the multiple input threads every 0.02 seconds.

[0098] In this embodiment, after the live broadcast terminal 2 pushes an updated video frame B1 to the viewer terminal 4, the updated video frame B1 is not released immediately. When the output thread does not get the next video frame, the updated video frame B1 is pushed to the viewer terminal 4 again to prevent the viewer terminal 4 from having an unsmooth picture due to frame loss.

[0099] In this embodiment, the output thread outputs the updated video frame at a frame rate of 50 frames per second. That is, the output thread pushes in sequence: the first video frame with a timestamp of 0.00 seconds, the first video frame with a timestamp of 0.00 seconds (regarded as the second video frame), the first video frame with a timestamp of 0.00 seconds (regarded as the third video frame), the fourth video frame with a timestamp of 0.06 seconds, ... In this way, it can be ensured that the output thread pushes 50 video frames to the viewer terminal 4 per second, ensuring that the playback picture of the viewer terminal 4 is smooth. The specific operation steps are as follows:

[0100] like Figure 8 As shown, the video push method in the live broadcast scenario may include: step S800, executing the i+1th frame acquisition task operation: controlling the output thread to acquire one or zero video frames provided by the multiple input threads for the i+1th time; step S802, if the output thread acquires a video frame based on the i+1th frame acquisition task, then the video frame obtained in the i+1th frame acquisition task is used as the i+1th video frame, and the reference count associated with the updated i-th video frame is released; step S804, if the output thread acquires zero video frames based on the i+1th frame acquisition task, then the updated i-th video frame is regarded as the i+1th video frame, and the updated i-th video frame is pushed again. After the reference count associated with the updated i-th video frame is released, the operating system can implement frame recycling.

[0101] Embodiment 3

[0102] Fig. 9 The block diagram of the video push system in the live broadcast scenario according to the third embodiment of the present application is schematically shown. The video push system in the live broadcast scenario can be divided into one or more program modules, and one or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment.

[0103] like Fig. 9 As shown, the video push system 900 in the live broadcast scenario may include:

[0104] An acquisition module 910 is used to acquire, through an output thread, an i-th video frame provided by multiple input threads;

[0105] An updating module 920 is used to update the timestamp of the i-th video frame through the output thread to obtain an updated i-th video frame: when i=1, the timestamp of the first video frame is updated to the reference time point; when i≥2, the timestamp of the i-th video frame is updated according to the reference time point, the preset frame rate and the value of i; and

[0106] The push module 930 is used to push the corresponding updated video frames through the output thread according to each updated timestamp.

[0107] In an exemplary embodiment, the acquisition module 910 is also used to: control the output thread to perform the i-th frame acquisition task through a timer: when the timer reaches a preset i-th time point, control the output thread to acquire one or zero video frames provided by the multiple input threads for the i-th time.

[0108] In an exemplary embodiment, the video push system in the live broadcast scenario also includes a creation module for: also including creating the timer: when the output thread obtains the first video frame, a timer is created, the timer includes multiple time points with the same time interval, and the first time point among the multiple time points is the time zero point of the timer.

[0109] In an exemplary embodiment, the video push system in the live broadcast scenario further includes a temporary storage module, which is used to temporarily store the updated i-th video frame until the output thread obtains the next video frame.

[0110] In an exemplary embodiment, the video push system in the live broadcast scenario also includes a counting module, which is used to: generate a reference count associated with the updated i-th video frame; the reference count is used to indicate the temporary storage state of the updated i-th video frame, and the temporary storage state includes a non-released state and a released state.

[0111] In an exemplary embodiment, the acquisition module 910 is also used to: execute the i+1th frame acquisition task operation: control the output thread to acquire one or zero video frames provided by the multiple input threads for the i+1th time; if the output thread acquires a video frame based on the i+1th frame acquisition task, then the video frame obtained in the i+1th frame acquisition task is used as the i+1th video frame, and the updated i-th video frame is released; if the output thread acquires zero video frames based on the i+1th frame acquisition task, then the updated i-th video frame is regarded as the i+1th video frame, and the updated i-th video frame is pushed again.

[0112] In an exemplary embodiment, the video push system in the live broadcast scenario also includes a queue control module, which is used to: control the execution strategy between the multiple input threads and the output thread through a serial queue: when the multiple input threads execute external input tasks, the frame acquisition task of the output thread is suspended; and when the output thread executes the frame acquisition task, the external input task of the input thread is suspended.

[0113] In an exemplary embodiment, the update module 920 is further used to: redraw the image of the i-th video frame; if the image redrawing is successful, update the timestamp of the i-th video frame after the image redrawing to obtain the updated i-th video frame; and if the image redrawing fails, maintain the original data of the i-th video frame.

[0114] Embodiment 4

[0115] Fig.10 The schematic diagram of the hardware architecture of a computer device 10 suitable for implementing a video push method in a live broadcast scenario according to the fourth embodiment of the present application is shown. The computer device 10 can be used as a host terminal 2 or a component of the host terminal 2. In this embodiment, the computer device 10 is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. For example, it can be an electronic device with audio and video processing capabilities such as a smart phone, a tablet computer, or a laptop computer. Fig.10 As shown, the computer device 10 includes at least but is not limited to: a memory 1010, a processor 1020, and a network interface 1030 which can be interconnected through a system bus for communication.

[0116] The memory 1010 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 1010 can be an internal storage module of the computer device 10, such as a hard disk or memory of the computer device 10. In other embodiments, the memory 1010 can also be an external storage device of the computer device 10, such as a plug-in hard disk equipped on the computer device 10, a smart memory card (Smart Media Card, referred to as SMC), a secure digital (Secure Digital, referred to as SD) card, a flash card, etc. Of course, the memory 1010 can also include both the internal storage module of the computer device 10 and its external storage device. In this embodiment, the memory 1010 is generally used to store the operating system and various application software installed on the computer device 10, such as the program code of the video push method in the live broadcast scene, etc. In addition, the memory 1010 can also be used to temporarily store various types of data that have been output or are to be output.

[0117] In some embodiments, the processor 1020 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 1020 is generally used to control the overall operation of the computer device 10, such as performing control and processing related to data interaction or communication with the computer device 10. In this embodiment, the processor 1020 is used to run the program code stored in the memory 1010 or process data.

[0118] The network interface 1030 may include a wireless network interface or a wired network interface, and the network interface 1030 is generally used to establish a communication link between the computer device 10 and other computer devices. For example, the network interface 1030 is used to connect the computer device 10 to an external terminal through a network, and to establish a data transmission channel and a communication link between the computer device 10 and the external terminal. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0119] It should be pointed out that Fig.10 Only a computer device having components 1010 - 1030 is shown, but it should be understood that implementing all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0120] In this embodiment, the video push method in the live broadcast scenario stored in the memory 1010 can also be divided into one or more program modules and executed by one or more processors (processor 1020 in this embodiment) to complete the embodiment of the present application.

[0121] Embodiment 5

[0122] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the video push method in the live broadcast scenario in embodiment one or two are implemented.

[0123] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of a computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, referred to as SMC), a secure digital (Secure Digital, referred to as SD) card, a flash card, etc. Of course, the computer-readable storage medium can also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the computer-readable storage medium is generally used to store the operating system and various application software installed on the computer device, such as the program code of the video push method in the live broadcast scene in the embodiment. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or are to be output.

[0124] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0125] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A video push method in a live broadcast scenario, characterized in that: The method comprises: Obtain the i-th video frame provided by multiple input threads through the output thread; Updating the timestamp of the i-th video frame through the output thread to obtain an updated i-th video frame: when i=1, updating the timestamp of the first video frame to a reference time point; when i≥2, updating the timestamp of the i-th video frame according to the reference time point, a preset frame rate and the value of i; and According to each updated timestamp, pushing the corresponding updated video frame through the output thread; Execute the (i+1)th frame acquisition task operation: control the output thread to acquire one or zero video frames provided by the multiple input threads for the (i+1)th time; If the output thread acquires a video frame based on the i+1th frame acquisition task, the video frame acquired in the i+1th frame acquisition task is used as the i+1th video frame, and a reference count associated with the updated i-th video frame is released; the reference count is used to indicate a temporary storage state of the updated i-th video frame, and the temporary storage state includes a non-released state and a released state; If the output thread acquires zero video frames based on the i+1th frame acquisition task, the updated i-th video frame is regarded as the i+1th video frame, and the updated i-th video frame is pushed again.

2. The video push method in a live broadcast scenario according to claim 1, characterized in that: When i ≥ 2, the i-th video frame provided by multiple input threads is obtained through the output thread, including: The output thread is controlled to execute the i-th frame acquisition task by a timer: when the timer reaches the preset i-th time point, the output thread is controlled to acquire one or zero video frames provided by the multiple input threads for the i-th time.

3. The video push method in a live broadcast scenario according to claim 2, characterized in that: It also includes creating the timer: When the output thread obtains the first video frame, a timer is created, wherein the timer includes multiple time points with the same time interval, and the first time point among the multiple time points is the time zero point of the timer.

4. The video push method in a live broadcast scenario according to any one of claims 1 to 3, characterized in that: Also includes: The updated i-th video frame is temporarily stored until the output thread obtains the next video frame.

5. The video push method in a live broadcast scenario according to claim 4, characterized in that: Also includes: A reference count associated with the updated i-th video frame is generated.

6. The video push method in a live broadcast scenario according to any one of claims 1 to 3, characterized in that: Also includes: Controlling the execution strategy between the multiple input threads and the output thread by a serial queue: When the multiple input threads execute external input tasks, the frame acquisition task of the output thread is suspended; and When the output thread executes the frame acquisition task, the external input task of the input thread is suspended.

7. The video push method in a live broadcast scenario according to any one of claims 1 to 3, characterized in that: Updating the timestamp of the i-th video frame by the output thread to obtain an updated i-th video frame includes: Redrawing the image of the i-th video frame; If the image is redrawn successfully, then the timestamp of the i-th video frame after the image is redrawn is updated to obtain the updated i-th video frame; and If the image redrawing fails, the original data of the i-th video frame is maintained.

8. A video push system in a live broadcast scenario, characterized in that: include: An acquisition module, used for acquiring the i-th video frame provided by multiple input threads through an output thread; An updating module, used for updating the timestamp of the i-th video frame through the output thread to obtain an updated i-th video frame: when i=1, the timestamp of the first video frame is updated to a reference time point; When i≥2, the timestamp of the i-th video frame is updated according to the reference time point, the preset frame rate and the value of i; and A push module, used for pushing corresponding updated video frames through the output thread according to each updated timestamp; The acquisition module is further used for: Execute the (i+1)th frame acquisition task operation: control the output thread to acquire one or zero video frames provided by the multiple input threads for the (i+1)th time; If the output thread acquires a video frame based on the i+1th frame acquisition task, the video frame acquired in the i+1th frame acquisition task is used as the i+1th video frame, and a reference count associated with the updated i-th video frame is released; the reference count is used to indicate a temporary storage state of the updated i-th video frame, and the temporary storage state includes a non-released state and a released state; If the output thread acquires zero video frames based on the i+1th frame acquisition task, the updated i-th video frame is regarded as the i+1th video frame, and the updated i-th video frame is pushed again.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it is used to implement the steps of the video pushing method in the live broadcast scenario described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program can be executed by at least one processor so that the at least one processor performs the steps of the video push method in the live broadcast scenario described in any one of claims 1 to 7.

Images