Live broadcast flow control method and device, equipment, storage medium and product

Abstract

The embodiment of the invention provides a live broadcast flow control method and device, equipment, a storage medium and a product. According to the technical scheme provided by the embodiment of the invention, the to-be-sent video stream is acquired, the traffic control mode of the to-be-sent video stream is determined according to the pre-configured traffic control configuration information, and the traffic control is performed on the to-be-sent video stream in the traffic control mode. Under the condition that the to-be-sent video stream is sent and the video packet of the traffic type corresponding to the to-be-sent video stream is sent, the to-be-sent video packet corresponding to the to-be-sent video stream is stored in a video packet queue, and the average data size of the video packet queue is determined according to the size of the video packet; according to the method, the to-be-sent video packet is extracted from the video packet queue according to the current sending rate, the preset rate threshold and the average data size, and the extracted to-be-sent video packet is sent to the transmission layer for data transmission, so that a set of flow control logic does not need to be set for different transmission protocols respectively, and the maintenance burden of live broadcast flow control is reduced.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular, to a live broadcast traffic control method, device, equipment, storage medium and product. Background Art

[0002] With the development of Internet technology, live broadcasting has become more widely used in the daily lives of netizens, and many live broadcasting scenarios have emerged, such as game live broadcasting, show live broadcasting, live shopping live broadcasting, and live sports event live broadcasting. Different live broadcasting scenarios may also have different traffic transmission methods, such as different transmission protocols to control the transmission of live broadcasting streams in different scenarios, or the same transmission protocol but with different transmission parameters.

[0003] Regardless of the transmission protocol, traffic control issues need to be addressed. For example, in order to reduce the impact of sudden increases in server traffic, the maximum transmission bit rate of data transmission needs to be controlled, otherwise the switch may lose packets, affecting data transmission. Faced with a variety of transmission protocols, multiple transmission protocols are generally integrated into one live broadcast service, and a set of traffic control logic is set for each different transmission protocol. This method makes the development of live broadcast traffic control more complicated and places a great burden on the maintenance of live broadcast traffic control. Summary of the invention

[0004] The embodiments of the present application provide a live broadcast traffic control method, device, equipment, storage medium and product to solve the technical problems in the related art that live broadcast traffic control needs to set up a set of traffic control logics for different transmission protocols, the development of live broadcast traffic control is relatively complex, and the maintenance of live broadcast traffic control causes a great burden. There is no need to set up a set of traffic control logics for different transmission protocols, which simplifies the complexity of live broadcast traffic control development and reduces the maintenance burden of live broadcast traffic control.

[0005] In a first aspect, an embodiment of the present application provides a live broadcast traffic control method, including:

[0006] Acquire a video stream to be sent, and determine a flow control method for the video stream to be sent according to pre-configured flow control configuration information;

[0007] When the flow control mode is to perform flow control on the video stream to be sent, and a video packet of a flow type corresponding to the video stream to be sent has been sent, the video packet to be sent corresponding to the video stream to be sent is saved in a video packet queue;

[0008] determining an average data size of the video packet queue according to video packet sizes of video packets to be sent at a plurality of preset positions in the video packet queue;

[0009] The video packets to be sent are extracted from the video packet queue according to the current sending rate, the preset rate threshold and the average data size, and the extracted video packets to be sent are sent to the transport layer for data transmission.

[0010] In a second aspect, an embodiment of the present application provides a live broadcast traffic control device, including a control judgment module, a video buffer module, a video determination module and a data sending module, wherein:

[0011] The control judgment module is configured to obtain a video stream to be sent, and determine a flow control method for the video stream to be sent according to pre-configured flow control configuration information;

[0012] The video cache module is configured to save the to-be-sent video packets corresponding to the to-be-sent video stream into a video packet queue when the flow control mode is to perform flow control on the to-be-sent video stream and a video packet of a flow type corresponding to the to-be-sent video stream has been sent;

[0013] The video determination module is configured to determine the average data size of the video packet queue according to the video packet sizes of the video packets to be sent at a plurality of preset positions in the video packet queue;

[0014] The data sending module is configured to extract the video packets to be sent from the video packet queue according to the current sending rate, the preset rate threshold and the average data size, and send the extracted video packets to be sent to the transport layer for data transmission.

[0015] In a third aspect, an embodiment of the present application provides a live broadcast traffic control device, including: a memory and one or more processors;

[0016] The memory is used to store one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the live broadcast traffic control method as described in the first aspect.

[0018] In a fourth aspect, an embodiment of the present application provides a non-volatile storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the live broadcast traffic control method as described in the first aspect when executed by a computer processor.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor of a device reads and executes the computer program from the computer-readable storage medium, so that the device performs the live broadcast traffic control method as described in the first aspect.

[0020] The embodiment of the present application obtains the video stream to be sent, determines the flow control method for the video stream to be sent according to the pre-configured flow control configuration information, and when the flow control method is to perform flow control on the video stream to be sent and the video packets of the flow type corresponding to the video stream to be sent have been sent, saves the video packets to be sent corresponding to the video stream to be sent into the video packet queue, determines the average data size of the video packet queue according to the video packet sizes of the video packets to be sent at multiple preset positions in the video packet queue, extracts the video packets to be sent from the video packet queue according to the current sending rate, the preset rate threshold and the average data size, and sends the extracted video packets to be sent to the transport layer for data transmission, performs flow control on the data size of the video stream to be sent that needs flow control, is compatible with different transmission protocols, does not need to set a set of flow control logic for different transmission protocols, simplifies the complexity of live broadcast flow control development, and reduces the maintenance burden of live broadcast flow control. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of a live broadcast traffic control method provided by an embodiment of the present application;

[0022] Figure 2 is a flow chart of another live broadcast traffic control method provided by an embodiment of the present application;

[0023] Figure 3 This is a schematic diagram of a target flow control configuration information matching process provided by an embodiment of the present application;

[0024] Figure 4 This is a schematic diagram of a process for extracting a video packet to be sent provided in an embodiment of the present application;

[0025] Figure 5 It is a structural schematic diagram of a live broadcast flow control device provided in an embodiment of the present application;

[0026] Figure 6 It is a structural diagram of a live broadcast traffic control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The above process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The above process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0028] The live broadcast flow control method provided in the present application can be applied to flow control in live broadcast scenarios. It aims to perform flow control on the data size of the video stream to be sent that requires flow control. It is compatible with different transmission protocols and does not require a set of flow control logic to be set for different transmission protocols. It simplifies the development complexity of live broadcast flow control and reduces the maintenance burden of live broadcast flow control.

[0029] In different live broadcast scenarios, there are many different differences in traffic transmission, such as different transmission protocols controlling live streaming transmission in different scenarios, or the same transmission protocol but with different transmission parameters. The transmission protocol can be RTMP (Real-Time Messaging Protocol), HLS (HTTP Live Streaming, HTTP-based streaming media transmission) protocol, QUIC (Quick UDP Internet Connections) protocol, TFRC (TCP-Friendly Rate Control) protocol, etc. Both RTMP and HLS protocols use TCP as the transport layer protocol, while QUIC and TFRC protocols use UDP as the transport layer protocol. However, no matter what the scenario or transmission protocol is, there may be problems that require traffic control. For example, in order to reduce the impact of micro-bursts of server traffic, it is necessary to control the maximum sending bit rate, otherwise it may cause packet loss on the switch, affecting data transmission. Faced with a variety of live broadcast scenarios and different transport layer protocols, live broadcast traffic control often provides interfaces for traffic control at the protocol level. For the integration of multiple scenario transmission protocols in a live broadcast service, it is basically to call these traffic interfaces provided by the corresponding protocols to control the sending of traffic. For example, the live broadcasting scenarios include game live broadcasting and multi-person room live broadcasting. For game live broadcasting, the QUIC protocol needs to be used to transmit the live stream, and for multi-person room live broadcasting, the TFRC live stream needs to be used. However, both require the server to limit the maximum sending bit rate to the audience, and the rate control interface of the corresponding protocol needs to be called. At the same time, new application scenarios may be added in the future to use different protocols for transmission, but the requirements for limiting the sending bit rate are the same. The development of live broadcast traffic control is relatively complex, which brings a burden to code development and maintenance. Based on this, a live broadcast traffic control method according to an embodiment of the present application is provided to solve the technical problem that the development of live broadcast traffic control in the existing live broadcast traffic control scheme is relatively complex, which imposes a great burden on the maintenance of live broadcast traffic control.

[0030] Figure 1 A flow chart of a live broadcast traffic control method provided in an embodiment of the present application is given. The live broadcast traffic control method provided in an embodiment of the present application can be executed by a live broadcast traffic control device, which can be implemented in hardware and / or software and integrated in a live broadcast traffic control device (such as a server).

[0031] The following description is made by taking the live broadcast flow control device executing the live broadcast flow control method as an example. Figure 1 , the live broadcast flow control method includes:

[0032] S110: Obtain a video stream to be sent, and determine a flow control method for the video stream to be sent according to pre-configured flow control configuration information.

[0033] Exemplarily, the video stream to be sent to the client (the audience end in the live broadcast room) is obtained, and the flow control method for the video stream to be sent is determined according to the pre-configured flow control configuration information. The video stream to be sent can be a video stream provided by the anchor end, or a video stream provided by other terminals in the live broadcast room (for example, other anchor ends when multiple anchors are connected to the microphone, and other clients in the multi-person room). The video stream provided by this solution can be a source video stream provided by the corresponding terminal, or a transcoded video stream obtained by transcoding the source video stream through the transcoding server.

[0034] In one embodiment, the flow control configuration information provided by the solution can be used to record the video stream type that needs flow control, and different flow control configuration information can be configured for different video stream types that need flow control. Optionally, the video stream type provided by the solution can be a combination of one or more of room type, gear type, client type, region information, and application version information.

[0035] For example, it is determined whether there is corresponding flow control configuration information based on the video stream type of the video stream to be sent. When the corresponding flow control configuration information exists, it can be determined that the flow control method for the video stream to be sent is to perform flow control on the video stream to be sent. When the corresponding flow control configuration information does not exist, it can be determined that the flow control method for the video stream to be sent is not to perform flow control on the video stream to be sent.

[0036] In one embodiment, when the flow control method for the video stream to be sent is not to perform flow control on the video stream to be sent, the video packet to be sent corresponding to the video stream to be sent can be directly sent to the transport layer for data transmission, and the video stream to be sent is sent to the corresponding client at the transport layer.

[0037] S120: When the flow control mode is to perform flow control on the video stream to be sent, and video packets of a flow type corresponding to the video stream to be sent have been sent, the video packets to be sent corresponding to the video stream to be sent are saved in a video packet queue.

[0038] Exemplarily, when the flow control mode is to perform flow control on the video stream to be sent, it is determined whether a video packet of the flow type corresponding to the video stream to be sent has been sent. In one embodiment, when the flow control mode is to perform flow control on the video stream to be sent, and the video packet of the flow type corresponding to the video stream to be sent has been sent, the video packet to be sent corresponding to the video stream to be sent can be saved in the video packet queue. Among them, the video stream provided by this scheme includes multiple video packets, and a video packet queue for caching video packets is configured in the live broadcast flow control device, and the video packets stored in the video packet queue are sent when the current sending rate is less than the preset rate threshold. Optionally, the flow type provided by this scheme may include a combination of one or more of video gears (such as source stream, blue-ray, ultra-high definition, high definition, standard definition, etc.), room type (such as ordinary live broadcast room, game live broadcast room, single-person live broadcast room, multi-person live broadcast room, etc.) and microphone position (for example, different terminals in a multi-person microphone or battle live broadcast room correspond to different microphone positions, and different microphone positions correspond to different video streams).

[0039] In a possible embodiment, the live broadcast flow control method provided by the present scheme further includes, after determining the flow control method for the video stream to be sent according to the pre-configured flow control configuration information,: when the flow control method is to perform flow control on the video stream to be sent, and when video packets of the flow type corresponding to the video stream to be sent have not been sent, sending the video packets to be sent corresponding to the video stream to be sent to the transport layer for data transmission.

[0040] Exemplarily, when the flow control mode is to perform flow control on the video stream to be sent, and the video packet of the flow type corresponding to the video stream to be sent has not been sent (for example, the first video packet in the video stream to be sent is the first video packet sent to the client based on the flow type), the video packet to be sent corresponding to the video stream to be sent is directly sent to the transport layer for data transmission, and the flow type and data size of the sent video packet are recorded. This solution ensures normal data transmission by directly sending the video packet to be sent to the transport layer for data transmission when the flow control mode is to perform flow control on the video stream to be sent, but the video packet of the flow type corresponding to the video stream to be sent has not been sent, and providing a basis for subsequent flow control on the data of the flow type.

[0041] S130: Determine an average data size of the video packet queue according to the video packet sizes of the to-be-sent video packets at a plurality of preset positions in the video packet queue.

[0042] Exemplarily, according to a set time interval, the video packet sizes of the video packets to be sent at multiple preset positions in the video packet queue are determined, and the average data size of the video packet queue is determined according to the video packet sizes of the video packets to be sent at the multiple preset positions.

[0043] Optionally, the multiple preset positions in the video packet queue may be a combination of multiple positions in the queue head, the queue middle (e.g., multiple video packets in the queue middle), and the queue tail in the video packet queue, or may be all positions in the video packet queue. Optionally, the average of the video packet sizes of the video packets to be sent at the multiple preset positions may be used as the average data size of the video packet queue.

[0044] S140: extracting to-be-sent video packets from the video packet queue according to the current sending rate, a preset rate threshold, and an average data size, and sending the extracted to-be-sent video packets to a transport layer for data transmission.

[0045] Exemplarily, a data transmission rate at which the current transport layer sends data to the corresponding client is determined, and the data transmission rate is used as the current transmission rate, and the video packets to be sent are extracted from the video packet queue according to the current transmission rate, a preset rate threshold and the above-determined average data size. The preset rate threshold may be a preset rate threshold or a rate threshold configured in the flow control configuration information.

[0046] Optionally, the data size of the video packets in the video queue available for transmission can be determined according to the preset rate threshold and the current transmission rate, and the video packets to be transmitted whose total data volume is less than or equal to the data size can be extracted from the video packet queue. For example, the remaining transmission rate can be determined according to the difference between the preset rate threshold and the current transmission rate, and the data size of the video packets in the video queue available for transmission can be determined according to the product of the remaining transmission rate and the preset control time length (the preset control time length can range from 1 to 5 seconds, for example, the preset control time length is 3 seconds), and the video packets to be transmitted whose total data volume is less than or equal to the data size can be extracted from the video packet queue.

[0047] In one embodiment, after extracting the to-be-sent video packets from the video packet queue, the to-be-sent video packets may be sent to the transport layer, which performs data transmission on the to-be-sent video packets based on a set data transmission strategy to send the to-be-sent video packets to the client.

[0048] In the above, by obtaining the video stream to be sent, the flow control method for the video stream to be sent is determined according to the pre-configured flow control configuration information. When the flow control method is to perform flow control on the video stream to be sent and the video packets of the flow type corresponding to the video stream to be sent have been sent, the video packets to be sent corresponding to the video stream to be sent are saved in the video packet queue. According to the video packet sizes of the video packets to be sent at multiple preset positions in the video packet queue, the average data size of the video packet queue is determined. According to the current sending rate, the preset rate threshold and the average data size, the video packets to be sent are extracted from the video packet queue, and the extracted video packets to be sent are sent to the transport layer for data transmission. Flow control is performed on the data size of the video stream to be sent that needs flow control. Different transmission protocols are compatible, and there is no need to set a set of flow control logics for different transmission protocols. The complexity of live broadcast flow control development is simplified and the maintenance burden of live broadcast flow control is reduced.

[0049] Based on the above embodiments, Figure 2 A flowchart of another live broadcast traffic control method provided by an embodiment of the present application is given, and the live broadcast traffic control method is a specific embodiment of the above live broadcast traffic control method. Figure 2 , the live broadcast flow control method includes:

[0050] S210: Acquire the video stream to be sent, and determine the video attribute information of the video stream to be sent.

[0051] S220: Match target flow control configuration information in a plurality of pre-configured flow control configuration information according to the video attribute information.

[0052] S230: When the target flow control configuration information is matched, determine that the flow control method for the video stream to be sent is to perform flow control on the video stream to be sent.

[0053] Exemplarily, after obtaining the video stream to be sent, the video stream to be sent is parsed and video attribute information carried in the video stream to be sent is determined based on the parsing result, wherein the video attribute information includes a room type (i.e., the room type corresponding to the video stream to be sent, such as a game live broadcast room, a multi-person live broadcast room, etc.), a gear type (i.e., the video gear corresponding to the video stream to be sent, such as the source stream, Blu-ray, HD, etc. subscribed by the client to the server), a client type (i.e., the operating system type corresponding to the client receiving the video stream to be sent, such as Android system, iOS system, etc.), regional information (i.e., the location of the client) and application version information (i.e., the version information of the application run by the client) One or more combinations thereof.

[0054] The information recorded in the flow control configuration information provided by this solution includes one or more combinations of room type restriction information (i.e., the room type that requires flow restriction), gear type restriction information (i.e., the video gear that requires flow restriction), client type restriction information (i.e., the client type that requires flow restriction), region restriction information (i.e., the region that requires flow restriction), application version restriction information (i.e., the application version that requires flow restriction, which may be one or more specific application versions or one or more application version ranges), and preset rate threshold (i.e., limiting the maximum rate at which data is sent to the client). Different flow control configuration information may be pre-configured for different flow control requirements, and flow control switches may be configured for different flow control configuration information. When the flow control switch is turned on, the corresponding flow control configuration information participates in the live flow control, and the flow control mode may also be configured, such as limiting the maximum sending bit rate, limiting the sending time of the first video packet, or only applying flow control at the initial time of video stream sending or within a certain time range in the middle, so as to flexibly configure the flow control of the live video stream.

[0055] In one embodiment, target flow control configuration information is matched among multiple pre-configured flow control configuration information based on video attribute information. When there is flow control configuration information among the multiple flow control configuration information that is consistent with the video attribute information of the video stream to be sent or the corresponding range includes the video attribute information, the flow control configuration information can be used as the matched target flow control configuration information.

[0056] For example, in the flow control configuration information, the room type restriction information: room type = "nomal" (normal means broadcasting in an ordinary room), the gear type restriction information: channel type = "save" (save means the flow saving mode, i.e., restricting viewers who choose to watch the standard definition gear), the client type restriction information: client type = "0" (type = "0" means the Android system), the region restriction information: cc = "AA", the application version restriction information: minVer = "6755" maxVer = "6808", the preset rate threshold: maxSendRate = "500", which means that for the room where the host broadcasts in an ordinary room, the viewers in the AA region whose Android client versions are between 6755 and 6808 will be limited to a maximum sending bit rate (preset rate threshold) of 500 kbps when watching the standard definition gear. Room type restriction information in the flow control configuration information: room type = "multi" (multi means broadcasting in a multi-person room), gear type restriction information: channel type = "origin" (origin means the source stream gear), client type restriction information: client type = "1" (type = "1" means iOS system), region restriction information: cc = "BB", preset rate threshold: maxSendRate = "600", indicating that iOS client viewers in the BB region will be limited to a maximum sending bit rate of 600kbps (preset rate threshold) when watching the source stream.

[0057] In one embodiment, the flow control configuration information provided by this solution can be described in XML format, which is convenient for viewing and modifying the flow control configuration information. At the same time, the hierarchical and structured form of the XML format is also conducive to the diversification of flow control configuration. The flow control configuration information can provide a callback function sendVideo, which is used to forward the video packets that can be forwarded to the transport layer. The flow control configuration information can be connected to the live broadcast project in the form of source code, or it can be compiled into a static library separately in the form of a library, and then connected to the live broadcast project by providing a header file. It can limit the burst traffic during the live broadcast process, such as the pre-pulled live stream, and adopts a modular design of the flow control function. The flow control has good compatibility and high configurability, and minimizes the impact on the live broadcast freeze within the flow rate limit.

[0058] In one possible embodiment, Figure 3 As shown in a target flow control configuration information matching process diagram provided, the live broadcast flow control method provided by the present solution matches the target flow control configuration information in a plurality of pre-configured flow control configuration information according to the video attribute information, including:

[0059] S221: Determine the duration of the client corresponding to the video stream to be sent entering the live broadcast room.

[0060] S222: Match target flow control configuration information whose preset control time length is greater than or equal to the duration from among the pre-configured multiple flow control configuration information according to the video attribute information.

[0061] In order to increase the speed of video playback and reduce the impact of bitrate restrictions on viewers watching live broadcasts, more video packets can be sent as quickly as possible within the bitrate limit. For example, for a 2Mbps live stream, if each video packet is 500 bytes long, there will be 500 video packets in one second. If a viewer enters the room and pre-pull 2 ​​seconds of video, 1000 video packets need to be forwarded to the viewer. Assuming the bitrate limit (bitrate limit is configured by preset rate threshold) is 2.5Mbps, 625 video packets will be sent first, and the remaining 375 video packets will be saved in the video packet queue and accumulated with the newly received video packets, waiting to be sent again in the next second. Assuming the bitrate limit is 2Mbps and is consistent with the live stream bitrate, video playback may be affected by network jitter and cause jamming. The preset control time length can be combined with the flow control to limit only the video packet flow within the preset control time length when entering the room, and the flow rate after the preset control time length is not restricted, reducing the jamming of video playback.

[0062] In one embodiment, the flow control configuration information provided by the present solution corresponds to a preset control time length, which can be configured in the flow control configuration information, or the preset control time length can be separately specified for the flow control configuration information. The preset control time length can be understood as the time length for which the live broadcast flow needs to be controlled, that is, when the duration of the user entering the live broadcast room is within the preset control time length, the flow control is performed on the video stream matching the flow control configuration information sent to the client, and when the duration of the user entering the live broadcast room reaches or exceeds the preset control time length, the flow control is not performed on the video stream sent to the client.

[0063] Exemplarily, the timestamp of the client entering the live broadcast room required for the video stream to be sent is determined, and the duration of the client entering the live broadcast room is determined according to the difference between the current time and the timestamp. The duration and the preset control time length corresponding to each flow control configuration information are determined, and the flow control configuration information whose preset control time length is greater than the duration is determined, and the target flow control configuration information is matched among the flow control configuration information whose preset control time length is greater than the duration according to the video attribute information, and the target flow control configuration information whose preset control time length is greater than or equal to the duration is determined, or the candidate flow control configuration information is matched among multiple flow control configuration information according to the video attribute information, and the candidate flow control configuration information whose preset control time length is greater than or equal to the duration is used as the target flow control configuration information.

[0064] This solution flexibly controls the time length of flow control by determining the duration of the client corresponding to the video stream to be sent entering the live broadcast room, and matching the target flow control configuration information with a preset control time length greater than or equal to the duration in multiple pre-configured flow control configuration information according to the video attribute information.

[0065] In one embodiment, when the target flow control configuration information is matched, it is determined that the flow control method for the video stream to be sent is to perform flow control on the video stream to be sent. When the target flow control configuration information is not matched, it is determined that the flow control method for the video stream to be sent is not to perform flow control on the video stream to be sent, and the video packet to be sent corresponding to the video stream to be sent can be directly sent to the transport layer for data transmission, and the video stream to be sent is sent to the corresponding client at the transport layer. This scheme matches the target flow control configuration information in a plurality of pre-configured flow control configuration information according to the video attribute information of the video stream to be sent, and when the target flow control configuration information is matched, it is determined that the flow control method for the video stream to be sent is to perform flow control on the video stream to be sent, thereby realizing flexible configuration of live broadcast traffic, simplifying the complexity of live broadcast traffic control development, and reducing the maintenance burden of live broadcast traffic control.

[0066] S240: When the flow control mode is to perform flow control on the video stream to be sent, and video packets of a flow type corresponding to the video stream to be sent have been sent, the video packets to be sent corresponding to the video stream to be sent are saved in a video packet queue.

[0067] S250: Determine an average data size of the video packet queue according to the video packet sizes of the to-be-sent video packets at a plurality of preset positions in the video packet queue.

[0068] In a possible embodiment, the live broadcast flow control method provided by the present solution determines the average data size of the video packet queue based on the video packet sizes of the video packets to be sent at multiple preset positions in the video packet queue, and can be determined based on the video packet sizes of the video packets to be sent at the head and tail of the queue in the video packet queue.

[0069] Exemplarily, the video packet sizes of the to-be-sent video packets at the head and tail of the video packet queue are determined, and the average video packet sizes of the to-be-sent video packets at the head and tail of the queue are determined, and the average is used as the average data size of the video packet queue.

[0070] Among them, due to the large number of video packets generated during the live broadcast process, the video packets entering the flow control stage are usually received in a list manner. For video streams with higher bit rates, the number of video packets will be greater. For example, an 8Mbps video stream is divided according to the length of each video packet of 500 bytes, with 2 video packets per millisecond, and 2000 video packets per second. In order to improve the audience's experience of watching live broadcasts, video packets of a certain length of time (for example, 2 seconds or longer) are usually pre-pulled to the audience when the audience enters the live broadcast room. These pre-pulled video packets need to be quickly transmitted to the audience's client through the network for cache playback, thereby reducing the jamming caused by network jitter and other problems and improving the viewing experience. For a large number of video packets, if the data length of each packet is to be checked, all video packets of the video stream to be sent need to be traversed at once, which is likely to cause an increase in CPU time consumption and an increase in audience delay, especially in scenarios where there are many video streams and flow control is required. This solution estimates the size of video packets stored in the video packet queue based on the average size of the video packets to be sent at the head and tail of the queue, and calculates the number of video packets that can be sent currently based on the preset rate threshold and the average data size, thereby reducing the load problem that may be caused by traversing all video packets and improving the efficiency of live broadcast traffic control.

[0071] S260: extracting to-be-sent video packets from the video packet queue according to the current sending rate, the preset rate threshold, and the average data size, and sending the extracted to-be-sent video packets to the transport layer for data transmission.

[0072] In one possible embodiment, in one possible embodiment, as Figure 4 As shown in the schematic diagram of a process of extracting a video packet to be sent, the live broadcast flow control method provided by the present solution extracts the video packet to be sent from the video packet queue according to the current sending rate, the preset rate threshold and the average data size, including:

[0073] S261: When the current sending rate is less than a preset rate threshold, determine the number of sendable video packets according to the preset rate threshold and the average data size.

[0074] S262: extracting the video packets to be sent from the video packet queue according to the number of sendable video packets.

[0075] Exemplarily, the current sending rate of data to the client corresponding to the video stream to be sent is determined, and it is determined whether the current sending rate is less than a preset rate threshold. When the current sending rate is greater than or equal to the preset rate threshold, the received video packets to be sent in the video stream to be sent continue to be saved in the video packet queue, and the video packets are sent when the current sending rate is less than the preset rate threshold.

[0076] When the current sending rate is less than the preset rate threshold, the number of sendable video packets is determined according to the preset rate threshold and the average data size, and the number of to-be-sent video packets less than or equal to the number of sendable video packets is extracted from the video packet queue according to the number of sendable video packets, and the extracted to-be-sent video packets are sent to the transport layer for data transmission, thereby effectively limiting the sending rate of data sent to the client within the preset rate threshold, reducing the situation where video packet loss due to a sudden increase in server traffic affects the live broadcast effect.

[0077] In one embodiment, the live broadcast traffic control method provided by the present solution determines the number of video packets that can be sent according to a preset rate threshold and an average data size, which can be: determining the maximum sending length according to a preset rate threshold and a preset control time length; determining the sendable video packet length according to the maximum sending length and the sent video packet length; determining the number of sendable video packets according to the sendable video packet length and the average data size.

[0078] Exemplarily, the cumulative length of video packets of the corresponding traffic type that have been sent to the client corresponding to the video stream to be sent is determined, the cumulative length is used as the length of the sent video packets, the product of the preset rate threshold and the preset control time length is used as the maximum sending length, the difference between the maximum sending length and the length of the sent video packets is determined as the sendable video packet length, and the ratio of the sendable video packet length to the average data size is determined as the number of sendable video packets. Wherein, each time a video packet to be sent is sent to the transport layer, the length of the video packet to be sent to the transport layer is recorded, and the length of the sent video packet can be determined based on the cumulative video packet length. For example, the number of sendable video packets provided by this solution can be determined by the following formula:

[0079] len=SMax*t

[0080] lne_now=lne-lne_old

[0081] num=len_now / x

[0082] Among them, len is the maximum sending length, SMax is the preset rate threshold, t is the preset control time length, len_now is the sendable video packet length, len_old is the sent video packet length, num is the number of sendable video packets, and x is the average data size. This solution effectively limits the sending rate of data to the client within the preset rate threshold by accurately determining the number of sendable video packets based on the preset rate threshold, the preset control time length, the sent video packet length and the average data size, thereby reducing the situation where video packets are lost due to a sudden increase in server traffic, affecting the live broadcast effect.

[0083] In the above, by obtaining the video stream to be sent, the flow control method for the video stream to be sent is determined according to the pre-configured flow control configuration information. When the flow control method is to perform flow control on the video stream to be sent and the video packets of the flow type corresponding to the video stream to be sent have been sent, the video packets to be sent corresponding to the video stream to be sent are saved in the video packet queue. According to the video packet sizes of the video packets to be sent at multiple preset positions in the video packet queue, the average data size of the video packet queue is determined. According to the current sending rate, the preset rate threshold and the average data size, the video packets to be sent are extracted from the video packet queue, and the extracted video packets to be sent are sent to the transport layer for data transmission. Flow control is performed on the data size of the video stream to be sent that needs flow control. Different transmission protocols are compatible, and there is no need to set a set of flow control logics for different transmission protocols. The complexity of live broadcast flow control development is simplified and the maintenance burden of live broadcast flow control is reduced. By matching target flow control configuration information in multiple pre-configured flow control configuration information according to the video attribute information of the video stream to be sent, and when matching the target flow control configuration information, determining that the flow control method for the video stream to be sent is to perform flow control on the video stream to be sent, flexible configuration of live broadcast flow is achieved, the complexity of live broadcast flow control development is simplified, and the maintenance burden of live broadcast flow control is reduced.

[0084] Figure 5 Schematic diagram of the structure of a live broadcast flow control device provided by an embodiment of the present application. Figure 5 The live broadcast flow control device includes a control judgment module 51, a video buffer module 52, a video determination module 53 and a data sending module 54.

[0085] Among them, the control judgment module 51 is configured to obtain the video stream to be sent, and determine the flow control method for the video stream to be sent according to the pre-configured flow control configuration information; the video caching module 52 is configured to save the video packets to be sent corresponding to the video stream to be sent into the video packet queue when the flow control method is to perform flow control on the video stream to be sent and the video packets of the flow type corresponding to the video stream to be sent have been sent; the video determination module 53 is configured to determine the average data size of the video packet queue according to the video packet sizes of the video packets to be sent at multiple preset positions in the video packet queue; the data sending module 54 is configured to extract the video packets to be sent in the video packet queue according to the current sending rate, the preset rate threshold and the average data size, and send the extracted video packets to be sent to the transport layer for data transmission.

[0086] It is worth noting that in the embodiment of the above-mentioned live broadcast traffic control device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present application.

[0087] The embodiment of the present application also provides a live broadcast traffic control device, which can integrate the live broadcast traffic control device provided by the embodiment of the present application. Figure 6 Schematic diagram of a live broadcast flow control device provided by an embodiment of the present application. Figure 6 The live broadcast traffic control device includes: an input device 63, an output device 64, a memory 62 and one or more processors 61; the memory 62 is used to store one or more programs; when one or more programs are executed by one or more processors 61, one or more processors 61 implement the live broadcast traffic control method provided in the above embodiment. The live broadcast traffic control device, equipment and computer provided above can be used to execute the live broadcast traffic control method provided in any of the above embodiments, and have corresponding functions and beneficial effects.

[0088] The embodiments of the present application also provide a non-volatile storage medium storing computer executable instructions, which are used to execute the live broadcast traffic control method provided in the above embodiments when executed by a computer processor. Of course, the non-volatile storage medium storing computer executable instructions provided in the embodiments of the present application, whose computer executable instructions are not limited to the live broadcast traffic control method provided above, can also execute the related operations in the live broadcast traffic control method provided in any embodiment of the present application. The live broadcast traffic control device, equipment and storage medium provided in the above embodiments can execute the live broadcast traffic control method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the live broadcast traffic control method provided in any embodiment of the present application.

[0089] On the basis of the above embodiments, the embodiments of the present application also provide a computer program product. The technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer program product is stored in a storage medium, and includes a number of instructions for enabling a computer device, a mobile terminal or a processor therein to execute all or part of the steps of the live broadcast traffic control method provided in each embodiment of the present application.

Claims

1. A live broadcast flow control method, characterized in that: include: Acquire a video stream to be sent, and determine a flow control method for the video stream to be sent according to pre-configured flow control configuration information; When the flow control mode is to perform flow control on the video stream to be sent, and a video packet of a flow type corresponding to the video stream to be sent has been sent, the video packet to be sent corresponding to the video stream to be sent is saved in a video packet queue; determining an average data size of the video packet queue according to video packet sizes of video packets to be sent at a plurality of preset positions in the video packet queue; The video packets to be sent are extracted from the video packet queue according to the current sending rate, the preset rate threshold and the average data size, and the extracted video packets to be sent are sent to the transport layer for data transmission.

2. The live broadcast flow control method according to claim 1, characterized in that: The determining of the flow control method for the video stream to be sent according to the pre-configured flow control configuration information includes: Determine video attribute information of the video stream to be sent, wherein the video attribute information includes one or more combinations of room type, gear type, client type, region information, and application version information; Matching target flow control configuration information in a plurality of pre-configured flow control configuration information according to the video attribute information, wherein the flow control configuration information includes a combination of one or more of room type restriction information, gear type restriction information, client type restriction information, region restriction information, application version restriction information, and a preset rate threshold; In the case where the target flow control configuration information is matched, it is determined that the flow control method for the video stream to be sent is to perform flow control on the video stream to be sent.

3. The live broadcast flow control method according to claim 2, characterized in that: The matching of target flow control configuration information among a plurality of pre-configured flow control configuration information according to the video attribute information includes: Determine the duration of the client corresponding to the video stream to be sent entering the live broadcast room; According to the video attribute information, target flow control configuration information having a preset control time length greater than or equal to the duration is matched among a plurality of pre-configured flow control configuration information.

4. The live broadcast flow control method according to claim 1, characterized in that: The step of determining the average data size of the video packet queue according to the video packet sizes of the video packets to be sent at a plurality of preset positions in the video packet queue comprises: The average data size of the video packet queue is determined according to the video packet sizes of the to-be-sent video packets at the head and the tail of the video packet queue.

5. The live broadcast flow control method according to claim 1, characterized in that: The extracting the to-be-sent video packets from the video packet queue according to the current sending rate, the preset rate threshold and the average data size comprises: When the current sending rate is less than a preset rate threshold, determining the number of video packets that can be sent according to the preset rate threshold and the average data size; The video packets to be sent are extracted from the video packet queue according to the number of sendable video packets.

6. The live broadcast flow control method according to claim 5, characterized in that: The determining the number of transmittable video packets according to the preset rate threshold and the average data size includes: Determine the maximum sending length according to the preset rate threshold and the preset control time length; Determine the transmittable video packet length according to the maximum transmit length and the transmitted video packet length; The number of transmittable video packets is determined according to the transmittable video packet length and the average data size.

7. The live broadcast flow control method according to claim 1, characterized in that: After determining the flow control method for the video stream to be sent according to the pre-configured flow control configuration information, the method further includes: When the flow control mode is to perform flow control on the video stream to be sent and video packets of a flow type corresponding to the video stream to be sent have not been sent, the video packets to be sent corresponding to the video stream to be sent are sent to the transport layer for data transmission.

8. A live broadcast flow control device, characterized in that: It includes a control judgment module, a video buffer module, a video determination module and a data sending module, wherein: The control judgment module is configured to obtain a video stream to be sent, and determine a flow control method for the video stream to be sent according to pre-configured flow control configuration information; The video cache module is configured to save the to-be-sent video packets corresponding to the to-be-sent video stream into a video packet queue when the flow control mode is to perform flow control on the to-be-sent video stream and a video packet of a flow type corresponding to the to-be-sent video stream has been sent; The video determination module is configured to determine the average data size of the video packet queue according to the video packet sizes of the video packets to be sent at a plurality of preset positions in the video packet queue; The data sending module is configured to extract the video packets to be sent from the video packet queue according to the current sending rate, the preset rate threshold and the average data size, and send the extracted video packets to be sent to the transport layer for data transmission.

9. A live broadcast flow control device, characterized in that: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the live broadcast traffic control method as described in any one of claims 1-7.

10. A non-volatile storage medium storing computer executable instructions, characterized in that: The computer executable instructions are used to execute the live broadcast traffic control method as described in any one of claims 1-7 when executed by a computer processor.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the live broadcast traffic control method described in any one of claims 1-7 is implemented.

Images