Bitrate control method, device, media server, terminal device and storage medium

Abstract

An embodiment of the present disclosure provides a bitrate control method, an apparatus, a media server, a terminal device, and a storage medium, the method including: generating discrete bandwidth information for a plurality of subscriber clients based on available bandwidth information of the subscriber clients, the discrete bandwidth information representing the available bandwidth allocated to the subscriber clients and the bandwidth intervals in which the available bandwidth exists; aggregating the plurality of discrete bandwidth information into aggregated bandwidth information representing candidate available bandwidths for each bandwidth interval and the total number of subscriber clients in each bandwidth interval; and pushing the aggregated bandwidth information to the publisher clients for determining a push stream bitrate by the publisher clients.

Description

[Technical Field]

[0001] This application claims priority from Chinese Patent Application No. 202310139262.5, filed on February 20, 2023, the entire disclosure of which is incorporated herein by reference. [Technical field]

[0002] The present disclosure relates to a bitrate control method, device, media server, terminal device, and storage medium. [Background technology]

[0003] Currently, in a multi-party real-time communication process, a publisher client can push an audio-video stream to multiple subscriber clients, specifically, the publisher client typically encodes the audio-video stream at a predetermined bit rate and pushes the encoded audio-video stream to each subscriber client.

[0004] However, in actual application scenarios, when a subscriber client loads an encoded audio-video stream via downlink, the bit rate of the audio-video stream is likely to be inconsistent with its available bandwidth, and if the bit rate of the audio-video stream is significantly higher or lower than the available bandwidth of the subscriber client, this will result in poor communication quality during the communication process. Summary of the Invention [Problem to be solved by the invention]

[0005] In view of this, one or more embodiments of the present disclosure provide a bitrate control method, device, media server, terminal device, and storage medium that can improve communication quality in a communication process involving multiple people. [Means for solving the problem]

[0006] One aspect of the present disclosure provides a bitrate control method, including: generating discrete bandwidth information for a plurality of subscriber clients based on available bandwidth information of the subscriber clients, the discrete bandwidth information representing available bandwidth allocated to the subscriber clients and bandwidth intervals in which the available bandwidth exists; aggregating the plurality of discrete bandwidth information into aggregated bandwidth information representing candidate available bandwidths for each bandwidth interval and a total number of subscriber clients in each bandwidth interval; and pushing the aggregated bandwidth information to the publisher clients for determining a push stream bitrate by the publisher clients.

[0007] Another aspect of the present disclosure further provides a bitrate control device including: a discrete bandwidth information generating unit for generating discrete bandwidth information for a plurality of subscriber clients based on available bandwidth information of the subscriber clients, the discrete bandwidth information representing available bandwidth allocated to the subscriber clients and bandwidth intervals in which the available bandwidth exists; an aggregation unit for aggregating the plurality of discrete bandwidth information into aggregated bandwidth information representing candidate available bandwidths for each bandwidth interval and a total number of subscriber clients in each bandwidth interval; and a push unit for pushing the aggregated bandwidth information to the publisher clients for determining a push stream bitrate by the publisher clients.

[0008] Another aspect of the present disclosure further provides a media server including a memory for storing a computer program and a processor, the computer program implementing the above bitrate control method when executed by the processor.

[0009] Another aspect of the present disclosure further provides a bitrate control method, including receiving, from a media server, aggregate bandwidth information representing candidate available bandwidths for each bandwidth interval and a total number of subscriber clients in each of the bandwidth intervals; determining a push stream bitrate based on the aggregate bandwidth information; encoding an audio-video stream at the push stream bitrate; and pushing the encoded audio-video stream to each subscriber client.

[0010] Another aspect of the present disclosure further provides a bitrate control device, including: an information receiving unit for receiving aggregate bandwidth information from a media server, the aggregate bandwidth information representing candidate available bandwidths for each bandwidth interval and a total number of subscriber clients in each of the bandwidth intervals; a bitrate determination unit for determining a push stream bitrate based on the aggregate bandwidth information and encoding an audio-video stream at the push stream bitrate; and an audio-video stream push unit for pushing the encoded audio-video stream to each subscriber client.

[0011] Another aspect of the present disclosure further provides a terminal device including a memory for storing a computer program and a processor, the terminal device realizing the above-described bit rate control method when the computer program is executed by the processor.

[0012] Another aspect of the present disclosure further provides a computer-readable storage medium, the computer-readable storage medium being used to store a computer program, the computer program realizing the above-described bitrate control method when executed by a processor.

[0013] The features and advantages of the embodiments of the present disclosure can be more clearly understood by referring to the drawings, which are illustrative and should not be construed as limiting the present disclosure. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 shows a schematic diagram of steps of a bit rate control method according to one embodiment of the present disclosure. [Figure 2] FIG. 2 shows a schematic diagram of a system architecture according to one embodiment of the present disclosure. [Figure 3] FIG. 3 shows a schematic diagram of the steps of generating discrete bandwidth information according to one embodiment of the present disclosure. [Figure 4] FIG. 4 shows a schematic diagram of a multi-tiered system architecture according to one embodiment of the present disclosure. [Figure 5] FIG. 5 shows a schematic diagram of functional modules of a bit rate control device according to one embodiment of the present disclosure. [Figure 6] FIG. 6 shows a structural diagram of a media server according to one embodiment of the present disclosure. [Figure 7] FIG. 7 shows a schematic diagram of steps of a bit rate control method according to another embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0015] In order to make the objectives, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings of the embodiments of the present disclosure. It is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without requiring creative work fall within the protection scope of the present disclosure.

[0016] A communication process involving multiple participants may include, for example, a video conference process, in which a client initiating desktop sharing may be a publisher client, and other clients participating in the video conference may be subscriber clients. When a publisher client pushes an audio-video stream to a subscriber client via a media server, the stream is typically encoded at a preset bitrate built into the video conference software. For example, the video conference software may set the image quality of the audio-video stream to 720P resolution by default, and the 720P resolution corresponds to a default bitrate (typically 3 Mb / s). In this case, the publisher client encodes the audio-video stream at a bitrate of 3 Mb / s, and the encoded 720P audio-video stream is pushed to each subscriber client participating in the conference via the media server of the video conference software.

[0017] Conventionally, multi-party communication processes are typically implemented using a selective forwarding unit (SFU) architecture. In the SFU architecture, encoding by a publisher client may be performed as described above. However, in actual application scenarios, different subscriber clients may be located in different network environments. Therefore, if the bit rate of the audio-video stream is significantly higher than the available bandwidth of the majority of subscriber clients, the communication process may freeze for the majority of subscriber clients. If the bit rate of the audio-video stream is significantly lower than the available bandwidth of the majority of subscriber clients, the image or sound quality may deteriorate. Traditional SFU architectures typically cannot adaptively adjust the encoding bit rate of publisher clients. The reason for this is that some current bitrate control protocols, such as the Temporary Maximum Media Stream Bit Rate Request (TMMBR) protocol and the Receiver Estimated Maximum Bitrate (REMB) protocol, are usually only applicable to Point-to-Point (P2P) or MultiPoint Control Unit (MCU) communication architectures, and cannot be well extended to the SFU architecture.

[0018] To solve the above problems, one embodiment of the present disclosure provides a bitrate control method, which can be applied to a media server of multi-party communication software. The media server may be a server in an SFU architecture and is used to store and / or transfer audio-video data between publisher clients and subscriber clients. In this embodiment, the data processing logic of the media server can be improved. Referring to FIG. 1, the method may include the following steps:

[0019] S11: Based on available bandwidth information of a plurality of subscriber clients, generate discrete bandwidth information for the subscriber clients, where the discrete bandwidth information represents available bandwidth allocated to the subscriber clients and a bandwidth section in which the available bandwidth exists.

[0020] Referring to FIG. 2, in this embodiment, multiple subscriber clients may be connected to the same media server. In practical applications, the subscriber clients may spontaneously report their available bandwidth information to the media server, and the media server may detect the available bandwidth information of each subscriber client through a preset congestion control and bandwidth allocation algorithm. The available bandwidth information spontaneously reported by the subscriber clients may represent the downlink bitrate that the subscriber clients can support, or may be the idle downlink bitrate under the current load condition. Of course, the available bandwidth information may have different meanings depending on the scenario, and this disclosure is not limited thereto. The available bandwidth information detected by the media server may generally represent the downlink bitrate that the media server can allocate to the subscriber clients. In practical applications, the downlink bitrate detected by the media server is generally less than or equal to the downlink bitrate spontaneously reported by the subscriber clients.

[0021] In this embodiment, when detecting available bandwidth information using a congestion control and bandwidth allocation algorithm, the media server can initialize a congestion window, which represents the amount of data the media server pushes to the subscriber client. The media server can send a detection message to the subscriber client using the congestion window, and the media server can increase the congestion window upon receiving a response message from the subscriber client. If the subscriber client is unable to return the detection message in a timely manner due to network congestion, the media server can appropriately reduce the congestion window. With this congestion control method, the media server can ultimately determine the available bandwidth allocated to the subscriber client.

[0022] Of course, in practical applications, there are many ways to implement congestion control and bandwidth allocation algorithms, and different algorithms can be flexibly selected according to the needs of the scene, and the present disclosure does not limit this.

[0023] In this embodiment, regardless of the method used, after obtaining the available bandwidth information of the subscriber client, the media server can further determine the available bandwidth allocated to the subscriber client. Specifically, if the available bandwidth reported by the subscriber client is smaller than the maximum available bandwidth that can be allocated by the media server, the available bandwidth reported by the subscriber client can be used as the available bandwidth allocated to the subscriber client by the media server. If the available bandwidth reported by the subscriber client is equal to or greater than the maximum available bandwidth that can be allocated by the media server, the maximum available bandwidth that can be allocated by the media server can be used as the available bandwidth allocated to the subscriber client by the media server. Furthermore, if the media server detects the available bandwidth through a congestion control and bandwidth allocation algorithm, the detected available bandwidth can be used as the available bandwidth allocated to the subscriber client by the media server.

[0024] In some cases, to control the load on the media server, the available bandwidth allocated to the subscriber client by the media server may not reach the subscriber client's own maximum available bandwidth. For example, the subscriber client may spontaneously report a maximum available bandwidth of 150 Mb / s, but the maximum available bandwidth allocated to the subscriber client by the media server may be only 10 Mb / s.

[0025] In this embodiment, the media server can allocate available bandwidth to each subscriber client and then generate discrete bandwidth information for each subscriber client. Specifically, the media server can pre-configure multiple bandwidth intervals based on the upper limit of the available bandwidth that can be provided. For example, if the media server can allocate a maximum available bandwidth of 10 Mb / s to each subscriber client, the media server can generate four bandwidth intervals, which may be, for example, 0-2 Mb / s, 2 Mb / s-6 Mb / s, 6 Mb / s-8 Mb / s, and 8 Mb / s-10 Mb / s. Of course, the number of bandwidth intervals can be pre-allocated according to actual circumstances, and this disclosure is not limited thereto.

[0026] In this embodiment, after dividing the multiple bandwidth intervals, it is possible to determine, for a certain subscriber client, the bandwidth interval in which the available bandwidth allocated by the media server to the subscriber client exists. For example, if the available bandwidth allocated to a certain subscriber client is 4 Mb / s, the available bandwidth of the subscriber client may be within the second bandwidth interval. In practical application, the discrete bandwidth information of the subscriber client may record the available bandwidth allocated by the media server to the subscriber client and the bandwidth interval in which the available bandwidth exists.

[0027] In one embodiment, in order to improve the efficiency of data processing, the discrete bandwidth information of the subscriber client can be represented in the form of a vector. Specifically, referring to FIG. 3, the process of generating the discrete bandwidth information may include the following steps:

[0028] S111: Based on a plurality of preset bandwidth intervals, an interval discrete vector and a bandwidth discrete vector are generated, each of whose vector elements corresponds one-to-one to the plurality of preset bandwidth intervals.

[0029] In this embodiment, the media server can generate interval discrete vectors and bandwidth discrete vectors based on the multiple bandwidth intervals pre-divided by the media server. The number of vector elements in the interval discrete vectors and bandwidth discrete vectors may match the number of bandwidth intervals. For example, if the media server currently divides four bandwidth intervals, then both the interval discrete vector and the bandwidth discrete vector may include four vector elements. In this way, the vector elements in the interval discrete vector and the bandwidth discrete vector may correspond one-to-one to the multiple bandwidth intervals pre-defined. For example, an interval discrete vector V1 and a bandwidth discrete vector V2 may be generated for the four pre-defined bandwidth intervals.

[0030]

number

[0031] Therefore, the initial values ​​of the vector elements in the interval discrete vector V1 and the bandwidth discrete vector V2 can both be 0, and each vector element corresponds to one bandwidth interval. For example, the first vector element corresponds to the bandwidth interval of 0 to 2 Mb / s, and the third vector element corresponds to the bandwidth interval of 6 Mb / s to 8 Mb / s.

[0032] S113: Determine a bandwidth interval in which the available bandwidth allocated to the subscriber client exists, and set a mark value to the vector element in the interval discrete vector corresponding to the existing bandwidth interval, and also set an available bandwidth value to the vector element in the bandwidth discrete vector corresponding to the existing bandwidth interval.

[0033] In this embodiment, values ​​can be assigned to some of the vector elements in the interval discrete vector and the bandwidth discrete vector by identifying the available bandwidth of the subscriber client. For example, if the available bandwidth allocated to subscriber client A is 4 Mb / s, the available bandwidth is in the second bandwidth interval. Therefore, according to the rule of step S113, a mark value can be set to the second vector element in the interval discrete vector. In practical applications, the mark value can be 1, indicating that the available bandwidth of a subscriber client is in the bandwidth interval. Also, an available bandwidth value can be set to the second vector element in the bandwidth discrete vector. The available bandwidth value can be the available bandwidth allocated to the subscriber client by the media server. Thus, the allocated interval discrete vector V1 and bandwidth discrete vector V2 can be expressed as follows:

[0034]

number

[0035] By identifying these two vectors, it can be seen that the available bandwidth of the current subscriber client is 4 Mb / s, and that the available bandwidth is within the second bandwidth interval.

[0036] S115: A combination of the allocated section discrete vector and the allocated bandwidth discrete vector is set as the discrete bandwidth information of the subscriber client.

[0037] In this embodiment, after assigning values ​​to the interval discrete vector and the bandwidth discrete vector, the combination of the two assigned vectors can be taken as the discrete bandwidth information of the subscriber client.

[0038] Through the above steps S111 to S115, the media server can generate discrete bandwidth information corresponding to each connected subscriber client one by one.

[0039] S13: Aggregate the plurality of discrete bandwidth information into aggregate bandwidth information representing the candidate available bandwidth for each bandwidth interval and the total number of subscriber clients in each bandwidth interval.

[0040] In this embodiment, the media server generates discrete bandwidth information for each subscriber client, and then aggregates the discrete bandwidth information to obtain aggregate bandwidth information. Specifically, the aggregate bandwidth information may be a summary of the discrete bandwidth information. In practical application, each pre-defined bandwidth section can be traversed, and the following similar processing method is used for each bandwidth section, taking any target bandwidth section as an example:

[0041] For a target bandwidth interval, the total number of subscriber clients present within the target bandwidth interval of the available bandwidth can be calculated based on each generated discrete bandwidth information, and candidate available bandwidths can be determined based on the multiple available bandwidths present within the target bandwidth interval.

[0042] Specifically, assuming the target bandwidth interval is 2 Mb / s to 6 Mb / s, the total number of subscriber clients present within the target bandwidth interval of the available bandwidth can be calculated based on the discrete bandwidth information of each subscriber client, and the total number may be the total number obtained by the calculation. For example, if the available bandwidths of two subscriber clients are currently 2.2 Mb / s and 4.5 Mb / s, respectively, both of these available bandwidths are within the target bandwidth interval, and the total number corresponding to the target bandwidth interval may be 2. When determining candidate available bandwidths corresponding to the target bandwidth interval, multiple available bandwidths (e.g., the above 2.2 Mb / s and 4.5 Mb / s) present within the target bandwidth interval can be calculated, and then the candidate available bandwidths can be determined based on the multiple available bandwidths. Typically, the maximum or minimum available bandwidth among the multiple available bandwidths present within the target bandwidth interval can be determined as the candidate available bandwidth. Alternatively, the multiple available bandwidths present within the target bandwidth interval can be weighted and added, and the resulting available bandwidth can be determined as the candidate available bandwidth. That is, the candidate available bandwidth is generated based on the actual available bandwidth of the subscriber client, and can represent the overall situation of the actual available bandwidth of the subscriber client within the target bandwidth section to some extent. In actual applications, the specific method used to determine the candidate available bandwidth can be determined according to actual service needs.

[0043] According to the above method, two parameters, the total number and the candidate available bandwidth, can be generated for the target bandwidth interval, and the combination of these two parameters may be used as the interval information for the target bandwidth interval. Similarly, for other bandwidth intervals, interval information for each bandwidth interval can be obtained, and the set of this interval information may be used as the aggregated bandwidth information obtained by aggregation.

[0044] The aggregated bandwidth information generated in the above manner can represent the overall distribution of available bandwidth of subscriber clients, thereby providing an accurate data basis for later determining the push stream bit rate, and also making the push stream bit rate more consistent with actual service needs.

[0045] In one embodiment, the vector representation method of steps S111 to S115 can be continued to be used to generate aggregated bandwidth information. Specifically, the interval aggregated vector can be obtained by adding the interval discrete vector in each discrete bandwidth information to the interval discrete vector. For example, if the interval discrete vectors of two subscriber clients are V11=[1,0,0,0] and V12=[0,1,0,0] respectively, the interval aggregated vector can be the sum of both, that is, [1,1,0,0].

[0046] The method for determining the candidate available bandwidth for a bandwidth discrete vector involves traversing a plurality of predetermined bandwidth intervals, identifying available bandwidth values ​​corresponding to the target bandwidth interval in the bandwidth discrete vector of each discrete bandwidth information for any target bandwidth interval, and determining the candidate available bandwidth for the target bandwidth interval based on the identified available bandwidth values. The specific method for determining the candidate available bandwidth is not described in detail here, and reference may be made to the method described above. After determining the candidate available bandwidth for each bandwidth interval among the plurality of predetermined bandwidth intervals, a bandwidth aggregate vector including the candidate available bandwidths can be generated. For example, if the bandwidth discrete vectors of two subscriber clients are V21 = [0, 4.5, 0, 0] and V22 = [0, 2, 0, 0], respectively, and the maximum available bandwidth is the candidate available bandwidth, the generated bandwidth aggregate vector may be [0, 4.5, 0, 0].

[0047] Further, for example, in FIG. 2, for two subscriber clients connected to media server A, if the bandwidth discrete vectors in the discrete bandwidth information are [0,0,0,7.5] and [0,3,0,0], respectively, the bandwidth aggregate vector obtained by aggregation may be [0,3,0,7.5].

[0048] In this embodiment, after generating the interval aggregate vector and the bandwidth aggregate vector, the combination of the two can be used as the aggregated bandwidth information obtained by aggregation. By expressing the aggregated bandwidth information in the form of a vector, the aggregated bandwidth information can be made more accurate and intuitive, and subsequent data processing can be facilitated.

[0049] S15: Push the aggregate bandwidth information to the publisher client for determining a push stream bitrate by the publisher client.

[0050] In this embodiment, the media server can obtain aggregated bandwidth information through aggregation and then push the aggregated bandwidth information to the publisher client. The publisher client can analyze the aggregated bandwidth information to determine the current available bandwidth distribution of each subscriber client. For example, in one specific application example, the aggregated bandwidth information received by the publisher client can be represented by the following interval aggregate vector and bandwidth aggregate vector:

[0051]

number

[0052] As a result, within the four pre-defined bandwidth sections, the number of distributed subscriber clients is 2, 4, 8, and 1, respectively, and the candidate available bandwidths for each of the four bandwidth sections are 1.2 Mb / s, 3.5 Mb / s, 7 Mb / s, and 10 Mb / s, respectively.

[0053] In this embodiment, the publisher client can determine the push stream bitrate based on the aggregate bandwidth information according to a preset rule. For example, the publisher client can determine the bandwidth interval with the largest total number of subscriber clients, and then use the candidate available bandwidth corresponding to the bandwidth interval as the push stream bitrate. In this way, encoding the audio-video stream at the push stream bitrate can accommodate the bandwidth required by the majority of subscriber clients.

[0054] In one embodiment, considering that there may be a certain hierarchy of media servers in the SFU architecture, for example, in FIG. 4, the media servers can be divided into two tiers. The first-tier media server is connected to the subscriber client, and this media server generates discrete bandwidth information for the subscriber client according to the description of steps S11 and S13, and aggregates the discrete bandwidth information to obtain aggregated bandwidth information. The second-tier media server can receive the aggregated bandwidth information sent by the first-tier media server, then re-aggregate it, and push the final aggregated result to the publisher client. Applying the technical solution of the present disclosure to a multi-tier architecture can better suit actual business scenarios and facilitate the implementation and extension of the technical solution of the present disclosure.

[0055] Specifically, in one embodiment, the media server to which the subscriber client is connected can obtain aggregated bandwidth information by aggregating it, and then upload the aggregated bandwidth information to the downstream media server, so that the downstream media server can re-aggregate the received aggregated bandwidth information, and then push the re-aggregated aggregated bandwidth information to the publisher client.

[0056] In addition, the media server is located at the second tier, and when the media server receives first aggregated bandwidth information uploaded by multiple upper media servers, the media server re-aggregates the multiple first aggregated bandwidth information into second aggregated bandwidth information and pushes the second aggregated bandwidth information to the publisher client, thereby enabling the publisher client to determine a push stream bitrate based on the second aggregated bandwidth information and encode the audio-video stream at the push stream bitrate.

[0057] The second-tier media server may re-aggregate the first aggregated bandwidth information using a method similar to that used to generate the first aggregated bandwidth information. Specifically, the second-tier media server may still traverse each bandwidth interval, and for any target bandwidth interval, calculate the sum of the total number of subscriber clients in the target bandwidth interval based on the plurality of first aggregated bandwidth information, and determine a second candidate available bandwidth after re-aggregation based on the plurality of first candidate available bandwidths corresponding to the target bandwidth interval. The logic for determining the second candidate available bandwidth after re-aggregation may be the same as the logic for determining the first candidate available bandwidth in the first aggregated bandwidth information.

[0058] In this embodiment, after obtaining the sum of the above total numbers and the second candidate available bandwidth, the sum of the total numbers and the second candidate available bandwidth can be used as the re-aggregated section information of the target bandwidth section, and after generating re-aggregated section information corresponding to each bandwidth section among a plurality of predetermined bandwidth sections, the collection of generated re-aggregated section information can be used as the second aggregated bandwidth information.

[0059] 4 as an example, the first tier includes two media servers, and the two media servers can upload their respective first aggregated bandwidth information to the media server in the second tier. The two first aggregated bandwidth information can be respectively expressed in vector form as follows:

[0060]

number

[0061]

number

[0062] By re-aggregating these two pieces of first aggregated bandwidth information, the following second aggregated bandwidth information can be obtained.

[0063]

number

[0064] The second tier media server can push the second aggregated bandwidth information to the publisher client, and the publisher client can determine the push stream bitrate and encode the audio-video stream, as described in step S15.

[0065] In one embodiment, there may be a third or further tier media server after the second tier media server, and they can be aggregated layer by layer in a similar manner. Specifically, the media server can upload the second aggregated bandwidth information to the lower media servers, so that the lower media servers can re-aggregate the second aggregated bandwidth information received by each of the lower media servers, and then push the re-aggregated third aggregated bandwidth information to the publisher client.

[0066] Thus, according to the technical solutions of one or more embodiments of the present disclosure, the media server can obtain available bandwidth information for each subscriber client and then generate corresponding discrete bandwidth information for each subscriber client. The discrete bandwidth information can record the actual available bandwidth of the subscriber client and the bandwidth interval in which the available bandwidth exists. The media server can then obtain aggregated bandwidth information by aggregating the discrete bandwidth information for each subscriber client. In the aggregated bandwidth information, for each bandwidth interval, the total number of subscriber clients whose available bandwidth exists in that bandwidth interval can be counted, and a candidate available bandwidth can be set for each bandwidth interval. In this way, when the media server pushes the aggregated bandwidth information to the publisher client, the publisher client can grasp the current distribution of available bandwidth for each subscriber client. Based on the distribution of available bandwidth for the subscriber clients, the publisher client can generate a push stream bitrate that matches the available bandwidth of the majority of subscriber clients. In this way, after the audio / video stream encoded at the push stream bit rate is pushed to each subscriber client, the communication quality of the majority of subscriber clients can be ensured, further improving the overall communication quality of the multi-party communication process.

[0067] Referring to FIG. 5, one embodiment of the present disclosure includes: a discrete bandwidth information generating unit 110 for generating, based on available bandwidth information of a plurality of subscriber clients, discrete bandwidth information for the subscriber clients, the discrete bandwidth information representing available bandwidths allocated to the subscriber clients and bandwidth intervals in which the available bandwidths exist; an aggregation unit 130 for aggregating a plurality of said discrete bandwidth information into aggregate bandwidth information representing a candidate available bandwidth for each bandwidth interval and a total number of subscriber clients in each bandwidth interval; a push unit 150 for pushing the aggregated bandwidth information to publisher clients for determining a push stream bit rate by the publisher clients; The specific processing logic of each functional module may refer to the description of the above method embodiments, and detailed description will be omitted here.

[0068] Referring to FIG. 6, the present disclosure further provides a media server including a memory for storing a computer program and a processor, the computer program implementing the above bitrate control method when executed by the processor.

[0069] The present disclosure further provides a bitrate control method that is applicable to the above publisher client, and which, with reference to FIG. 7, may include the following steps:

[0070] S21: Receive aggregate bandwidth information from a media server, representing the candidate available bandwidth for each bandwidth interval and the total number of subscriber clients in each said bandwidth interval.

[0071] S23: Determine a push stream bit rate based on the aggregate bandwidth information, and encode the audio-video stream at the push stream bit rate.

[0072] S25: The encoded audio-video stream is pushed to each subscriber client.

[0073] In this embodiment, the publisher client can receive aggregated bandwidth information pushed by the media server. For specific ways of generating the aggregated bandwidth information, please refer to the description of the above method embodiment. By analyzing the aggregated bandwidth information, the publisher client can grasp the available bandwidth distribution status of the subscriber clients participating in the current multi-party communication. In this way, according to the preset rules, the publisher client can determine the push stream bitrate based on the available bandwidth distribution status.

[0074] Specifically, in one embodiment, the publisher client can use the maximum or minimum candidate available bandwidth in the aggregated bandwidth information as the push stream bitrate to meet the needs of various application scenarios. Furthermore, the publisher client can determine the target bandwidth section with the largest total number of subscriber clients in the aggregated bandwidth information, and use the candidate available bandwidth corresponding to the target bandwidth section as the push stream bitrate. In this way, the push stream bitrate can be matched to the network environment of the majority of subscriber clients.

[0075] Of course, those skilled in the art can conceive of more ways to determine the push stream bit rate after understanding the spirit and concept of the present disclosure, but any push stream bit rate determined based on aggregate bandwidth information should fall within the scope of protection of the present disclosure.

[0076] In one embodiment, considering that the media server originally has a default encoding rule, for example, if a 720P audio-video stream can be encoded as a 3 Mb / s stream, in order to be compatible with the conventional encoding rule, the present disclosure can generate a multi-bitrate audio-video stream. Specifically, if a preset push stream bitrate exists, the publisher client can still use the above steps to determine a push stream bitrate that matches the network environment of the subscriber client, and then encode the audio-video stream at the determined push stream bitrate and the preset push stream bitrate, respectively, to generate a multi-bitrate audio-video stream, and the multi-bitrate audio-video stream media server can push the media server to each subscriber client. The purpose of this processing is to provide a network optimization function for the subscriber client in some scenarios. When the function is turned on, the subscriber client can load an audio-video stream that is encoded at a determined push stream bitrate, and when the function is turned off, the subscriber client can load an audio-video stream that is encoded at a pre-set push stream bitrate, thereby making the technical solution of the present disclosure compatible with conventional SFU architecture.

[0077] One embodiment of the present disclosure is an information receiving unit for receiving aggregate bandwidth information from the media server, the aggregate bandwidth information representing the candidate available bandwidth for each bandwidth interval and the total number of subscriber clients in each said bandwidth interval; a bitrate determination unit for determining a push stream bitrate based on the aggregate bandwidth information and encoding an audio-video stream at the push stream bitrate; and an audio-video stream push unit for pushing the encoded audio-video stream to each subscriber client.

[0078] The specific processing logic of each functional module may refer to the description of the above method embodiments, and detailed description will be omitted here.

[0079] One embodiment of the present disclosure further provides a terminal device including a memory for recording a computer program and a processor, the computer program, when executed by the processor, realizing a bitrate control method applied to the publisher client.

[0080] Each unit described in the above embodiments may be specifically implemented by a computer chip or a product having a certain function. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a mobile phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game control panel, a tablet computer, a wearable device, or any combination of these devices.

[0081] For ease of explanation, the above devices will be described by dividing them into various units according to their functions. Of course, when implementing the present disclosure, the functions of each unit may be realized by the same or multiple pieces of software and / or hardware.

[0082] The present disclosure further provides a computer-readable storage medium, which is used to store a computer program, which, when executed by a processor, implements the above bitrate control method.

[0083] The processor may be a central processing unit (CPU), other general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware assembly, or other chip, or a combination of these various chips.

[0084] The memory can be used as a non-transitory computer-readable storage medium to store non-transitory software programs, non-transitory computer-executable programs and modules, such as program instructions / modules corresponding to the methods of the embodiments of the present disclosure. The processor executes the non-transitory software programs, instructions and modules stored in the memory to perform various functional applications and data processing, i.e., to realize the methods of the above-described method embodiments.

[0085] The memory may include a program storage area and a data storage area. The program storage area may store an operating system or an application program required for at least one function, and the data storage area may store data created by the processor. The memory may also include high-speed random access memory or non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory located remotely from the processor, and the remote memory may be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0086] As will be understood by those skilled in the art, all or part of the processes of the above-mentioned exemplary methods may be realized by instructing relevant hardware using a computer program, and the program may be stored in a computer-readable storage medium, which, when executed, may include the processes described in the above-mentioned exemplary methods. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), a solid-state drive (SSD), etc., or the storage medium may include a combination of the above types of memory.

[0087] Each embodiment of the present disclosure will be described step by step, and the same and similar parts of each embodiment may be referred to, and the emphasis in each embodiment will be on the differences from other embodiments. In particular, the device, apparatus, and storage medium embodiments are basically similar to the method embodiments, so they will be briefly described, and the relevant parts may be referred to in part of the description of the method embodiments.

[0088] The above are merely examples of the present disclosure and are not intended to limit the present disclosure. Those skilled in the art may have various modifications and variations to the present disclosure. Any modifications, equivalent replacements, improvements, etc. made without departing from the spirit and principles of the present disclosure shall be included within the scope of the claims of the present disclosure.

[0089] Although the embodiments of the present disclosure have been described with reference to the drawings, various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the present disclosure, and all such modifications and variations are within the scope limited by the appended claims.

Claims

1. 1. A bit rate control method, comprising: generating discrete bandwidth information for a plurality of subscriber clients based on available bandwidth information of the subscriber clients, the discrete bandwidth information representing available bandwidths allocated to the subscriber clients and bandwidth intervals in which the available bandwidths exist; aggregating a plurality of said discrete bandwidth information into aggregate bandwidth information representing candidate available bandwidth for each bandwidth interval and a total number of subscriber clients in each bandwidth interval; and pushing the aggregate bandwidth information to a publisher client for determining a push stream bit rate by the publisher client.

2. aggregating a plurality of the discrete bandwidth information into aggregate bandwidth information, For any target bandwidth section among a plurality of preset bandwidth sections, a step of counting the total number of subscriber clients whose available bandwidth exists in the target bandwidth section based on a plurality of pieces of discrete bandwidth information, and determining candidate available bandwidths based on a plurality of available bandwidths existing in the target bandwidth section, wherein section information of the target bandwidth section includes the total number and the candidate available bandwidths; aggregating a set of interval information corresponding to each bandwidth interval among the plurality of preset bandwidth intervals to form the aggregated bandwidth information.

3. The step of determining candidate available bandwidths comprises: a step of determining a maximum available bandwidth or a minimum available bandwidth among a plurality of available bandwidths present in the target bandwidth section as a candidate available bandwidth; or 3. The method according to claim 2, further comprising the step of performing a weighted sum of a plurality of available bandwidths present within the target bandwidth interval, and determining the resulting available bandwidth as a candidate available bandwidth.

4. generating discrete bandwidth information for the subscriber client, generating an interval discrete vector and a bandwidth discrete vector, each of whose vector elements corresponds one-to-one to the predetermined plurality of bandwidth intervals, based on the predetermined plurality of bandwidth intervals; determining a bandwidth interval in which the available bandwidth allocated to the subscriber client exists, and setting a mark value in the interval discrete vector to a vector element corresponding to the existing bandwidth interval, and setting an available bandwidth value in the bandwidth discrete vector to a vector element corresponding to the existing bandwidth interval; The method of claim 1 , further comprising the step of: determining a combination of the allocated interval discrete vector and the allocated bandwidth discrete vector as the discrete bandwidth information of the subscriber client.

5. aggregating a plurality of the discrete bandwidth information into aggregate bandwidth information, Adding interval discrete vectors in each of the discrete bandwidth information to obtain an interval aggregate vector; traversing the plurality of predetermined bandwidth sections, and for any target bandwidth section among them, identifying each available bandwidth value corresponding to the target bandwidth section in a bandwidth discrete vector of each of the discrete bandwidth information, and determining candidate available bandwidths for the target bandwidth section based on each identified available bandwidth value; determining a corresponding candidate available bandwidth for each bandwidth interval among the plurality of predetermined bandwidth intervals, and then generating a bandwidth aggregation vector including each candidate available bandwidth; The method of claim 4 , further comprising: aggregating a combination of the interval aggregate vector and the bandwidth aggregate vector to obtain the aggregated bandwidth information.

6. After aggregating the plurality of discrete bandwidth information into aggregated bandwidth information, The method of any one of claims 1 to 5, further comprising a step of re-aggregating the aggregated bandwidth information received by the lower media server by uploading the aggregated bandwidth information to the lower media server, and pushing the aggregated bandwidth information obtained by the re-aggregation to a publisher client.

7. The method according to any one of claims 1 to 6, further comprising the step of, when receiving first aggregated bandwidth information uploaded by multiple upstream media servers, re-aggregating the multiple first aggregated bandwidth information into second aggregated bandwidth information, and pushing the second aggregated bandwidth information to a publisher client, so that the publisher client determines a push stream bitrate based on the second aggregated bandwidth information, and encodes the audio-video stream at the push stream bitrate.

8. The step of re-aggregating the plurality of first aggregated bandwidth information into second aggregated bandwidth information includes: For any target bandwidth interval among a plurality of preset bandwidth intervals, calculating the sum of the total number of subscriber clients in the target bandwidth interval based on the plurality of first aggregated bandwidth information, and determining a second candidate available bandwidth after re-aggregation based on the plurality of first candidate available bandwidths corresponding to the target bandwidth interval; The method of claim 7, further comprising the steps of: generating re-aggregated section information corresponding to each bandwidth section among the plurality of predetermined bandwidth sections using the sum of the total number and the second candidate available bandwidth as section information after re-aggregating the target bandwidth section; and then using the set of generated re-aggregated section information as the second aggregated bandwidth information.

9. 8. The method of claim 7, further comprising a step of re-aggregating each second aggregated bandwidth information received by the lower media server by uploading the second aggregated bandwidth information to the lower media server, and pushing the third aggregated bandwidth information obtained by the re-aggregation to the publisher client.

10. A bit rate control device, a discrete bandwidth information generating unit configured to generate discrete bandwidth information for a plurality of subscriber clients based on available bandwidth information of the subscriber clients, the discrete bandwidth information representing available bandwidth allocated to the subscriber clients and a bandwidth interval in which the available bandwidth exists; an aggregation unit configured to aggregate a plurality of said discrete bandwidth information into aggregate bandwidth information representing a candidate available bandwidth for each bandwidth interval and a total number of subscriber clients in each bandwidth interval; a push unit configured to push the aggregated bandwidth information to a publisher client for determining a push stream bit rate by the publisher client.

11. A media server comprising: a memory for storing a computer program; and a processor, said computer program implementing the method of any one of claims 1 to 9 when executed by said processor.

12. 1. A bit rate control method, comprising: receiving aggregate bandwidth information from a media server representing candidate available bandwidth for each bandwidth interval and a total number of subscriber clients in each said bandwidth interval; determining a push stream bitrate based on the aggregate bandwidth information; encoding the audio-video stream at the push stream bit rate; and pushing the encoded audio-video stream to each subscriber client.

13. determining a push stream bitrate based on the aggregate bandwidth information, determining a maximum candidate available bandwidth or a minimum candidate available bandwidth in the aggregated bandwidth information as a push stream bitrate; or 13. The method of claim 12, further comprising determining a target bandwidth interval with the highest total number of subscriber clients in the aggregated bandwidth information, and setting a candidate available bandwidth corresponding to the target bandwidth interval as a push stream bit rate.

14. 14. The method of claim 12 or 13, further comprising the step of, if a preset push stream bit rate exists, encoding the audio-video stream at the determined push stream bit rate and the preset push stream bit rate, respectively, to form a multi-bitrate audio-video stream, and pushing the multi-bitrate audio-video stream to each subscriber client.

15. A bit rate control device, an information receiving unit for receiving aggregate bandwidth information from the media server, the aggregate bandwidth information representing the candidate available bandwidth for each bandwidth interval and the total number of subscriber clients in each said bandwidth interval; a bitrate determination unit for determining a push stream bitrate based on the aggregate bandwidth information and encoding an audio-video stream at the push stream bitrate; an audio-video stream push unit for pushing the encoded audio-video stream to each subscriber client.

16. A terminal device comprising a memory for storing a computer program and a processor, the computer program implementing the method of any one of claims 12 to 14 when executed by the processor.

17. A computer-readable storage medium, the computer-readable storage medium being used to store a computer program, the computer program implementing the method of any one of claims 1 to 9 or the method of any one of claims 12 to 14 when executed by a processor.

Images