Streaming media video coding method and device, equipment, storage medium and product

Abstract

The embodiment of the invention provides a streaming media video coding method and device, equipment, a storage medium and a product. According to the technical scheme provided by the embodiment of the invention, the coding target code rate is determined according to the sampling bandwidth and the maximum target code rate of the current coding resolution, and the maximum instantaneous code rate is determined according to the coding target code rate, the maximum target code rate, the sampling bandwidth and the bandwidth fluctuation information; and determining the size of a coding buffer area according to the maximum instantaneous code rate and the preset constraint delay, and carrying out streaming media video coding based on a video buffer area checker according to the coding target code rate, the maximum instantaneous code rate and the size of the coding buffer area, thereby accurately estimating the network condition and the network fluctuation condition. The coding target code rate and the size of the coding buffer area are adaptively adjusted, and while the delay constraint is satisfied, the overall picture quality and the transmission stability of the video stream are effectively improved, and the video coding quality is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of video encoding technology, and in particular to a streaming media video encoding method, apparatus, device, storage medium and product. Background Art

[0002] With the rapid development of Internet technology, video streaming services have become one of the mainstream applications and are widely used in online live broadcasts, video conferencing, cloud games and other scenarios. These scenarios have high requirements for video transmission quality and real-time performance. Especially when the network conditions are unstable, it is particularly important to maintain the smoothness and image quality of the video. In practical applications, network bandwidth fluctuation is one of the main issues affecting the quality of video transmission. Bandwidth fluctuations may be caused by factors such as network congestion, changes in the wireless network environment, or an increase in the number of users. In the case of bandwidth fluctuations, video transmission needs to dynamically adjust the bit rate to avoid freezes or image quality degradation to ensure a good user experience.

[0003] At present, the bit rate control method is generally based on the adaptive bit rate (ABR) technology. The adaptive bit rate technology dynamically selects the appropriate bit rate level to transmit the video stream by detecting the current network bandwidth and the cache status of the client. This technology performs well in a stable network environment and can effectively adjust the bit rate of the video stream. However, in scenarios with large network fluctuations and strict latency requirements, the existing adaptive bit rate technology has problems such as adjustment lag and slow response, resulting in reduced video stream quality or unstable transmission, and poor video encoding quality. Summary of the invention

[0004] The embodiments of the present application provide a streaming media video encoding method, apparatus, device, storage medium and product to solve the technical problem in the related art that the adaptive bit rate technology has adjustment lag and slow response in scenarios with large network fluctuations and strict delay requirements, resulting in reduced video stream quality or unstable transmission and poor video encoding quality. The video encoding bit rate can be adjusted in time to ensure the video stream quality and transmission stability, thereby improving the video encoding quality.

[0005] In a first aspect, an embodiment of the present application provides a streaming media video encoding method for determining a sampling bandwidth and bandwidth fluctuation information of a transmitted video stream;

[0006] Determine a coding target bit rate according to the sampling bandwidth and a maximum target bit rate of a current coding resolution, and determine a maximum instantaneous bit rate according to the coding target bit rate, the maximum target bit rate, the sampling bandwidth, and the bandwidth fluctuation information;

[0007] Determining a coding buffer size according to the maximum instantaneous bit rate and a preset constrained delay;

[0008] Streaming media video encoding based on a video buffer verifier is performed according to the encoding target bit rate, the maximum instantaneous bit rate and the encoding buffer size.

[0009] In a second aspect, an embodiment of the present application provides a streaming media video encoding device, including a bandwidth prediction module, a bit rate determination module, a buffer determination module and a video encoding module, wherein:

[0010] The bandwidth prediction module is configured to determine the sampling bandwidth and bandwidth fluctuation information of the transmitted video stream;

[0011] The bit rate determination module is configured to determine a coding target bit rate according to the sampling bandwidth and a maximum target bit rate of a current coding resolution, and determine a maximum instantaneous bit rate according to the coding target bit rate, the maximum target bit rate, the sampling bandwidth and the bandwidth fluctuation information;

[0012] The buffer determination module is configured to determine the encoding buffer size according to the maximum instantaneous bit rate and a preset constrained delay;

[0013] The video encoding module is configured to perform streaming media video encoding based on a video buffer verifier according to the encoding target bit rate, the maximum instantaneous bit rate and the encoding buffer size.

[0014] In a third aspect, an embodiment of the present application provides a streaming media video encoding device, including: a memory and one or more processors;

[0015] The memory is used to store one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the streaming media video encoding method as described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a non-volatile storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the streaming media video encoding method as described in the first aspect when executed by a computer processor.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor of a device reads and executes the computer program from the computer-readable storage medium, so that the device performs the streaming media video encoding method as described in the first aspect.

[0019] The embodiment of the present application determines the encoding target bit rate according to the maximum target bit rate of the sampling bandwidth and the current encoding resolution, determines the maximum instantaneous bit rate according to the encoding target bit rate, the maximum target bit rate, the sampling bandwidth and the bandwidth fluctuation information, determines the encoding buffer size according to the maximum instantaneous bit rate and the preset constraint delay, and performs streaming media video encoding based on a video buffer verifier according to the encoding target bit rate, the maximum instantaneous bit rate and the encoding buffer size, accurately estimates the network status and network fluctuation, and adaptively adjusts the encoding target bit rate and the encoding buffer size, while meeting the delay constraint, effectively improving the overall picture quality and transmission stability of the video stream, and improving the video encoding quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of a streaming media video encoding method provided by an embodiment of the present application;

[0021] Figure 2 is a flowchart of another streaming media video encoding method provided by an embodiment of the present application;

[0022] Figure 3 This is a schematic diagram of a sampling bandwidth determination process provided by an embodiment of the present application;

[0023] Figure 4 This is a schematic diagram of a bandwidth fluctuation information determination process provided by an embodiment of the present application;

[0024] Figure 5 It is a structural diagram of a streaming media video encoding device provided in an embodiment of the present application;

[0025] Figure 6 It is a structural diagram of a streaming media video encoding device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The above process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The above process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0027] The streaming media video encoding method provided in this application can be applied to video encoding in video live broadcast scenarios. It aims to adaptively adjust the encoding target bit rate and encoding buffer size according to the estimation of network conditions and network fluctuations, effectively improve the overall picture quality and transmission stability of the video stream while meeting the delay constraints, improve the video encoding quality, and provide bit rate adaptation strategies with different delay requirements for soft real-time live broadcast and real-time microphone connection. While meeting real-time and smoothness, it improves the encoded video quality and its stability, and improves live broadcast interaction and viewing experience.

[0028] Among the existing video encoding schemes, the adaptive bitrate (ABR) technology and bandwidth estimation are mainly used. The adaptive bitrate technology dynamically selects the appropriate bitrate level to transmit the video stream by detecting the current network bandwidth and the cache status of the client. This method performs well in a stable network environment and can effectively adjust the bitrate of the video stream. However, in scenarios with large network fluctuations and strict delay requirements, the existing adaptive bitrate technology has problems such as adjustment lag and slow response, resulting in reduced video stream quality or unstable transmission. In addition, the video encoder plays a key role in bitrate control. Commonly used video encoders use the video buffer verifier (VBV) mechanism to limit the bitrate of the video stream to prevent buffer overflow or transmission delay during video transmission. However, the current encoder adjustment mostly uses fixed VBV parameters and fails to dynamically adjust according to the actual network conditions, which easily leads to reduced transmission quality when the network fluctuates. At the same time, the bandwidth estimation of video encoding based on the video buffer verifier is separated from the encoder adjustment, that is, the existing bandwidth estimation technology is processed separately from the encoder's VBV mechanism, and the result of bandwidth prediction cannot be directly used to adjust the encoder's parameters. The encoder cannot adjust the output bit rate in time according to bandwidth fluctuations, and the response speed to bandwidth fluctuations is slow. Especially when the network environment fluctuates violently, the bit rate adjustment lags behind, and the real-time and smoothness of streaming media transmission cannot be guaranteed. As a result, when the network conditions change, the video streaming transmission cannot be optimized in real time, and the video quality and transmission stability are affected. At the same time, most of the existing encoder VBV mechanisms use static maximum instantaneous bit rate (max bitrate) and encoding buffer size (buffersize), which cannot be dynamically adjusted according to the real-time network conditions. This means that when the bandwidth is good, the video buffer verifier fails to fully utilize the network bandwidth to improve the video quality, and when the bandwidth is poor, the video buffer verifier cannot reasonably limit the video bit rate, resulting in network transmission jams or delays. Based on this, a streaming media video encoding method is provided in an embodiment of the present application to solve the technical problems that the existing adaptive bit rate technology has adjustment lag and slow response in scenarios with large network fluctuations and strict delay requirements, resulting in reduced video stream quality or unstable transmission and poor video encoding quality.

[0029] Figure 1 A flowchart of a streaming media video encoding method provided in an embodiment of the present application is given. The streaming media video encoding method provided in an embodiment of the present application can be executed by a streaming media video encoding device, which can be implemented by hardware and / or software and integrated in a streaming media video encoding device.

[0030] The following description is made by taking a streaming media video encoding device executing a streaming media video encoding method as an example. Figure 1 , the streaming media video encoding method comprises:

[0031] S110: Determine the sampling bandwidth and bandwidth fluctuation information of the transmitted video stream.

[0032] Exemplarily, bandwidth sampling is performed on a data transmission link that transmits a video stream (for example, a host uploads a video stream to a media server in a live video broadcast), and the sampling bandwidth and bandwidth fluctuation information are determined based on the bandwidth sampling result.

[0033] In one embodiment, the minimum value of the ratio of the sending rate of the transmitted video stream and the sending window size to the average packet feedback delay can be multiplied by the difference between the first preset parameter and the packet loss rate, and the product result can be used as the sampling bandwidth of the transmitted video stream. The above sampling bandwidth and the historically calculated sampling bandwidth can also be smoothed and filtered to obtain the final sampling bandwidth.

[0034] The bandwidth fluctuation information provided by this solution can be used to indicate the bandwidth fluctuation of the data transmission link within a set time period. Optionally, the bandwidth fluctuation information can be determined based on the difference between the sampling bandwidth and the average sampling bandwidth within the set time period, or can be determined based on the difference between the current sampling bandwidth and the previous sampling bandwidth, or can be determined based on the variance, standard deviation, etc. of the sampling bandwidth within the set time period. In one embodiment, the difference (absolute value) between the average bandwidth of the transmitted video stream and the sampling bandwidth can be used as the bandwidth fluctuation information, and the above-mentioned bandwidth fluctuation information and the bandwidth fluctuation information calculated historically can be smoothed and filtered to obtain the final bandwidth fluctuation information.

[0035] S120: Determine a coding target bit rate according to the sampling bandwidth and the maximum target bit rate of the current coding resolution, and determine a maximum instantaneous bit rate according to the coding target bit rate, the maximum target bit rate, the sampling bandwidth, and the bandwidth fluctuation information.

[0036] Exemplarily, the maximum target bit rate of the current encoding resolution is determined, and the encoding target bit rate is determined according to the above-determined sampling bandwidth and the maximum target bit rate. The encoding target bit rate can be understood as the average encoding bit rate expected to be achieved in streaming video encoding. The encoder will try to keep the output bit rate close to the encoding target bit rate to optimize transmission efficiency and compatibility while ensuring video quality.

[0037] Optionally, the minimum value between the sampling bandwidth and the maximum target bit rate may be determined as the encoding target bit rate, or the average value or weighted sum of the sampling bandwidth and the maximum target bit rate may be determined as the encoding target bit rate. In one embodiment, different maximum target bit rates may be configured in advance for different encoding resolutions, and the corresponding relationship between the encoding resolution and the maximum target bit rate may be determined. Based on the corresponding relationship, the maximum target bit rate may be determined according to the encoding resolution currently used for streaming media video encoding.

[0038] In one embodiment, after determining the target encoding rate, the maximum instantaneous rate can be determined according to the target encoding rate, the maximum target rate, the sampling bandwidth, and the bandwidth fluctuation information. The maximum instantaneous rate can be understood as the highest instantaneous rate that the encoder is allowed to reach during the encoding process, which can effectively reduce the situation where the encoder allocates too many bits on some frames, thereby causing insufficient bits in other frames or buffer overflow.

[0039] Optionally, the maximum value among the encoding target bit rate, the maximum target bit rate (or the product of the maximum target bit rate and a preset floating ratio), the difference between the sampling bandwidth and the bandwidth fluctuation information can be used as the maximum instantaneous bit rate; the minimum value among the maximum target bit rate (or the product of the maximum target bit rate and a preset floating ratio) and the difference between the sampling bandwidth and the bandwidth fluctuation information, and the maximum value among the encoding target bit rates can be used as the maximum instantaneous bit rate; or the weighted sum or average value of the encoding target bit rate, the maximum target bit rate (or the product of the maximum target bit rate and a preset floating ratio) and the difference between the sampling bandwidth and the bandwidth fluctuation information can be used as the maximum instantaneous bit rate.

[0040] S130: Determine the encoding buffer size according to the maximum instantaneous bit rate and the preset constrained delay.

[0041] Exemplarily, after determining the maximum instantaneous bit rate, the encoding buffer size may be determined according to the maximum instantaneous bit rate and the preset constrained delay. For example, the product of the maximum instantaneous bit rate and the preset constrained delay may be used as the encoding buffer size, or a preset time coefficient may be determined according to the preset constrained delay, and the product of the preset time coefficient and the maximum instantaneous bit rate may be used as the encoding buffer size.

[0042] Among them, the preset constrained delay can be understood as a constraint value for data transmission delay (for example, the delay from the data sender to the data receiver). Optionally, the preset constrained delay provided by the present solution can be determined based on a preset scenario. For example, different preset constrained delays can be configured according to the application scenario of streaming video encoding. For example, in a real-time call scenario, the preset constrained delay can be set between 200 and 500ms. In a live video broadcast scenario, the preset constrained delay can be set between 2000 and 10000ms. Different preset constrained delays are used according to different scenarios to ensure that streaming media transmission meets the delay requirements of various scenarios.

[0043] S140: Perform streaming media video encoding based on a video buffer verifier according to the encoding target bit rate, the maximum instantaneous bit rate and the encoding buffer size.

[0044] Exemplarily, streaming media video encoding based on a video buffer verifier is performed according to the above-determined encoding target bit rate, maximum instantaneous bit rate and encoding buffer size. Wherein, when performing streaming media video encoding based on a video buffer verifier, the buffer size of the encoder is configured to be the encoding buffer size determined above, the instantaneous maximum bit rate allowed during the encoding process is limited to the maximum instantaneous bit rate, and the average encoding bit rate is controlled within the encoding target bit rate or the error range corresponding to the encoding target bit rate. This solution achieves the effect of real-time streaming media providing higher and more stable quality video under different delay constraints through bandwidth estimation and encoding control, adapts to the bit rate changes caused by changes in the complexity of video content on the basis of ensuring delay and smoothness, improves and stabilizes video quality, and brings better user live broadcast experience.

[0045] In the above, the encoding target bit rate is determined according to the maximum target bit rate of the sampling bandwidth and the current encoding resolution, and the maximum instantaneous bit rate is determined according to the encoding target bit rate, the maximum target bit rate, the sampling bandwidth and the bandwidth fluctuation information, and the encoding buffer size is determined according to the maximum instantaneous bit rate and the preset constraint delay, and the streaming video encoding based on the video buffer verifier is performed according to the encoding target bit rate, the maximum instantaneous bit rate and the encoding buffer size, the network status and network fluctuation are accurately estimated, and the encoding target bit rate and the encoding buffer size are adaptively adjusted. While meeting the delay constraint, the overall picture quality and transmission stability of the video stream are effectively improved, and the video encoding quality is improved.

[0046] Based on the above embodiments, Figure 2 A flowchart of another streaming media video encoding method provided in an embodiment of the present application is given, and the streaming media video encoding method is a specific embodiment of the above streaming media video encoding method. Figure 2 , the streaming media video encoding method comprises:

[0047] S210: Determine the sending rate, sending window size, average packet feedback delay and packet loss rate of the transmitted video stream.

[0048] S220: Determine a sampling bandwidth for transmitting the video stream according to the sending rate, the sending window size, the average packet feedback delay, and the packet loss rate.

[0049] Exemplarily, the sending rate, sending window size, average packet feedback delay and packet loss rate of the transmitted video stream are determined. Taking the BBR (Bottleneck Bandwidth and RTT, bottleneck bandwidth and round-trip delay congestion control algorithm) congestion control strategy to implement video stream transmission control as an example, the sending rate can be used to control the amount of data sent per unit time, the sending window size can be used to control the total amount of packets sent, the average packet feedback delay can be the observed average feedback delay of the data packets, and the packet loss rate can be the proportion of data packets lost within the window time.

[0050] In one embodiment, the sampling bandwidth of the video stream transmission may be determined according to the sending rate, the sending window size, the average packet feedback delay, and the packet loss rate. For example, the minimum value of the ratio of the sending rate and the sending window size to the average packet feedback delay may be multiplied by the difference between the first preset parameter and the packet loss rate, and the product result is used as the sampling bandwidth.

[0051] In one embodiment, the sampling bandwidth may be smoothed by filtering (eg, exponentially weighted average (EWMA) smoothing filtering). Based on this, Figure 3 As shown in a schematic diagram of a sampling bandwidth determination process provided, the streaming media video encoding method provided by this solution determines the sampling bandwidth of the transmitted video stream according to the sending rate, the sending window size, the average packet feedback delay and the packet loss rate, which may include:

[0052] S221: Determine a preliminary sampling bandwidth according to the sending rate, the sending window size, the average packet feedback delay and the packet loss rate.

[0053] S222: Performing smoothing filtering on the preliminary sampling bandwidth according to a weighting factor to obtain a sampling bandwidth for the transmitted video stream, wherein the weighting factor is determined according to a baseline filtering strength, a preset bandwidth sampling period, and a preset constraint delay.

[0054] Exemplarily, the preliminary sampling bandwidth is determined according to the sending rate, the sending window size, the average packet feedback delay and the packet loss rate, for example, the minimum value of the ratio of the sending rate and the sending window size to the average packet feedback delay is multiplied by the difference between the first preset parameter and the packet loss rate, and the product result is used as the preliminary sampling bandwidth. Optionally, the preliminary sampling bandwidth can be determined by the following formula:

[0055]

[0056] Among them, i is the data sampling sequence number, PacingRate i is the sending rate, Cwnd i is the send window size, For LossRate i is the packet loss rate.

[0057] In one embodiment, after the preliminary sampling bandwidth is determined, the preliminary sampling bandwidth may be smoothed and filtered according to the weighting factor and the sampling bandwidth determined last time to obtain the sampling bandwidth of the transmitted video stream. For example, the sampling bandwidth may be determined by the following formula:

[0058]

[0059] Among them, B i is the initial sampling bandwidth, is the sampling bandwidth determined last time, Alpha i The weighting factor provided by this solution is determined according to the baseline filtering strength, the preset bandwidth sampling period and the preset constraint delay. For example, the weighting factor can be determined by the following formula:

[0060]

[0061] Wherein, σ is the baseline filter strength, and the value range of σ can be between 0 and 0.5, for example, σ = 0.1, T i is the preset bandwidth sampling period, and L is the preset constraint delay. The initial sampling bandwidth is determined according to the baseline filtering strength, the preset bandwidth sampling period, and the preset constraint delay, and the initial sampling bandwidth is smoothed according to the weighting factor to obtain the sampling bandwidth, which effectively improves the bandwidth response followability under different delay requirements, effectively improves the overall picture quality and transmission stability of the video stream, and improves the video encoding quality while meeting the delay constraint.

[0062] S230: Determine bandwidth fluctuation information.

[0063] Exemplarily, after determining the sampling bandwidth of the transmitted video stream, the bandwidth fluctuation information of the transmitted video stream may be determined based on the sampling bandwidth and the previously determined sampling bandwidth. For example, the average bandwidth may be determined based on a plurality of previously determined sampling bandwidths within a preset time length, and the difference between the currently determined sampling bandwidth and the average bandwidth is used as the bandwidth fluctuation information.

[0064] Optionally, the bandwidth fluctuation information may be smoothed by filtering (eg, exponentially weighted smoothing filtering). Based on this, Figure 4 As shown in a schematic diagram of a bandwidth fluctuation information determination process, the streaming media video encoding method provided by the present solution determines bandwidth fluctuation information, including:

[0065] S231: Determine an average bandwidth of the transmitted video stream, and determine preliminary bandwidth fluctuation information according to the sampling bandwidth and the average bandwidth.

[0066] S232: Perform smoothing filtering on the preliminary bandwidth fluctuation information according to a weighting factor to obtain bandwidth fluctuation information, wherein the weighting factor is determined according to a baseline filtering strength, a preset bandwidth sampling period, and a preset constraint delay.

[0067] Exemplarily, the average bandwidth of the transmitted video stream is determined. The average bandwidth can be determined based on the average value of multiple previously determined sampling bandwidths, or can be determined based on the currently determined sampling bandwidth and the average value of multiple previously determined sampling bandwidths. After determining the average bandwidth, preliminary bandwidth fluctuation information can be determined based on the sampling bandwidth and the average bandwidth, for example, by calculating the difference between the sampling bandwidth and the average bandwidth, and using the absolute value of the difference as preliminary bandwidth fluctuation information. For example, preliminary bandwidth fluctuation information can be determined by the following formula:

[0068]

[0069] Among them, B i is the sampling bandwidth, is the average bandwidth.

[0070] In one embodiment, when determining the preliminary bandwidth fluctuation information, the preliminary bandwidth fluctuation information may be smoothed and filtered according to the weighting factor and the bandwidth fluctuation information determined last time to obtain the bandwidth fluctuation information. For example, the bandwidth fluctuation information may be determined by the following formula:

[0071]

[0072] Among them, Alpha i is the weighting factor, BF i For preliminary bandwidth fluctuation information, The weighting factor provided by this solution is determined according to the baseline filtering strength, the preset bandwidth sampling period and the preset constraint delay. For example, the weighting factor can be determined by the following formula:

[0073]

[0074] Wherein, σ is the baseline filter strength, and the value range of σ can be between 0 and 0.5, for example, σ = 0.1, T i is the preset bandwidth sampling period, and L is the preset constraint delay. This solution effectively improves the bandwidth response follow-up under different delay requirements by smoothing and filtering the bandwidth fluctuation information. While meeting the delay constraint, it effectively improves the overall picture quality and transmission stability of the video stream and improves the video encoding quality.

[0075] S240: Determine a coding target bit rate according to the sampling bandwidth and the maximum target bit rate of the current coding resolution, and determine a maximum instantaneous bit rate according to the coding target bit rate, the maximum target bit rate, the sampling bandwidth, and the bandwidth fluctuation information.

[0076] In a possible embodiment, the streaming media video encoding method provided by the present solution determines the encoding target bit rate according to the maximum target bit rate of the sampling bandwidth and the current encoding resolution, and the minimum value of the maximum target bit rate of the sampling bandwidth and the current encoding resolution can be determined as the encoding target bit rate. For example, the encoding target bit rate provided by the present solution can be determined by the following formula:

[0077]

[0078] in, is the sampling bandwidth, and MR is the maximum target bit rate of the current encoding resolution. This solution uses the minimum value of the sampling bandwidth and the maximum target bit rate as the encoding target bit rate to achieve accurate control of the average bit rate during video encoding and ensure video encoding quality.

[0079] In order to adapt to the network bandwidth while taking into account the bit rate fluctuations introduced by changes in video content complexity, and to balance video quality with network bandwidth and delay constraints, this solution can be based on the encoder's video buffer verifier mechanism to more efficiently utilize the encoder's performance, limit the maximum instantaneous bit rate through the maximum instantaneous bit rate VbvMaxBitRate, and limit the maximum encoding bit capacity within the window time with the encoding buffer size VbvBuffer. Without adding additional delays and freezes, this solution can take into account the bit rate fluctuations introduced by changes in content complexity to the greatest extent possible, so as to provide videos with more stable encoding quality.

[0080] In a possible embodiment, the streaming video encoding method provided by the present solution determines the maximum instantaneous bit rate according to the encoding target bit rate, the maximum target bit rate, the sampling bandwidth and the bandwidth fluctuation information, which may be to determine the minimum value of the product of the maximum target bit rate and the preset maximum floating ratio, and the difference between the sampling bandwidth and the bandwidth fluctuation information, and determine the maximum value of the minimum value and the encoding target bit rate as the maximum instantaneous bit rate. For example, the maximum instantaneous bit rate provided by the present solution can be determined by the following formula:

[0081]

[0082] Among them, TR i is the encoding target bit rate, μ is the preset maximum floating ratio, and the value range of μ can be between 0 and 2, for example, μ=1.5, MR is the maximum target bit rate, is the sampling bandwidth, This solution determines the minimum value of the product of the maximum target bit rate and the preset maximum floating ratio, and the difference between the sampling bandwidth and the bandwidth fluctuation information, and uses the maximum value of the minimum value and the encoding target bit rate as the maximum instantaneous bit rate. It accurately determines the maximum instantaneous bit rate allowed during the encoding process, and takes into account the bit rate fluctuation introduced by the change of content complexity to the greatest extent without adding additional delay and jamming, so as to provide more stable video encoding quality.

[0083] S250: Determine the encoding buffer size according to the maximum instantaneous bit rate and the preset constrained delay.

[0084] In a possible embodiment, the streaming media video encoding method provided by the present solution determines the encoding buffer size according to the maximum instantaneous bit rate and the preset constraint delay, which may be to multiply the maximum instantaneous bit rate and the preset constraint delay, and determine the product as the encoding buffer size. For example, the encoding buffer size provided by the present solution may be determined by the following formula:

[0085] VbvBuffer i =VbvMaxBitRate i *L

[0086] Among them, VbvMaxBitRate i is the maximum instantaneous bitrate, and L is the preset constraint delay. Since the encoding buffer size can affect the accuracy of bitrate followability, a larger encoding buffer size means a larger video frame cache window, which can provide more stable video quality, but its bitrate followability will be lower, and the risk of delay and freeze will be higher when the bandwidth drops suddenly. Based on this, in order to meet the bitrate control under different delay constraints, this solution adaptively controls the encoding buffer size according to the preset constraint delay to achieve a balance between video quality and bitrate followability.

[0087] S260: Perform streaming media video encoding based on a video buffer verifier according to the encoding target bit rate, the maximum instantaneous bit rate and the encoding buffer size.

[0088] In the above, the target encoding bit rate is determined according to the maximum target bit rate of the sampling bandwidth and the current encoding resolution, and the maximum instantaneous bit rate is determined according to the encoding target bit rate, the maximum target bit rate, the sampling bandwidth and the bandwidth fluctuation information, and the encoding buffer size is determined according to the maximum instantaneous bit rate and the preset constraint delay, and the streaming video encoding based on the video buffer verifier is performed according to the encoding target bit rate, the maximum instantaneous bit rate and the encoding buffer size, the network status and network fluctuation are accurately estimated, and the encoding target bit rate and the encoding buffer size are adaptively adjusted. While meeting the delay constraint, the overall picture quality and transmission stability of the video stream are effectively improved, and the video encoding quality is improved. At the same time, the weighting factor smoothes the sampling bandwidth and bandwidth fluctuation information, effectively improving the bandwidth response follow-up under different delay requirements, while meeting the delay constraint, the overall picture quality and transmission stability of the video stream are effectively improved, and the video encoding quality is improved. By determining the sampling bandwidth and bandwidth fluctuation information of the transmitted video stream based on real-time prediction of different preset constrained delays, adaptive bandwidth prediction based on preset constrained delays is implemented to ensure that streaming media transmission meets the delay requirements of various scenarios. A closed-loop adjustment mechanism based on bandwidth prediction and encoder control organically combines bandwidth prediction with encoder parameter adjustment, dynamically optimizes bit rate control, and effectively solves the problem of separation between bandwidth estimation and encoder adjustment in existing video encoding, ensuring transmission quality and stability, and improving video quality and stability while meeting real-time and smoothness requirements. This allows video transmission to be adaptively adjusted under different network conditions, which not only ensures transmission smoothness, but also effectively improves video quality and enhances user viewing experience.

[0089] Figure 5 Schematic diagram of a streaming media video encoding device provided by an embodiment of the present application. Figure 5 The streaming media video encoding device includes a bandwidth prediction module 51, a bit rate determination module 52, a buffer determination module 53 and a video encoding module 54.

[0090] Among them, the bandwidth prediction module 51 is configured to determine the sampling bandwidth and bandwidth fluctuation information of the transmitted video stream; the bit rate determination module 52 is configured to determine the encoding target bit rate according to the sampling bandwidth and the maximum target bit rate of the current encoding resolution, and determine the maximum instantaneous bit rate according to the encoding target bit rate, the maximum target bit rate, the sampling bandwidth and the bandwidth fluctuation information; the buffer determination module 53 is configured to determine the encoding buffer size according to the maximum instantaneous bit rate and the preset constrained delay; the video encoding module 54 is configured to perform streaming video encoding based on the video buffer verifier according to the encoding target bit rate, the maximum instantaneous bit rate and the encoding buffer size.

[0091] In the above, the encoding target bit rate is determined according to the maximum target bit rate of the sampling bandwidth and the current encoding resolution, and the maximum instantaneous bit rate is determined according to the encoding target bit rate, the maximum target bit rate, the sampling bandwidth and the bandwidth fluctuation information, and the encoding buffer size is determined according to the maximum instantaneous bit rate and the preset constraint delay, and the streaming video encoding based on the video buffer verifier is performed according to the encoding target bit rate, the maximum instantaneous bit rate and the encoding buffer size, the network status and network fluctuation are accurately estimated, and the encoding target bit rate and the encoding buffer size are adaptively adjusted. While meeting the delay constraint, the overall picture quality and transmission stability of the video stream are effectively improved, and the video encoding quality is improved.

[0092] In a possible embodiment, the bandwidth prediction module 51 determines the sampling bandwidth of the transmission video stream and is configured as follows:

[0093] Determine the sending rate, sending window size, average packet feedback delay and packet loss rate of the transmitted video stream;

[0094] Determine the preliminary sampling bandwidth according to the sending rate, the sending window size, the average packet feedback delay and the packet loss rate;

[0095] The initial sampling bandwidth is smoothed and filtered according to a weighting factor to obtain a sampling bandwidth for the transmission video stream, wherein the weighting factor is determined according to a baseline filtering strength, a preset bandwidth sampling period, and a preset constraint delay.

[0096] In a possible embodiment, the bandwidth prediction module 51 determines the preliminary sampling bandwidth according to the sending rate, the sending window size, the average packet feedback delay and the packet loss rate, and is configured as follows:

[0097] The minimum value of the ratio of the sending rate and the sending window size to the average packet feedback delay is multiplied by the difference between the first preset parameter and the packet loss rate, and the product result is used as the preliminary sampling bandwidth.

[0098] In a possible embodiment, the bandwidth prediction module 51 determines the bandwidth fluctuation information and is configured as follows:

[0099] Determine an average bandwidth of the transmitted video stream, and determine preliminary bandwidth fluctuation information based on the sampled bandwidth and the average bandwidth;

[0100] The preliminary bandwidth fluctuation information is smoothed and filtered according to a weighting factor to obtain the bandwidth fluctuation information, wherein the weighting factor is determined according to a baseline filtering strength, a preset bandwidth sampling period, and a preset constraint delay.

[0101] In a possible embodiment, the bit rate determination module 52 determines the encoding target bit rate according to the sampling bandwidth and the maximum target bit rate of the current encoding resolution, and is configured as follows:

[0102] The minimum value of the sampling bandwidth and the maximum target bit rate of the current encoding resolution is determined as the encoding target bit rate.

[0103] In a possible embodiment, the bit rate determination module 52 determines the maximum instantaneous bit rate according to the encoding target bit rate, the maximum target bit rate, the sampling bandwidth and the bandwidth fluctuation information, and is configured as follows:

[0104] The minimum value of the product of the maximum target bit rate and the preset maximum floating ratio and the difference between the sampling bandwidth and the bandwidth fluctuation information is determined, and the maximum value of the minimum value and the encoding target bit rate is determined as the maximum instantaneous bit rate.

[0105] In a possible embodiment, the buffer determination module 53 determines the encoding buffer size according to the maximum instantaneous bit rate and the preset constraint delay, and is configured as follows:

[0106] The maximum instantaneous bit rate and the preset constrained delay are multiplied, and the product is determined as the encoding buffer size.

[0107] In a possible embodiment, the preset constraint delay is determined based on a preset scenario.

[0108] It is worth noting that in the embodiment of the above-mentioned streaming media video encoding device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present application.

[0109] The embodiment of the present application also provides a streaming media video encoding device, which can integrate the streaming media video encoding apparatus provided in the embodiment of the present application. Figure 6 Schematic diagram of the structure of a streaming media video encoding device provided by an embodiment of the present application. Figure 6 The streaming video encoding device is configured as follows: an input device 63, an output device 64, a memory 62, and one or more processors 61; the memory 62 is used to store one or more programs; when one or more programs are executed by one or more processors 61, the one or more processors 61 implement the streaming video encoding method provided in the above embodiment. The streaming video encoding device, equipment, and computer provided above can be used to execute the streaming video encoding method provided in any of the above embodiments, and have corresponding functions and beneficial effects.

[0110] The embodiment of the present application also provides a non-volatile storage medium storing computer executable instructions, which are used to execute the streaming media video encoding method provided in the above embodiment when executed by a computer processor. Of course, the non-volatile storage medium storing computer executable instructions provided in the embodiment of the present application, whose computer executable instructions are not limited to the streaming media video encoding method provided above, can also execute the related operations in the streaming media video encoding method provided in any embodiment of the present application. The streaming media video encoding device, equipment and storage medium provided in the above embodiment can execute the streaming media video encoding method provided in any embodiment of the present application. For technical details not described in detail in the above embodiment, please refer to the streaming media video encoding method provided in any embodiment of the present application.

[0111] On the basis of the above embodiments, the embodiments of the present application also provide a computer program product. The technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes a number of instructions for enabling a computer device, a mobile terminal or a processor therein to execute all or part of the steps of the streaming media video encoding method provided in each embodiment of the present application.

Claims

1. A streaming media video encoding method, characterized in that: include: Determine the sampling bandwidth and bandwidth fluctuation information of the transmitted video stream; Determine a coding target bit rate according to the sampling bandwidth and a maximum target bit rate of a current coding resolution, and determine a maximum instantaneous bit rate according to the coding target bit rate, the maximum target bit rate, the sampling bandwidth, and the bandwidth fluctuation information; Determining a coding buffer size according to the maximum instantaneous bit rate and a preset constrained delay; Streaming media video encoding based on a video buffer verifier is performed according to the encoding target bit rate, the maximum instantaneous bit rate and the encoding buffer size.

2. The streaming media video encoding method according to claim 1, characterized in that: The step of determining the sampling bandwidth of the transmitted video stream comprises: Determine the sending rate, sending window size, average packet feedback delay and packet loss rate of the transmitted video stream; Determine a preliminary sampling bandwidth according to the sending rate, the sending window size, the average packet feedback delay and the packet loss rate; The preliminary sampling bandwidth is smoothed and filtered according to a weighting factor to obtain a sampling bandwidth for a transmission video stream, wherein the weighting factor is determined according to a baseline filtering strength, a preset bandwidth sampling period, and a preset constraint delay.

3. The streaming media video encoding method according to claim 2, characterized in that: The determining of the preliminary sampling bandwidth according to the sending rate, the sending window size, the average packet feedback delay and the packet loss rate comprises: The minimum value of the sending rate and the ratio of the sending window size to the average packet feedback delay is multiplied by the difference between the first preset parameter and the packet loss rate, and the product result is used as the preliminary sampling bandwidth.

4. The streaming media video encoding method according to claim 1, characterized in that: Determine bandwidth fluctuation information, including: Determine an average bandwidth of the transmitted video stream, and determine preliminary bandwidth fluctuation information according to the sampling bandwidth and the average bandwidth; The preliminary bandwidth fluctuation information is smoothed and filtered according to a weighting factor to obtain bandwidth fluctuation information, wherein the weighting factor is determined according to a baseline filtering strength, a preset bandwidth sampling period, and a preset constraint delay.

5. The streaming media video encoding method according to claim 1, characterized in that: The determining the encoding target bit rate according to the maximum target bit rate of the sampling bandwidth and the current encoding resolution includes: The minimum value of the sampling bandwidth and the maximum target bit rate of the current encoding resolution is determined as the encoding target bit rate.

6. The streaming media video encoding method according to claim 1, characterized in that: The determining the maximum instantaneous bit rate according to the encoding target bit rate, the maximum target bit rate, the sampling bandwidth and the bandwidth fluctuation information comprises: The minimum value of the product of the maximum target bit rate and the preset maximum floating ratio and the difference between the sampling bandwidth and the bandwidth fluctuation information is determined, and the maximum value of the minimum value and the encoding target bit rate is determined as the maximum instantaneous bit rate.

7. The streaming media video encoding method according to claim 1, characterized in that: The determining the encoding buffer size according to the maximum instantaneous bit rate and the preset constraint delay includes: The maximum instantaneous bit rate is multiplied by the preset constraint delay, and the product is determined as the encoding buffer size.

8. The streaming media video encoding method according to any one of claims 1 to 7, characterized in that: The preset constraint delay is determined based on a preset scenario.

9. A streaming media video encoding device, characterized in that: It includes a bandwidth prediction module, a bit rate determination module, a buffer determination module and a video encoding module, wherein: The bandwidth prediction module is configured to determine the sampling bandwidth and bandwidth fluctuation information of the transmitted video stream; The bit rate determination module is configured to determine a coding target bit rate according to the sampling bandwidth and a maximum target bit rate of a current coding resolution, and determine a maximum instantaneous bit rate according to the coding target bit rate, the maximum target bit rate, the sampling bandwidth and the bandwidth fluctuation information; The buffer determination module is configured to determine the encoding buffer size according to the maximum instantaneous bit rate and a preset constrained delay; The video encoding module is configured to perform streaming media video encoding based on a video buffer verifier according to the encoding target bit rate, the maximum instantaneous bit rate and the encoding buffer size.

10. A streaming media video encoding device, characterized in that: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the streaming media video encoding method as described in any one of claims 1-9.

11. A non-volatile storage medium storing computer executable instructions, characterized in that: The computer executable instructions are used to execute the streaming media video encoding method as described in any one of claims 1-9 when executed by a computer processor.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the streaming media video encoding method according to any one of claims 1 to 9 is implemented.

Images