Streaming media video encoding method, apparatus and device, storage medium and product

Abstract

Provided in the embodiments of the present application are a streaming media video encoding method, apparatus and device, a storage medium and a product. In the technical solution provided in the embodiments of the present application, an encoding target bitrate is determined on the basis of a sampling bandwidth and a maximum target bitrate corresponding to a current encoding resolution, a maximum instantaneous bitrate is determined on the basis of the encoding target bitrate, the maximum target bitrate, the sampling bandwidth and bandwidth fluctuation information, an encoding buffer size is determined on the basis of the maximum instantaneous bitrate and a preset constraint delay, and streaming media video encoding based on a video buffer verifier is performed on the basis of the encoding target bitrate, the maximum instantaneous bitrate and the encoding buffer size. The present application achieves accurate estimation of network conditions and network fluctuation conditions and adaptive adjustment of the encoding target bitrate and the encoding buffer size, effectively improving the overall picture quality and the stability of transmission of video streams while satisfying delay constraints, thus improving the video encoding quality.

Description

A streaming video encoding method, apparatus, device, storage medium, and product.

[0001] This application claims priority to Chinese Patent Application No. 202411717235.2, filed on November 27, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of video coding technology, and in particular to a streaming media video coding method, apparatus, device, storage medium, and product. Background Technology

[0003] With the rapid development of internet technology, video streaming services have become one of the mainstream applications, widely used in scenarios such as online live streaming, video conferencing, and cloud gaming. These scenarios have high requirements for video transmission quality and real-time performance, especially under unstable network conditions, where maintaining smooth video playback and image quality is particularly important. In practical applications, network bandwidth fluctuations are one of the main problems affecting video transmission quality. Bandwidth fluctuations may stem from factors such as network congestion, changes in the wireless network environment, or an increase in the number of users. Under conditions of bandwidth fluctuations, video transmission needs to dynamically adjust the bitrate to avoid stuttering or image quality degradation, ensuring a good user experience.

[0004] Currently, bitrate control methods are generally based on Adaptive Bit Rate (ABR) technology. ABR dynamically selects an appropriate bitrate level to transmit the video stream by detecting the current network bandwidth and the client's buffer status. This technology performs well in stable network environments, effectively adjusting the video stream's bitrate. However, in scenarios with significant network fluctuations and strict latency requirements, existing adaptive bitrate technologies suffer from adjustment lag and slow response, leading to decreased video stream quality or unstable transmission, and poor video encoding quality. Summary of the Invention

[0005] This application provides a streaming video encoding method, apparatus, device, storage medium, and product to solve the technical problem that adaptive bitrate technology in related technologies suffers from adjustment lag and slow response in scenarios with large network fluctuations and strict latency requirements, resulting in decreased video stream quality or unstable transmission and poor video encoding quality. It can adjust the video encoding bitrate in a timely manner to ensure video stream quality and transmission stability, thereby improving video encoding quality.

[0006] In a first aspect, embodiments of this application provide a streaming video encoding method for determining the sampling bandwidth and bandwidth fluctuation information of the transmitted video stream;

[0007] The target bitrate is determined based on the sampling bandwidth and the maximum target bitrate of the current coding resolution, and the maximum instantaneous bitrate is determined based on the target bitrate, the maximum target bitrate, the sampling bandwidth, and the bandwidth fluctuation information.

[0008] The size of the encoding buffer is determined based on the maximum instantaneous bit rate and the preset constraint delay;

[0009] Streaming video encoding based on a video buffer verifier is performed according to the target bitrate, the maximum instantaneous bitrate, and the encoding buffer size.

[0010] In a second aspect, embodiments of this application provide a streaming media video encoding apparatus, including a bandwidth prediction module, a bitrate determination module, a buffer determination module, and a video encoding module, wherein:

[0011] The bandwidth prediction module is configured to determine the sampling bandwidth and bandwidth fluctuation information of the transmitted video stream;

[0012] The bitrate determination module is configured to determine the target bitrate based on the sampling bandwidth and the maximum target bitrate of the current encoding resolution, and to determine the maximum instantaneous bitrate based on the target bitrate, the maximum target bitrate, the sampling bandwidth, and the bandwidth fluctuation information.

[0013] The buffer determination module is configured to determine the size of the encoding buffer based on the maximum instantaneous bit rate and the preset constraint delay;

[0014] The video encoding module is configured to perform streaming video encoding based on a video buffer verifier according to the target encoding bitrate, the maximum instantaneous bitrate, and the encoding buffer size.

[0015] In a third aspect, embodiments of this application provide a streaming media video encoding device, including: a memory and one or more processors;

[0016] The memory is used to store one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the streaming video encoding method as described in the first aspect.

[0018] In a fourth aspect, embodiments of this application provide a non-volatile storage medium for storing computer-executable instructions, which, when executed by a computer processor, are used to perform the streaming video encoding method as described in the first aspect.

[0019] In a fifth aspect, embodiments of this application provide a computer program product comprising a computer program stored in a computer-readable storage medium, wherein at least one processor of the device reads from the computer-readable storage medium and executes the computer program, causing the device to perform the streaming video encoding method as described in the first aspect.

[0020] This application embodiment determines the target bitrate based on the sampling bandwidth and the maximum target bitrate of the current encoding resolution, and determines the maximum instantaneous bitrate based on the target bitrate, the maximum target bitrate, the sampling bandwidth, and bandwidth fluctuation information. It then determines the encoding buffer size based on the maximum instantaneous bitrate and a preset constraint delay, and performs streaming video encoding based on a video buffer checker according to the target bitrate, the maximum instantaneous bitrate, and the encoding buffer size. This accurately estimates network conditions and network fluctuations, adaptively adjusts the target bitrate and the encoding buffer size, and effectively improves the overall picture quality and transmission stability of the video stream while meeting delay constraints, thus enhancing video encoding quality. Attached Figure Description

[0021] Figure 1 is a flowchart of a streaming video encoding method provided in an embodiment of this application;

[0022] Figure 2 is a flowchart of another streaming video encoding method provided in an embodiment of this application;

[0023] Figure 3 is a schematic diagram of a sampling bandwidth determination process provided in an embodiment of this application;

[0024] Figure 4 is a schematic diagram of a bandwidth fluctuation information determination process provided in an embodiment of this application;

[0025] Figure 5 is a schematic diagram of the structure of a streaming media video encoding device provided in an embodiment of this application;

[0026] Figure 6 is a schematic diagram of the structure of a streaming media video encoding device provided in an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but additional steps not included in the drawings may also be present. The above processes can correspond to methods, functions, procedures, subroutines, subroutines, etc.

[0028] The streaming video encoding method provided in this application can be applied to video encoding in live video scenarios. It aims to adaptively adjust the target bitrate and encoding buffer size based on the estimation of network conditions and network fluctuations. While meeting latency constraints, it effectively improves the overall picture quality and transmission stability of the video stream, enhances video encoding quality, and provides bitrate adaptive strategies with different latency requirements for soft real-time live streaming and real-time interactive live streaming. While meeting real-time performance and smoothness, it improves the quality and stability of encoded video, and enhances live streaming interaction and viewing experience.

[0029] Existing video coding schemes primarily rely on Adaptive Bitrate (ABR) technology and bandwidth estimation. ABR dynamically selects an appropriate bitrate level for video stream transmission by detecting current network bandwidth and client buffer status. This approach performs well in stable network environments, effectively adjusting the video stream's bitrate. However, in scenarios with significant network fluctuations and strict latency requirements, existing ABR technologies suffer from adjustment lag and slow response, leading to degraded video stream quality or unstable transmission. Furthermore, the video encoder plays a crucial role in bitrate control. Commonly used video encoders employ a Video Buffering Verifier (VBV) mechanism to limit the video stream's bitrate, preventing buffer overflows or transmission delays during transmission. However, current encoder adjustments often use fixed VBV parameters, failing to dynamically adjust according to actual network conditions. This can easily lead to a decline in transmission quality during network fluctuations. Meanwhile, bandwidth estimation and encoder adjustment in video encoding based on video buffer verifiers are disconnected. Existing bandwidth estimation techniques are processed separately from the encoder's VBV mechanism. The bandwidth prediction results cannot be directly used to adjust encoder parameters, and the encoder cannot adjust the output bitrate in a timely manner according to bandwidth fluctuations. This results in a slow response to bandwidth fluctuations, especially when the network environment fluctuates drastically. The bitrate adjustment lags, failing to guarantee the real-time performance and smoothness of streaming media transmission. Consequently, video stream transmission cannot be optimized in real time when network conditions change, affecting video quality and transmission stability. Furthermore, most existing encoder VBV mechanisms use static maximum instantaneous bitrate and buffer size, which cannot be dynamically adjusted according to real-time network conditions. This means that when bandwidth is good, the video buffer verifier fails to fully utilize network bandwidth to improve video quality, while when bandwidth is poor, the video buffer verifier cannot reasonably limit the video bitrate, leading to network transmission stuttering or delays. Based on this, an embodiment of the present application provides a streaming video encoding method to solve the technical problem that existing adaptive bitrate technology has adjustment lag and slow response in scenarios with large network fluctuations and strict latency requirements, resulting in decreased video stream quality or unstable transmission and poor video encoding quality.

[0030] Figure 1 shows a flowchart of a streaming video encoding method provided in an embodiment of this application. The streaming video encoding method provided in this application can be executed by a streaming video encoding device, which can be implemented by hardware and / or software and integrated into a streaming video encoding equipment.

[0031] The following description uses a streaming video encoding device performing a streaming video encoding method as an example. Referring to Figure 1, the streaming video encoding method includes:

[0032] S110: Determine the sampling bandwidth and bandwidth fluctuation information of the transmitted video stream.

[0033] For example, bandwidth sampling is performed on the data transmission link for transmitting video streams (e.g., uploading video streams from the broadcaster's end to a media server in a live video broadcast), and the sampling bandwidth and bandwidth fluctuation information are determined based on the bandwidth sampling results.

[0034] In one embodiment, the minimum value of the ratio of the transmission rate of the transmitted video stream and the transmission window size to the average packet feedback delay can be multiplied by the difference between the first preset parameter and the packet loss rate, and the product result can be used as the sampling bandwidth of the transmitted video stream. Alternatively, the sampling bandwidth can be smoothed and filtered with the historically calculated sampling bandwidth to obtain the final sampling bandwidth.

[0035] The bandwidth fluctuation information provided by this solution can be used to indicate the bandwidth fluctuation of the data transmission link within a set time period. Optionally, the bandwidth fluctuation information can be determined based on the difference between the sampled bandwidth and the average sampled bandwidth within the set time period, or based on the difference between the current sampled bandwidth and the previous sampled bandwidth, or based on the sampled bandwidth variance, standard deviation, etc., within the set time period. In one embodiment, the absolute value of the difference between the average bandwidth of the transmitted video stream and the sampled bandwidth can be used as the bandwidth fluctuation information. Alternatively, the bandwidth fluctuation information and the historically calculated bandwidth fluctuation information can be smoothed and filtered to obtain the final bandwidth fluctuation information.

[0036] S120: Determine the target bitrate based on the sampling bandwidth and the maximum target bitrate of the current coding resolution, and determine the maximum instantaneous bitrate based on the target bitrate, the maximum target bitrate, the sampling bandwidth, and the bandwidth fluctuation information.

[0037] For example, the maximum target bitrate for the current encoding resolution is determined, and the encoding target bitrate is determined based on the sampling bandwidth and the maximum target bitrate determined above. The encoding target bitrate can be understood as the average encoding bitrate expected to be achieved in streaming video encoding. The encoder will try to keep the output bitrate close to the encoding target bitrate to optimize transmission efficiency and compatibility while ensuring video quality.

[0038] Optionally, the minimum of the sampling bandwidth and the maximum target bitrate can be determined as the encoding target bitrate, or the average or weighted sum of the sampling bandwidth and the maximum target bitrate can be determined as the encoding target bitrate. In one embodiment, different maximum target bitrates can be pre-configured for different encoding resolutions, and the correspondence between encoding resolutions and maximum target bitrates can be determined. Based on this correspondence, the maximum target bitrate can be determined according to the encoding resolution currently used for streaming video encoding.

[0039] In one embodiment, after determining the target bitrate, the maximum instantaneous bitrate can be determined based on the target bitrate, the maximum target bitrate, the sampling bandwidth, and bandwidth fluctuation information. The maximum instantaneous bitrate can be understood as the highest instantaneous bitrate that the encoder is allowed to reach during the encoding process. This effectively reduces the possibility of the encoder allocating too many bits on some frames, leading to insufficient bits or buffer overflows in other frames.

[0040] Optionally, the maximum instantaneous bit rate can be the maximum value among the differences between the target bit rate, the maximum target bit rate (or the product of the maximum target bit rate and the preset upward ratio), the sampling bandwidth, and the bandwidth fluctuation information. Alternatively, the maximum instantaneous bit rate can be the minimum value among the differences between the maximum target bit rate (or the product of the maximum target bit rate and the preset upward ratio), the sampling bandwidth, and the bandwidth fluctuation information, and the maximum value among the target bit rates. Or, the maximum instantaneous bit rate can be the weighted sum or average of the differences between the target bit rate, the maximum target bit rate (or the product of the maximum target bit rate and the preset upward ratio), the sampling bandwidth, and the bandwidth fluctuation information.

[0041] S130: Determine the size of the encoding buffer based on the maximum instantaneous bit rate and the preset constraint delay.

[0042] For example, after determining the maximum instantaneous bit rate, the size of the encoding buffer can be determined based on the maximum instantaneous bit rate and the preset constraint delay. For instance, the product of the maximum instantaneous bit rate and the preset constraint delay can be used as the size of the encoding buffer, or a preset time coefficient can be determined based on the preset constraint delay, and the product of the preset time coefficient and the maximum instantaneous bit rate can be used as the size of the encoding buffer.

[0043] The preset constraint delay can be understood as a constraint value on the data transmission delay (e.g., the delay from the data sender to the data receiver). Optionally, the preset constraint delay provided by this solution can be determined based on a preset scenario. For example, different preset constraint delays can be configured according to the application scenario of streaming media video encoding. For example, in a real-time call scenario, the preset constraint delay can be set between 200 and 500 ms, and in a live video streaming scenario, the preset constraint delay can be set between 2000 and 10000 ms. Different preset constraint delays are used according to different scenarios to ensure that streaming media transmission meets the delay requirements of various scenarios.

[0044] S140: Perform streaming video encoding based on the video buffer checker according to the target bitrate, maximum instantaneous bitrate, and encoding buffer size.

[0045] For example, streaming video encoding based on a video buffer checker is performed according to the determined target bitrate, maximum instantaneous bitrate, and encoding buffer size. Specifically, during streaming video encoding based on the video buffer checker, the encoder buffer size is configured to the determined encoding buffer size, the maximum allowable instantaneous bitrate during encoding is limited to the maximum instantaneous bitrate, and the average encoded bitrate is controlled within the target bitrate or its corresponding error range. This solution, through bandwidth estimation and encoding control, achieves higher and more stable video quality in real-time streaming while meeting different latency constraints. It adapts to bitrate changes caused by variations in video content complexity while ensuring latency and smoothness, thereby improving and stabilizing video quality and providing a better live streaming experience for users.

[0046] The above describes a method that determines the target bitrate based on the sampling bandwidth and the maximum target bitrate of the current encoding resolution. It then determines the maximum instantaneous bitrate based on the target bitrate, the maximum target bitrate, the sampling bandwidth, and bandwidth fluctuation information. The encoding buffer size is determined based on the maximum instantaneous bitrate and a preset delay constraint. Finally, streaming video encoding based on a video buffer checker is performed using the target bitrate, the maximum instantaneous bitrate, and the encoding buffer size. This method accurately estimates network conditions and fluctuations, adaptively adjusts the target bitrate and encoding buffer size, and effectively improves the overall picture quality and transmission stability of the video stream while meeting delay constraints, thus enhancing video encoding quality.

[0047] Based on the above embodiments, Figure 2 shows a flowchart of another streaming media video encoding method provided by the embodiments of this application. This streaming media video encoding method is a specific embodiment of the above-described streaming media video encoding method. Referring to Figure 2, the streaming media video encoding method includes:

[0048] S210: Determine the transmission rate, transmission window size, average packet feedback delay, and packet loss rate of the transmitted video stream.

[0049] S220: Determine the sampling bandwidth of the transmitted video stream based on the transmission rate, transmission window size, average packet feedback delay, and packet loss rate.

[0050] For example, the transmission rate, transmission window size, average packet feedback delay, and packet loss rate of the transmitted video stream are determined. Taking the BBR (Bottleneck Bandwidth and RTT) congestion control strategy to implement the transmission control of the video stream as an example, the transmission rate can be used to control the amount of data transmitted per unit time, the transmission window size can be used to control the total number of packets sent, the average packet feedback delay can be the observed average feedback delay of data packets, and the packet loss rate can be the proportion of data packets lost within the window time.

[0051] In one embodiment, the sampling bandwidth of the transmitted video stream can be determined based on the transmission rate, transmission window size, average packet feedback delay, and packet loss rate. For example, the minimum of the ratios of the transmission rate and the transmission window size to the average packet feedback delay can be multiplied by the difference between a first preset parameter and the packet loss rate, and the product can be used as the sampling bandwidth.

[0052] In one embodiment, the sampling bandwidth can also be smoothed (e.g., exponentially weighted (EWMA) smoothing). Based on this, as shown in Figure 3, which illustrates a sampling bandwidth determination process, the streaming video encoding method provided in this solution determines the sampling bandwidth of the transmitted video stream based on the transmission rate, transmission window size, average packet feedback delay, and packet loss rate. This may include:

[0053] S221: Determine the initial sampling bandwidth based on the sending rate, sending window size, average packet feedback delay, and packet loss rate.

[0054] S222: The sampling bandwidth of the transmitted video stream is obtained by smoothing the initial sampling bandwidth according to the weighting factor. The weighting factor is determined according to the baseline filtering strength, the preset bandwidth sampling period and the preset constraint delay.

[0055] For example, the initial sampling bandwidth is determined based on the transmission rate, transmission window size, average packet feedback delay, and packet loss rate. For instance, the minimum of the ratios of the transmission rate and transmission window size to the average packet feedback delay is multiplied by the difference between a first preset parameter and the packet loss rate, and the product is used as the initial sampling bandwidth. Optionally, the initial sampling bandwidth can be determined using the following formula:

[0056] Where i is the data sampling number, and PacingRate i For the transmission rate, Cwnd i For the sending window size, LossRate i This refers to the packet loss rate.

[0057] In one embodiment, after determining the initial sampling bandwidth, the sampling bandwidth of the transmitted video stream can be obtained by smoothing and filtering the initial sampling bandwidth based on the weighting factor and the previously determined sampling bandwidth. For example, the sampling bandwidth can be determined by the following formula:

[0058] Among them, B i For the initial sampling bandwidth, For the previously determined sampling bandwidth, Alpha iThe weighting factor is determined based on the baseline filter strength, preset bandwidth sampling period, and preset constraint delay. For example, the weighting factor can be determined using the following formula:

[0059] Where σ is the baseline filter strength, and the value of σ can be between 0 and 0.5, for example, σ = 0.1, T i L is the preset bandwidth sampling period, and L is the preset constraint delay. The initial sampling bandwidth is determined based on the baseline filter strength, the preset bandwidth sampling period, and the preset constraint delay. The initial sampling bandwidth is then smoothed using a weighting factor to obtain the final sampling bandwidth. This effectively improves bandwidth response tracking under different delay requirements, and while meeting delay constraints, it effectively enhances the overall picture quality and transmission stability of the video stream, thereby improving video encoding quality.

[0060] S230: Determine bandwidth fluctuation information.

[0061] For example, after determining the sampling bandwidth of the transmitted video stream, bandwidth fluctuation information of the transmitted video stream can be determined based on the sampling bandwidth and the previously determined sampling bandwidth. For instance, the average bandwidth can be determined based on multiple previously determined sampling bandwidths within a preset time length, and the difference between the currently determined sampling bandwidth and the average bandwidth can be used as bandwidth fluctuation information.

[0062] Optionally, the bandwidth fluctuation information can be smoothed (e.g., by exponentially weighted smoothing). Based on this, as shown in Figure 4, which illustrates a bandwidth fluctuation information determination process, the streaming video encoding method provided in this solution determines bandwidth fluctuation information by including:

[0063] S231: Determine the average bandwidth of the transmitted video stream and determine preliminary bandwidth fluctuation information based on the sampling bandwidth and the average bandwidth.

[0064] S232: The bandwidth fluctuation information is obtained by smoothing and filtering the preliminary bandwidth fluctuation information according to the weighting factor. The weighting factor is determined according to the baseline filtering strength, the preset bandwidth sampling period and the preset constraint delay.

[0065] For example, the average bandwidth of the transmitted video stream is determined. This average bandwidth can be determined based on the average of multiple previously determined sampling bandwidths, or it can be determined based on the currently determined sampling bandwidth and the average of multiple previously determined sampling bandwidths. After determining the average bandwidth, preliminary bandwidth fluctuation information can be determined based on the sampling bandwidth and the average bandwidth. For example, the difference between the sampling bandwidth and the average bandwidth can be calculated, and the absolute value of the difference can be used as the preliminary bandwidth fluctuation information. For example, the preliminary bandwidth fluctuation information can be determined using the following formula:

[0066] Among them, Bi For sampling bandwidth, This represents the average bandwidth.

[0067] In one embodiment, to determine preliminary bandwidth fluctuation information, the bandwidth fluctuation information can be smoothed and filtered based on a weighting factor and the previously determined bandwidth fluctuation information. For example, the bandwidth fluctuation information can be determined using the following formula:

[0068] Among them, Alpha i As a weighting factor, BF i This is preliminary bandwidth fluctuation information. This refers to the bandwidth fluctuation information determined in the previous analysis. The weighting factor provided by this scheme is determined based on the baseline filter strength, the preset bandwidth sampling period, and the preset constraint delay. For example, the weighting factor can be determined using the following formula:

[0069] Where σ is the baseline filter strength, and the value of σ can be between 0 and 0.5, for example, σ = 0.1, T i L represents the preset bandwidth sampling period and L represents the preset constraint delay. This scheme effectively improves the bandwidth response tracking under different delay requirements by smoothing and filtering the bandwidth fluctuation information. While meeting the delay constraints, it effectively improves the overall picture quality and transmission stability of the video stream, and enhances the video encoding quality.

[0070] S240: Determine the target bitrate based on the sampling bandwidth and the maximum target bitrate of the current coding resolution, and determine the maximum instantaneous bitrate based on the target bitrate, the maximum target bitrate, the sampling bandwidth, and the bandwidth fluctuation information.

[0071] In one possible embodiment, the streaming video encoding method provided by this solution determines the target bitrate based on the sampling bandwidth and the maximum target bitrate of the current encoding resolution. Specifically, the target bitrate can be determined by the minimum value between the sampling bandwidth and the maximum target bitrate of the current encoding resolution. For example, the target bitrate provided by this solution can be determined using the following formula:

[0072] in, Where is the sampling bandwidth and MR is the maximum target bitrate at the current encoding resolution. This scheme uses the minimum of the sampling bandwidth and the maximum target bitrate as the encoding target bitrate to achieve accurate control of the average bitrate during video encoding, thus ensuring video encoding quality.

[0073] To adapt to network bandwidth while also considering bitrate fluctuations introduced by changes in video content complexity, and to balance video quality with network bandwidth and latency constraints, this solution leverages the encoder's video buffer checker mechanism to more efficiently utilize encoder performance. It limits the maximum instantaneous bitrate using the maximum instantaneous bitrate VbvMaxBitRate, and limits the maximum encoded bit capacity within a window using the same encoding buffer size VbvBuffer. This approach, without increasing additional latency or stuttering, maximizes the handling of bitrate fluctuations introduced by changes in content complexity, thus providing more stable video encoding quality.

[0074] In one possible embodiment, the streaming video encoding method provided by this solution determines the maximum instantaneous bitrate based on the target bitrate, the maximum target bitrate, the sampling bandwidth, and bandwidth fluctuation information. This can be achieved by determining the minimum value between the product of the maximum target bitrate and a preset maximum upsampling ratio, and the difference between the sampling bandwidth and the bandwidth fluctuation information. The maximum value between this minimum value and the target bitrate is then determined as the maximum instantaneous bitrate. For example, the maximum instantaneous bitrate provided by this solution can be determined using the following formula:

[0075] Among them, TR i The target bitrate is defined by μ, which is the preset maximum upsampling ratio. The value of μ can be between 0 and 2, for example, μ = 1.5. MR is the maximum target bitrate. For sampling bandwidth, This solution addresses bandwidth fluctuation information. It determines the maximum allowable instantaneous bitrate by taking the minimum difference between the product of the maximum target bitrate and the preset maximum upsampling ratio, and the difference between the sampled bandwidth and the bandwidth fluctuation information. The maximum instantaneous bitrate is then calculated by taking the maximum value between this minimum value and the target bitrate. This accurately determines the maximum allowable instantaneous bitrate during encoding, maximizing the tolerance for bitrate fluctuations introduced by changes in content complexity without adding extra latency or stuttering, thus providing more stable video encoding quality.

[0076] S250: Determine the size of the encoding buffer based on the maximum instantaneous bit rate and the preset constraint delay.

[0077] In one possible embodiment, the streaming video encoding method provided by this solution determines the encoding buffer size based on the maximum instantaneous bitrate and a preset constraint delay. This can be achieved by multiplying the maximum instantaneous bitrate and the preset constraint delay, and using the product as the encoding buffer size. For example, the encoding buffer size provided by this solution can be determined using the following formula: VbvBuffer i =VbvMaxBitRate i *L

[0078] Among them, VbvMaxBitRate iL represents the maximum instantaneous bitrate, and L is the preset constraint delay. Since the size of the encoding buffer can affect the accuracy of bitrate tracking, a larger encoding buffer size means a larger video frame buffer window, which can provide more stable video quality, but its bitrate tracking will be lower, and the risk of latency and stuttering will be higher when facing sudden bandwidth drops. Therefore, this solution adaptively controls the encoding buffer size according to the preset constraint delay to achieve a balance between video quality and bitrate tracking, in order to meet bitrate control under different delay constraints.

[0079] S260: Perform streaming video encoding based on the target bitrate, maximum instantaneous bitrate, and encoding buffer size using a video buffer checker.

[0080] The above describes a process where the target bitrate is determined based on the sampling bandwidth and the maximum target bitrate of the current encoding resolution. The maximum instantaneous bitrate is determined based on the target bitrate, the maximum target bitrate, the sampling bandwidth, and bandwidth fluctuation information. The encoding buffer size is determined based on the maximum instantaneous bitrate and a preset delay constraint. Then, streaming video encoding based on a video buffer checker is performed using the target bitrate, the maximum instantaneous bitrate, and the encoding buffer size. This accurately estimates network conditions and fluctuations, adaptively adjusting the target bitrate and encoding buffer size. While meeting delay constraints, this effectively improves the overall picture quality and transmission stability of the video stream, thus enhancing video encoding quality. Simultaneously, a weighting factor smooths the sampling bandwidth and bandwidth fluctuation information, effectively improving bandwidth response tracking under different delay requirements. This further enhances the overall picture quality and transmission stability of the video stream, improving video encoding quality while meeting delay constraints. By predicting and determining the sampling bandwidth and bandwidth fluctuation information of the transmitted video stream in real time based on different preset constraint delays, adaptive bandwidth prediction based on preset constraint delays is achieved. This ensures that streaming media transmission meets the delay requirements of various scenarios. Based on the closed-loop adjustment mechanism of bandwidth prediction and encoder control, bandwidth prediction and encoder parameter adjustment are organically combined to dynamically optimize bitrate control. This effectively solves the problem of the separation between bandwidth estimation and encoder adjustment in existing video encoding, ensuring transmission quality and stability. It achieves the goal of improving video quality and stability while meeting real-time and smoothness requirements. This allows video transmission to adaptively adjust under different network conditions, ensuring both smooth transmission and effectively improving video quality, thereby enhancing the user's viewing experience.

[0081] Figure 5 is a schematic diagram of the structure of a streaming media video encoding device provided in an embodiment of this application. Referring to Figure 5, the streaming media video encoding device includes a bandwidth prediction module 51, a bitrate determination module 52, a buffer determination module 53, and a video encoding module 54.

[0082] The bandwidth prediction module 51 is configured to determine the sampling bandwidth and bandwidth fluctuation information of the transmitted video stream; the bitrate determination module 52 is configured to determine the target bitrate based on the sampling bandwidth and the maximum target bitrate of the current encoding resolution, and to determine the maximum instantaneous bitrate based on the target bitrate, the maximum target bitrate, the sampling bandwidth, and the bandwidth fluctuation information; the buffer determination module 53 is configured to determine the size of the encoding buffer based on the maximum instantaneous bitrate and the preset constraint delay; and the video encoding module 54 is configured to perform streaming video encoding based on the video buffer checker based on the target bitrate, the maximum instantaneous bitrate, and the size of the encoding buffer.

[0083] The above describes a method that determines the target bitrate based on the sampling bandwidth and the maximum target bitrate of the current encoding resolution. It then determines the maximum instantaneous bitrate based on the target bitrate, the maximum target bitrate, the sampling bandwidth, and bandwidth fluctuation information. The encoding buffer size is determined based on the maximum instantaneous bitrate and a preset delay constraint. Finally, streaming video encoding based on a video buffer checker is performed using the target bitrate, the maximum instantaneous bitrate, and the encoding buffer size. This method accurately estimates network conditions and fluctuations, adaptively adjusts the target bitrate and encoding buffer size, and effectively improves the overall picture quality and transmission stability of the video stream while meeting delay constraints, thus enhancing video encoding quality.

[0084] In one possible embodiment, the bandwidth prediction module 51 determines the sampling bandwidth of the transmitted video stream and is configured as follows:

[0085] Determine the transmission rate, transmission window size, average packet feedback delay, and packet loss rate of the transmitted video stream;

[0086] The initial sampling bandwidth is determined based on the transmission rate, transmission window size, average packet feedback delay, and packet loss rate.

[0087] The sampling bandwidth of the transmitted video stream is obtained by smoothing the initial sampling bandwidth according to the weighting factor. The weighting factor is determined based on the baseline filtering strength, the preset bandwidth sampling period, and the preset constraint delay.

[0088] In one possible embodiment, the bandwidth prediction module 51 determines the initial sampling bandwidth based on the transmission rate, transmission window size, average packet feedback delay, and packet loss rate, and is configured as follows:

[0089] The minimum of the ratios of the transmission rate and the transmission window size to the average packet feedback delay is multiplied by the difference between the first preset parameter and the packet loss rate, and the product is used as the initial sampling bandwidth.

[0090] In one possible embodiment, the bandwidth prediction module 51 determines bandwidth fluctuation information and is configured as follows:

[0091] Determine the average bandwidth of the transmitted video stream, and determine preliminary bandwidth fluctuation information based on the sampling bandwidth and average bandwidth;

[0092] The bandwidth fluctuation information is obtained by smoothing and filtering the preliminary bandwidth fluctuation information based on the weighting factor, which is determined based on the baseline filtering strength, the preset bandwidth sampling period, and the preset constraint delay.

[0093] In one possible embodiment, the bitrate determination module 52 determines the target bitrate based on the sampling bandwidth and the maximum target bitrate of the current coding resolution, configured as follows:

[0094] The minimum of the sampling bandwidth and the maximum target bitrate of the current coding resolution is determined as the coding target bitrate.

[0095] In one possible embodiment, the bitrate determination module 52 determines the maximum instantaneous bitrate based on the target bitrate, the maximum target bitrate, the sampling bandwidth, and bandwidth fluctuation information, configured as follows:

[0096] The minimum value between the product of the maximum target bit rate and the preset maximum upsampling ratio and the difference between the sampling bandwidth and the bandwidth fluctuation information is determined, and the maximum value between the minimum value and the encoding target bit rate is determined as the maximum instantaneous bit rate.

[0097] In one possible embodiment, the buffer determination module 53 determines the size of the encoding buffer based on the maximum instantaneous bit rate and a preset constraint delay, configured as follows:

[0098] Multiply the maximum instantaneous bit rate by the preset constraint delay, and use the product as the size of the encoding buffer.

[0099] In one possible embodiment, the preset constraint delay is determined based on a preset scenario.

[0100] It is worth noting that in the embodiments of the above-mentioned streaming media video encoding device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this application.

[0101] This application also provides a streaming video encoding device, which can integrate the streaming video encoding apparatus provided in this application. Figure 6 is a schematic diagram of the structure of a streaming video encoding device provided in this application. Referring to Figure 6, the streaming video encoding device is configured as follows: an input device 63, an output device 64, a memory 62, and one or more processors 61; the memory 62 is used to store one or more programs; when one or more programs are executed by one or more processors 61, the one or more processors 61 implement the streaming video encoding method provided in the above embodiments. The streaming video encoding apparatus, device, and computer provided above can be used to execute the streaming video encoding method provided in any of the above embodiments, and have corresponding functions and beneficial effects.

[0102] This application also provides a non-volatile storage medium for storing computer-executable instructions, which, when executed by a computer processor, are used to perform the streaming video encoding method provided in the above embodiments. Of course, the computer-executable instructions provided in this application are not limited to the streaming video encoding method provided above, but can also perform related operations in the streaming video encoding method provided in any embodiment of this application. The streaming video encoding apparatus, device, and storage medium provided in the above embodiments can execute the streaming video encoding method provided in any embodiment of this application. Technical details not described in detail in the above embodiments can be found in the streaming video encoding method provided in any embodiment of this application.

[0103] Based on the above embodiments, this application also provides a computer program product. The technical solution of this application, in essence or in other words, the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes several instructions to cause a computer device, mobile terminal, or processor therein to execute all or part of the steps of the streaming media video encoding method provided in the various embodiments of this application.

Claims

1. A streaming video encoding method, wherein, include: Determine the sampling bandwidth and bandwidth fluctuation information of the transmitted video stream; The target bitrate is determined based on the sampling bandwidth and the maximum target bitrate of the current coding resolution, and the maximum instantaneous bitrate is determined based on the target bitrate, the maximum target bitrate, the sampling bandwidth, and the bandwidth fluctuation information. The size of the encoding buffer is determined based on the maximum instantaneous bit rate and the preset constraint delay; Streaming video encoding based on a video buffer verifier is performed according to the target bitrate, the maximum instantaneous bitrate, and the encoding buffer size.

2. The streaming video encoding method according to claim 1, wherein, Determining the sampling bandwidth of the transmitted video stream includes: Determine the transmission rate, transmission window size, average packet feedback delay, and packet loss rate of the transmitted video stream; The initial sampling bandwidth is determined based on the transmission rate, the transmission window size, the average packet feedback delay, and the packet loss rate. The sampling bandwidth of the transmitted video stream is obtained by smoothing and filtering the initial sampling bandwidth according to the weighting factor, which is determined based on the baseline filtering strength, the preset bandwidth sampling period, and the preset constraint delay.

3. The streaming video encoding method according to claim 2, wherein, The step of determining the initial sampling bandwidth based on the transmission rate, the transmission window size, the average packet feedback delay, and the packet loss rate includes: The minimum value of the ratio of the sending rate and the sending window size to the average packet feedback delay is multiplied by the difference between the first preset parameter and the packet loss rate, and the product is used as the initial sampling bandwidth.

4. The streaming video encoding method according to claim 1, wherein, Determine bandwidth fluctuation information, including: Determine the average bandwidth of the transmitted video stream, and determine preliminary bandwidth fluctuation information based on the sampling bandwidth and the average bandwidth; The bandwidth fluctuation information is obtained by smoothing and filtering the preliminary bandwidth fluctuation information according to the weighting factor, which is determined based on the baseline filtering strength, the preset bandwidth sampling period, and the preset constraint delay.

5. The streaming video encoding method according to claim 1, wherein, The step of determining the target coding bitrate based on the sampling bandwidth and the maximum target bitrate of the current coding resolution includes: The minimum value between the sampling bandwidth and the maximum target bitrate of the current coding resolution is determined as the coding target bitrate.

6. The streaming video encoding method according to claim 1, wherein, The step of determining the maximum instantaneous bit rate based on the target bit rate, the maximum target bit rate, the sampling bandwidth, and the bandwidth fluctuation information includes: The minimum value between the product of the maximum target bit rate and the preset maximum upswing ratio, and the difference between the sampling bandwidth and the bandwidth fluctuation information is determined, and the maximum value between the minimum value and the target bit rate is determined as the maximum instantaneous bit rate.

7. The streaming video encoding method according to claim 1, wherein, The step of determining the encoding buffer size based on the maximum instantaneous bit rate and the preset constraint delay includes: The maximum instantaneous bit rate and the preset constraint delay are multiplied together, and the product is used to determine the size of the encoding buffer.

8. The streaming video encoding method according to any one of claims 1-7, wherein, The preset constraint delay is determined based on a preset scenario.

9. A streaming video encoding apparatus, wherein, It includes a bandwidth prediction module, a bitrate determination module, a buffer determination module, and a video encoding module, among which: The bandwidth prediction module is configured to determine the sampling bandwidth and bandwidth fluctuation information of the transmitted video stream; The bitrate determination module is configured to determine the target bitrate based on the sampling bandwidth and the maximum target bitrate of the current encoding resolution, and to determine the maximum instantaneous bitrate based on the target bitrate, the maximum target bitrate, the sampling bandwidth, and the bandwidth fluctuation information. The buffer determination module is configured to determine the size of the encoding buffer based on the maximum instantaneous bit rate and the preset constraint delay; The video encoding module is configured to perform streaming video encoding based on a video buffer verifier according to the target encoding bitrate, the maximum instantaneous bitrate, and the encoding buffer size.

10. A streaming video encoding device, wherein, include: Memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the streaming video encoding method as described in any one of claims 1-9.

11. A non-volatile storage medium for storing computer-executable instructions, wherein, The computer-executable instructions, when executed by a computer processor, are used to perform the streaming video encoding method as described in any one of claims 1-9.

12. A computer program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the streaming video encoding method according to any one of claims 1-9.

Images