An encoding control method, apparatus, device, storage medium, and product

Abstract

The embodiment of the present application provides a kind of encoding control method, device, equipment, storage medium and product.The technical scheme provided by the embodiment of the present application determines the encodable code rate according to the estimated bandwidth, the size of sending buffer, and the set delay limit information and code rate control information, determines the budget frame rate according to the size of single frame image, and adjusts the frame rate of the first acquisition frame data stream collected to obtain the second acquisition frame data stream according to the budget frame rate, adjusts the frame rate corresponding to the first acquisition frame data stream collected, quickly adapts the estimated bandwidth of transmission link and the size of sending buffer of sending queue from frame level, can respond to the change of network bandwidth in time, reduces the fluctuation of sending queue cache, effectively improves image encoding quality.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an encoding control method, apparatus, device, storage medium, and product. Background Technology

[0002] During live video streaming, the broadcaster's network can change dynamically. For example, signal strength may fluctuate on mobile cellular networks, or network congestion may occur due to a large number of users accessing the network. Similarly, on public Wi-Fi networks, user movement may alter the signal, or other users may upload data, consuming network bandwidth. In these situations, the broadcaster's available bandwidth and data transmission rate will change accordingly. To ensure timely and effective delivery of the broadcaster's video stream to the receiving end, the streaming bitrate needs to dynamically adapt to changes in uplink bandwidth or transmission rate.

[0003] To improve transmission efficiency and ensure the quality and smoothness of live video streaming, the maximum transmittable bitrate is typically calculated based on estimated network bandwidth and transmission rate. Then, the encoding configuration bitrate and frame rate are determined according to the current resolution. The encoding parameters are reset to the encoder on a per-picture-group (GOP) basis. Due to the dependencies between encoded frames, frame data deletion is done on a per-picture-group basis. This means that the minimum adjustment precision of the encoding parameters is one per-picture-group. When network bandwidth changes during the generation of a picture group, the image encoding bitrate cannot be adjusted in a timely manner in response to changes in network bandwidth. Fluctuations in the transmission queue buffer can easily occur, resulting in poor image encoding quality. Summary of the Invention

[0004] This application provides an encoding control method, apparatus, device, storage medium, and product to solve the technical problems in related technologies where the image encoding bitrate cannot respond to changes in network bandwidth in a timely manner, the transmission queue buffer is prone to fluctuations, and the image encoding quality is poor. The method and product can respond to changes in network bandwidth in a timely manner, reduce fluctuations in the transmission queue buffer, and effectively improve the image encoding quality.

[0005] In a first aspect, embodiments of this application provide an encoding control method, comprising:

[0006] Obtain the estimated bandwidth corresponding to the transmission link and the size of the transmission buffer corresponding to the transmission queue. Determine the coded bit rate based on the estimated bandwidth, the size of the transmission buffer, and the set delay limit information and bit rate control information.

[0007] Obtain the size of a single frame image corresponding to the encoder's encoding processing, and determine the budgeted frame rate based on the encodeable bitrate and the size of the single frame image;

[0008] The frame rate of the first acquired frame data stream is adjusted based on the budgeted frame rate to obtain the second acquired frame data stream.

[0009] In a second aspect, embodiments of this application provide an encoding control device, including a bit rate determination module, a frame rate determination module, and a frame rate adjustment module, wherein:

[0010] The bit rate determination module is configured to obtain the estimated bandwidth corresponding to the transmission link and the size of the transmission buffer corresponding to the transmission queue, and determine the coded bit rate based on the estimated bandwidth, the size of the transmission buffer, and the set delay limit information and bit rate control information.

[0011] The frame rate determination module is configured to obtain the size of a single frame image corresponding to the encoding process performed by the encoder, and determine the budgeted frame rate based on the encodeable bitrate and the size of the single frame image.

[0012] The frame rate adjustment module is configured to adjust the frame rate of the first acquired frame data stream based on the budgeted frame rate to obtain the second acquired frame data stream.

[0013] In a third aspect, embodiments of this application provide an encoding control device, including: a memory and one or more processors;

[0014] The memory is used to store one or more programs;

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the encoding control method as described in the first aspect.

[0016] In a fourth aspect, embodiments of this application provide a non-volatile storage medium for storing computer-executable instructions, which, when executed by a computer processor, are used to perform the encoding control method as described in the first aspect.

[0017] In a fifth aspect, embodiments of this application provide a computer program product comprising a computer program stored in a computer-readable storage medium, wherein at least one processor of the device reads from the computer-readable storage medium and executes the computer program, causing the device to perform the coded control method as described in the first aspect.

[0018] This application embodiment determines the coded bitrate based on the estimated bandwidth, the size of the transmission buffer, and the set delay limit information and bitrate control information. It also determines the budgeted frame rate based on the size of a single frame image and adjusts the frame rate of the first acquired frame data stream according to the budgeted frame rate to obtain the second acquired frame data stream. By adjusting the frame rate corresponding to the first acquired frame data stream, it can quickly adapt to the estimated bandwidth of the transmission link and the size of the transmission buffer of the transmission queue at the frame level. This allows for timely response to changes in network bandwidth, reduces fluctuations in the transmission queue buffer, and effectively improves the image encoding quality. Attached Figure Description

[0019] Figure 1 This is a flowchart of an encoding control method provided in an embodiment of this application;

[0020] Figure 2 This is a schematic diagram of an encodeable bitrate determination process provided in an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of an encoding control network structure provided in an embodiment of this application;

[0022] Figure 4 This is a schematic diagram of the structure of an encoding control device provided in an embodiment of this application;

[0023] Figure 5 This is a schematic diagram of the structure of an encoding control device provided in an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but additional steps not included in the drawings may also be present. The above processes can correspond to methods, functions, procedures, subroutines, subroutines, etc.

[0025] The encoding control method provided in this application can be applied to the encoding processing of captured frame data streams in real-time live streaming scenarios. It rapidly adapts to the estimated bandwidth of the transmission link and the size of the transmission queue's transmission buffer at the frame level. The aim is to determine the budgeted frame rate based on the estimated bandwidth and transmission buffer size, and adjust the frame rate of the first captured frame data stream according to the budgeted frame rate. This timely response to changes in network bandwidth reduces fluctuations in the transmission queue buffer and effectively improves image encoding quality. Traditional responses to network bandwidth changes typically update the encoder's encoding parameters on a per-image-group basis. Due to the dependencies between encoded frames, frame data deletion occurs on an image-group basis. When network bandwidth changes during image group generation, the encoder cannot promptly adjust the image encoding rate in response to these changes, leading to fluctuations in the transmission queue buffer and even stuttering, resulting in poor image encoding quality. Therefore, this application provides an encoding control method that solves the technical problem of existing network bandwidth change response methods failing to respond promptly to network bandwidth changes and resulting in poor image encoding quality.

[0026] Figure 1 A flowchart of an encoding control method provided in an embodiment of this application is given. The encoding control method provided in this application embodiment can be executed by an encoding control device, which can be implemented by hardware and / or software and integrated into the encoding control equipment.

[0027] The following description uses an encoding control device executing an encoding control method as an example. (Reference) Figure 1 The coding control method includes:

[0028] S101: Obtain the estimated bandwidth corresponding to the transmission link and the size of the transmission buffer corresponding to the transmission queue. Based on the estimated bandwidth, the size of the transmission buffer, and the set delay limit information and rate control information, determine the coded rate.

[0029] The estimated bandwidth corresponding to the transmission link provided by this scheme can be determined by the congestion control algorithm configured in the sending window (sending window / congestion control module), and the size of the sending buffer corresponding to the sending queue can be determined by the sending queue based on the amount of data it has buffered. The set delay limit information is used to indicate the limitation of data transmission delay, and the code rate control information is used to indicate the expected time precision for controlling the code rate, that is, the coding code rate is adjusted once every time precision.

[0030] For example, the estimated bandwidth corresponding to the transmission link is obtained from the sending window, and the sending buffer size corresponding to the sending queue is obtained from the sending queue. The set delay limit information and bit rate control information are determined, and the coded bit rate is calculated based on the estimated bandwidth, the sending buffer size, and the set delay limit information and bit rate control information. The coded bit rate can be understood as the maximum coding bit rate for image encoding under the current estimated bandwidth and buffer size.

[0031] In one possible embodiment, such as Figure 2 The provided schematic diagram illustrates a process for determining the coded bitrate. The coding control method provided in this scheme, when determining the coded bitrate based on estimated bandwidth, transmit buffer size, and set delay limits and bitrate control information, includes:

[0032] S1011: Determine the size of the reserved buffer data based on the estimated bandwidth, the size of the transmission buffer, and the set delay limit information.

[0033] S1012: Determine the coded bitrate based on the reserved buffer data size, estimated bandwidth, and set bitrate control information.

[0034] The delay limit information provided in this solution is used to indicate the upper limit of the delay time for data transmission in the sending queue (e.g., 180ms), and the reserved buffer data size is used to indicate the size of the additional buffer data that needs to be sent under the set delay limit information.

[0035] For example, based on the obtained estimated bandwidth, transmit buffer size, and set delay limit information, the size of the reserved buffer data under the set delay limit information is calculated. Then, the coded bitrate is calculated based on the determined reserved buffer data size, estimated bandwidth, and set bitrate control information.

[0036] In one possible embodiment, the size of the reserved cache data provided by this solution can be determined based on the following formula:

[0037] Cm i =C i -B i *L max

[0038] Among them, B i To estimate bandwidth, C i L is the size of the send buffer. max This is for delay limit information;

[0039] In one possible embodiment, the coded code rate provided by this scheme is determined based on the following formula:

[0040]

[0041] Where T is the bitrate control information. For example, if the bitrate control information T = 0.8s, the budget frame rate is adjusted every 0.8s, thereby achieving the bitrate adjustment effect of adjusting the encoded bitrate every 0.8s.

[0042] This solution calculates the amount of reserved buffer data that needs to be transmitted under set delay constraints, and accurately determines the readable bitrate based on the reserved buffer data size, estimated bandwidth, and bitrate control information. Existing methods adapt to bandwidth changes by increasing encoding parameters, but their bitrate output depends on the encoder's bitrate control strategy. However, this strategy cannot promptly meet the expected bitrate adjustment target and can negatively impact the quality of the encoded frames (image groups consist of multiple encoded frames). This solution controls the bitrate output by controlling the frame rate, achieving precise control over bitrate and transmission delay without altering encoding parameters or quality.

[0043] S102: Obtain the size of a single frame image corresponding to the encoder's encoding processing, and determine the budgeted frame rate based on the coded bitrate and the size of the single frame image.

[0044] The single-frame image size provided by this solution can be understood as the actual size (data volume) of the encoded frame output by the encoder when it encodes the acquired frame data stream in real time, or the average size of each encoded frame in the actual output image group.

[0045] For example, the size of a single frame image corresponding to the encoder's encoding processing is obtained, and the budgeted frame rate is calculated based on the determined coded bitrate and the single frame image size. For instance, the ratio of the coded bitrate to the single frame image size can be used as the budgeted frame rate, or if the budgeted frame rate calculated based on the coded bitrate and the single frame image size is greater than a set frame rate upper limit, the set frame rate upper limit is used as the current budgeted frame rate.

[0046] In one possible embodiment, the encoding control method provided by this solution determines the budget frame rate based on the coded bitrate and the size of a single frame image, including: determining the candidate frame rate based on the coded bitrate and the size of a single frame image; and determining the budget frame rate based on the set frame rate constraints and the candidate frame rates.

[0047] For example, a candidate frame rate is calculated based on the coded bitrate and the size of a single frame. For instance, the ratio of the coded bitrate to the size of a single frame is used as the candidate frame rate. Further, a budget frame rate is determined based on set frame rate constraints and the candidate frame rate. The frame rate constraints can be represented by a set maximum output frame rate (e.g., 24 fps) and a set minimum output frame rate (e.g., 1 fps). For example, if the candidate frame rate falls within the frequency range corresponding to the maximum and minimum output frame rates, the candidate frame rate is used as the budget frame rate; if the candidate frame rate is greater than the maximum output frame rate, the set maximum output frame rate is used as the budget frame rate; and if the candidate frame rate is less than the minimum output frame rate, the set minimum output frame rate is used as the budget frame rate.

[0048] In one possible embodiment, the budgeted frame rate provided by this scheme can be determined based on the following formula:

[0049]

[0050] Among them, Fr i For the budget frame rate, Br i For the coded bitrate, Fb i For the size of a single frame image, Fr min For the set minimum output frame rate, Fr max This is the set maximum output frame rate. This scheme limits the selected value of the budgeted frame rate by setting frame rate constraints, ensuring the frame rate adjustment effect of the first acquired frame data stream and guaranteeing the image encoding effect.

[0051] S103: Adjust the frame rate of the first acquisition frame data stream based on the budgeted frame rate to obtain the second acquisition frame data stream.

[0052] For example, the first acquired frame data stream is received, and the frame rate of the first acquired frame data stream is adjusted based on the budgeted frame rate determined above, so that the frame rate of the second acquired frame data stream corresponds to the budgeted frame rate. The first acquired frame data stream can be multiple consecutive raw encoded frames acquired by a camera, or multiple consecutive encoded frames after preprocessing (e.g., beautification) the raw encoded frames.

[0053] It needs to be explained that by adjusting the frame rate of the first acquisition frame data stream based on the budgeted frame rate, when the frame rate of the first acquisition frame data stream increases, it is equivalent to increasing the output encoding bitrate proportionally. Conversely, when the frame rate of the first acquisition frame data stream decreases, it is equivalent to decreasing the output encoding bitrate proportionally. This adaptive adjustment of the encoding bitrate is achieved by adjusting the frame rate of the acquisition frame data stream.

[0054] This solution determines the coded bitrate by combining estimated bandwidth and transmit buffer size, and then converts the coded bitrate into a budgeted frame rate for adjusting the frame rate of the captured frames. This transforms the bitrate control problem in video encoding and transmission into a frame rate control problem, ensuring that the transmit latency is minimized while maximizing the bitrate. This achieves real-time and accurate control of the encoded bitrate. Compared to bitrate adjustment at the image group level or network bandwidth response methods that control encoding parameters in real time, this solution achieves the effect of bitrate adjustment by adjusting the frame rate of the captured frame data stream at the frame level. It responds more quickly and accurately to changes in network bandwidth and does not require changes to the encoder's encoding parameters, effectively maintaining image encoding quality.

[0055] In one possible embodiment, when adjusting the frame rate of the first acquired frame data stream based on the budgeted frame rate, the comparison result between the frame rate corresponding to the first acquired frame data stream and the budgeted frame rate can be used to determine whether to perform frame interpolation or frame dropping on the first acquired frame data stream. Based on this, this solution adjusts the frame rate of the acquired first acquired frame data stream based on the budgeted frame rate to obtain the second acquired frame data stream, including:

[0056] If the frame rate of the first acquired frame data stream is less than the budgeted frame rate, the first acquired frame data stream is interpolated to obtain the second acquired frame data stream.

[0057] If the frame rate of the first acquired frame data stream is greater than the budgeted frame rate, the first acquired frame data stream is dropped to obtain the second acquired frame data stream.

[0058] For example, the frame rate corresponding to the first acquired frame data stream is compared with the budgeted frame rate. If the frame rate of the first acquired frame data stream is less than the budgeted frame rate, frame interpolation is performed on the first acquired frame data stream based on a set frame interpolation algorithm to obtain the second acquired frame data stream. Conversely, if the frame rate of the first acquired frame data stream is greater than the budgeted frame rate, frame dropping is performed on the first acquired frame data stream based on a set frame dropping algorithm to obtain the second acquired frame data stream. In this case, the frame rate of the second acquired frame data stream is consistent with the budgeted frame rate. When the frame rate of the first acquired frame data stream is consistent with the budgeted frame rate, there is no need to adjust the frame rate of the first acquired frame data stream; it can be directly used as the second acquired frame data stream.

[0059] This solution performs frame interpolation or frame dropping on the first acquisition frame data stream based on the comparison between the frame rate corresponding to the first acquisition frame data stream and the budgeted frame rate, so that the frame rate of the second acquisition frame data stream corresponds to the budgeted frame rate. This enables a fast and accurate response to changes in network bandwidth. By adjusting the frame rate, accurate and timely bitrate control is achieved, effectively reducing the transmission latency and playback stuttering of image encoded data, without affecting the quality of the encoded image, thus optimizing the user's live streaming experience.

[0060] In one possible embodiment, after adjusting the frame rate of the first acquired frame data stream based on the budgeted frame rate to obtain the second acquired frame data stream, the encoding control method provided by this solution further includes: encoding the second acquired frame data stream to obtain an encoded image group, and sending the encoded image group to the sending window so that the sending window can output the encoded image group based on the set congestion control strategy.

[0061] For example, after adjusting the frame rate of the first acquisition frame data stream based on the budget frame rate to obtain the second acquisition frame data stream, the second acquisition frame data stream can be encoded by an encoder to obtain an encoded image group, and the encoded image group can be sent to the sending window, which outputs the encoded image group based on the set congestion control strategy.

[0062] In one possible embodiment, the encoding control method provided by this solution, when encoding based on the second acquisition frame data stream to obtain an encoded image group, includes: inputting the second acquisition frame data stream into an encoding queue; obtaining the second acquisition frame data stream from the encoding queue, encoding the obtained second acquisition frame data stream using an encoder based on set encoding parameters to obtain an encoded image group, and sending the encoded image group to a transmission queue; obtaining the encoded image group from the transmission queue, and performing frame dropping processing on the obtained encoded image group based on a set frame dropping strategy.

[0063] For example, after adjusting the frame rate of the first acquired frame data stream to obtain the second acquired frame data stream, the second acquired frame data stream is input into the encoding queue. The encoder retrieves the second acquired frame data stream from the encoding queue and encodes it based on the set encoding parameters to obtain an encoded image group. Optionally, the encoding parameters can be one or more combinations of resolution, encoding bitrate, and encoding frame rate. Further, the encoded image group obtained after encoding is sent to the transmission queue to await transmission.

[0064] The system retrieves and sends encoded image groups from the transmission queue. Based on a predefined frame-dropping strategy, it performs frame-dropping processing on the retrieved encoded image groups and then sends the processed encoded image groups, for example, by uploading them to a designated server. The server then transcodes the encoded image groups and sends the corresponding video data to various user terminals (e.g., viewers). This solution, by performing frame-dropping processing on the retrieved encoded image groups and smoothly filtering out dropped frames, adapts the output bitrate to the network transmission capacity, maximizing the video bitrate while minimizing transmission latency. This improves the real-time performance and smoothness of live streaming while maintaining video quality.

[0065] In one possible embodiment, the encoding control method provided by this solution, after encoding the acquired second acquisition frame data stream by the encoder based on the set encoding parameters to obtain an encoded image group, further includes: determining the size of a single frame image based on the real-time image size of each encoded frame in the encoded image group generated by the encoder.

[0066] For example, after the encoder encodes the second acquisition frame data stream based on the set encoding parameters to obtain an encoded image group, the real-time image size of each encoded frame in the encoded image group generated by the encoder is determined, and the single frame image size is determined according to the real-time image size of each encoded frame.

[0067] In one possible embodiment, the encoding control method provided by this solution determines the single-frame image size based on the real-time image size of each encoded frame in the encoded image group generated by the encoder, including: determining the current single-frame image size according to a set exponentially weighted moving average filtering strategy, based on the real-time image size of each encoded frame in the encoded image group generated by the encoder and the previously determined single-frame image size.

[0068] Optionally, the single-frame image size provided by this scheme can be determined by the real-time image size Fb of each image frame encoded by the encoder. o The result is obtained by applying an exponentially weighted moving average (ewma) filter, i.e.: in β is the size of the single frame image determined in the previous step, and β is the smoothing coefficient corresponding to the set exponentially weighted moving average filter. This scheme calculates the current single frame image size by using exponentially weighted moving average filtering to achieve smooth processing of frame rate adjustment.

[0069] like Figure 3 A schematic diagram of an encoding control network structure is provided. The encoding control network structure includes a frame-level bitrate control module, a frame rate control module, an encoding queue, an encoder, a transmission queue, a transmission window, and a bitrate control module. The frame-level bitrate control module obtains in real time the estimated bandwidth provided by the transmission window, the size of the transmission buffer provided by the transmission queue, and the size of a single frame image actually encoded by the encoder. The frame-level bitrate control module determines the coded bitrate based on the estimated bandwidth, the size of the transmission buffer, and the set delay limit information and bitrate control information. It then determines the budgeted frame rate based on the coded bitrate and the size of a single frame image and sends the budgeted frame rate to the frame rate control module.

[0070] The frame rate control module adjusts the frame rate of the first acquired frame data stream based on the received budgeted frame rate to obtain the second acquired frame data stream, and sends the second acquired frame data stream to the encoding queue. The bit rate control module determines the encoding parameters based on a set bit rate control algorithm and sends them to the encoder. The encoder extracts the second acquired frame data stream from the encoding queue, encodes it based on the encoding parameters to obtain an encoded image group, and sends the encoded image group to the transmission queue. Simultaneously, the encoder determines the current single-frame image size based on the real-time image size of each encoded frame in the real-time generated encoded image group, and the previously determined single-frame image size, according to a set exponentially weighted moving average filtering strategy, and sends this size to the frame-level bit rate control module. The transmission queue performs frame dropping processing on the encoded image group provided by the encoder based on a set frame dropping strategy. The transmission window retrieves the frame-dropped encoded image group from the transmission queue and uploads it to the set server based on a set congestion control algorithm. Simultaneously, it calculates the estimated bandwidth corresponding to the transmission link based on the set congestion control algorithm and sends it to the frame-level bit rate control module.

[0071] As described above, the coded bitrate is determined based on estimated bandwidth, transmit buffer size, and set delay limits and bitrate control information. The budgeted frame rate is determined based on the size of a single frame image. The second acquired frame data stream is obtained by adjusting the frame rate of the first acquired frame data stream according to the budgeted frame rate. By adjusting the frame rate corresponding to the first acquired frame data stream, the system quickly adapts to the estimated bandwidth of the transmission link and the transmit buffer size of the transmit queue at the frame level. This allows for timely response to changes in network bandwidth, reduces fluctuations in the transmit queue buffer, and effectively improves image encoding quality. Simultaneously, by controlling the frame rate to control the bitrate output, precise control of bitrate and transmit delay is achieved without changing encoding parameters and encoding quality. Furthermore, bitrate adjustment can be achieved at the frame level by adjusting the frame rate of the acquired frame data stream, resulting in a faster and more accurate response to network bandwidth changes without altering the encoder's encoding parameters, effectively maintaining image encoding quality.

[0072] Figure 4 This is a schematic diagram of the structure of an encoding control device provided in an embodiment of this application. (Reference) Figure 4 The encoding control device includes a bit rate determination module 41, a frame rate determination module 42, and a frame rate adjustment module 43.

[0073] The bit rate determination module 41 is configured to obtain the estimated bandwidth corresponding to the transmission link and the size of the transmission buffer corresponding to the transmission queue, and determine the coded bit rate based on the estimated bandwidth, the size of the transmission buffer, and the set delay limit information and bit rate control information; the frame rate determination module 42 is configured to obtain the size of a single frame image corresponding to the encoder encoding processing, and determine the budgeted frame rate based on the coded bit rate and the size of the single frame image; the frame rate adjustment module 43 is configured to adjust the frame rate of the first acquired frame data stream based on the budgeted frame rate to obtain the second acquired frame data stream.

[0074] The above describes a method that determines the coded bitrate based on estimated bandwidth, transmit buffer size, and set delay and bitrate control information. It also determines the budgeted frame rate based on the size of a single frame image and adjusts the frame rate of the first acquired frame data stream to obtain the second acquired frame data stream based on the budgeted frame rate. By adjusting the frame rate corresponding to the first acquired frame data stream, the method can quickly adapt to the estimated bandwidth of the transmission link and the transmit buffer size of the transmit queue at the frame level. This allows for timely response to changes in network bandwidth, reduces fluctuations in the transmit queue buffer, and effectively improves image encoding quality.

[0075] Based on the above embodiments, when determining the coded bit rate according to the estimated bandwidth, the size of the transmission buffer, and the set delay limit information and bit rate control information, the bit rate determination module 41 is configured as follows:

[0076] Based on the estimated bandwidth, the size of the transmission buffer, and the set delay limits, the size of the reserved buffer data is determined.

[0077] The coded bitrate is determined based on the reserved cache data size, estimated bandwidth, and set bitrate control information.

[0078] Based on the above embodiments, the size of the reserved cache data is determined according to the following formula:

[0079] Cm i =C i -B i *L max

[0080] Among them, B i To estimate bandwidth, C i L is the size of the send buffer. max This is for delay limit information;

[0081] The coded bitrate is determined based on the following formula:

[0082]

[0083] Where T represents the bit rate control information.

[0084] Based on the above embodiments, the frame rate determination module 42 is configured as follows when determining the budgeted frame rate based on the coded bitrate and the size of a single frame image:

[0085] Candidate frame rates are determined based on the coded bitrate and the size of a single frame image.

[0086] The budgeted frame rate is determined based on the set frame rate constraints and candidate frame rates.

[0087] Based on the above embodiments, the frame rate adjustment module 43 is configured as follows:

[0088] If the frame rate of the first acquired frame data stream is less than the budgeted frame rate, the first acquired frame data stream is interpolated to obtain the second acquired frame data stream.

[0089] If the frame rate of the first acquired frame data stream is greater than the budgeted frame rate, the first acquired frame data stream is dropped to obtain the second acquired frame data stream.

[0090] Based on the above embodiments, the encoding control device further includes an image sending module. The image sending module is configured to encode an encoded image group based on the second acquisition frame data stream and send the encoded image group to the sending window so that the sending window can output the encoded image group based on the set congestion control strategy.

[0091] Based on the above embodiments, when the image sending module encodes the encoded image group based on the second acquisition frame data stream, it is configured as follows:

[0092] The second acquisition frame data stream is input into the encoding queue;

[0093] The second acquisition frame data stream is obtained from the encoding queue. The encoder encodes the obtained second acquisition frame data stream based on the set encoding parameters to obtain an encoded image group, and then sends the encoded image group to the sending queue.

[0094] The encoded image group is obtained from the sending queue, and the obtained encoded image group is processed by dropping frames based on the set frame dropping strategy.

[0095] Based on the above embodiments, the encoding control device further includes an image size calculation module, which is configured to determine the size of a single frame image based on the real-time image size of each encoded frame in the encoded image group generated by the encoder.

[0096] Based on the above embodiments, when the image size calculation module determines the size of a single frame image based on the real-time image size of each encoded frame in the encoded image group generated by the encoder, it is configured to determine the current size of a single frame image according to the set exponentially weighted moving average filtering strategy, based on the real-time image size of each encoded frame in the encoded image group generated by the encoder and the previously determined size of a single frame image.

[0097] It is worth noting that in the embodiments of the above-mentioned encoding control device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of the present invention.

[0098] This application also provides an encoding control device that can integrate the encoding control apparatus provided in this application. Figure 5 This is a schematic diagram of the structure of an encoding control device provided in an embodiment of this application. (Reference) Figure 5 The encoding control device includes: an input device 53, an output device 54, a memory 52, and one or more processors 51; the memory 52 is used to store one or more programs; when one or more programs are executed by one or more processors 51, the one or more processors 51 implement the encoding control method provided in the above embodiments. The encoding control device, equipment, and computer provided above can be used to execute the encoding control method provided in any of the above embodiments, and have corresponding functions and beneficial effects.

[0099] This application also provides a non-volatile storage medium for storing computer-executable instructions, which, when executed by a computer processor, are used to perform the encoding control method provided in the above embodiments. Of course, the computer-executable instructions in the non-volatile storage medium for storing computer-executable instructions provided in this application are not limited to the encoding control method provided above, and can also perform related operations in the encoding control method provided in any embodiment of this application. The encoding control device, apparatus, and storage medium provided in the above embodiments can execute the encoding control method provided in any embodiment of this application. Technical details not described in detail in the above embodiments can be found in the encoding control method provided in any embodiment of this application.

[0100] Based on the above embodiments, this application also provides a computer program product. The technical solution of this application, in essence or in other words, the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes several instructions to cause a computer device, mobile terminal, or processor therein to execute all or part of the steps of the coded control method provided in the various embodiments of this application.

Claims

1. An encoding control method characterized by comprising: The method comprises: obtaining an estimated bandwidth corresponding to a transmission link and a sending buffer size corresponding to a sending queue, determining a reserved cache data size based on the estimated bandwidth, the sending buffer size and set delay limit information, wherein the delay limit information is used to indicate a limit of data sending delay, and the code rate control information is used to indicate an expected time precision of controlling the code rate; obtaining a single frame image size corresponding to encoding processing of an encoder, determining a budget frame rate based on the encodable code rate and the single frame image size; adjusting a frame rate of a first collected frame data stream based on the budget frame rate to obtain a second collected frame data stream.

2. The encoding control method according to claim 1, characterized by, The reserved cache data size is determined based on the following formula: wherein, BW is the estimated bandwidth, BS is the send buffer size, DL is the delay limit information; The encodable code rate is determined based on the following formula: wherein, is the rate control information.

3. The encoding control method according to claim 1, characterized by, The budget frame rate is determined based on the encodable code rate and the single frame image size, comprising: determining a candidate frame rate based on the encodable code rate and the single frame image size; determining the budget frame rate based on a set frame rate constraint and the candidate frame rate.

4. The encoding control method according to claim 1, characterized by The frame rate of the first collected frame data stream is adjusted based on the budget frame rate to obtain the second collected frame data stream, comprising: in a case where the frame rate of the first collected frame data stream is less than the budget frame rate, performing frame insertion processing on the first collected frame data stream to obtain the second collected frame data stream; in a case where the frame rate of the first collected frame data stream is greater than the budget frame rate, performing frame dropping processing on the first collected frame data stream to obtain the second collected frame data stream.

5. The encoding control method according to claim 1, characterized by, After the frame rate of the first collected frame data stream is adjusted based on the budget frame rate to obtain the second collected frame data stream, the method further comprises: encoding the second collected frame data stream to obtain an encoded image group, and sending the encoded image group to a sending window, so that the sending window outputs the encoded image group based on a set congestion control strategy.

6. The encoding control method according to claim 5, characterized by The encoding of the second collected frame data stream to obtain an encoded image group comprises: inputting the second collected frame data stream to an encoding queue; obtaining the second collected frame data stream from the encoding queue, encoding the obtained second collected frame data stream based on a set encoding parameter by an encoder to obtain an encoded image group, and sending the encoded image group to a sending queue; obtaining the encoded image group from the sending queue, and performing frame dropping processing on the obtained encoded image group based on a set frame dropping strategy.

7. The encoding control method according to claim 6, characterized by, After the encoding of the second collected frame data stream to obtain an encoded image group by the encoder based on the set encoding parameter, the method further comprises: determining a single frame image size based on a real-time image size of each encoded frame in the encoded image group generated by the encoder.

8. The encoding control method according to claim 7, characterized by, The determination of the single frame image size based on the real-time image size of each encoded frame in the encoded image group generated by the encoder comprises: According to a set exponential weighted moving average filtering strategy, a current single frame image size is determined based on a real-time image size of each encoded frame in an encoded image group generated by the encoder and a last determined single frame image size.

9. An encoding control device characterized by comprising: The method comprises a code rate determination module, a frame rate determination module and a frame rate adjustment module, wherein: The code rate determination module is configured to obtain an estimated bandwidth corresponding to a transmission link and a sending buffer size corresponding to a sending queue, determine a reserved cache data size based on the estimated bandwidth, the sending buffer size and set delay limit information, and determine an encodable code rate based on the reserved cache data size, the estimated bandwidth and set code rate control information, wherein the delay limit information is used to indicate a limit of data sending delay, and the code rate control information is used to indicate an expected time precision of controlling the code rate; The frame rate determination module is configured to obtain a single frame image size corresponding to encoding processing of an encoder, and determine a budget frame rate based on the encodable code rate and the single frame image size. The frame rate adjustment module is configured to perform frame rate adjustment on the collected first collected frame data stream based on the budget frame rate to obtain a second collected frame data stream.

10. An encoding control device characterized by comprising: The method comprises: a memory and one or more processors; the memory is used to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the encoding control method according to any one of claims 1-8.

11. A non-volatile storage medium for storing computer-executable instructions, characterized in that, The computer executable instructions, when executed by a computer processor, are used to perform the encoding control method according to any one of claims 1-8.

12. A computer program product comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the encoding control method according to any one of claims 1-8.

Images