Video encoding bitrate control methods, devices, systems, and storage media
By maintaining the budget state and performing closed-loop correction during video encoding, the problem of failing to allocate bitrate based on differences in image content in existing technologies is solved. This achieves high-quality encoding of key target areas and reasonable resource allocation, thereby improving encoding quality and stability.
Patent Information
- Application Number
- CN202611223176.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-12
- Publication Date
- 2026-11-17
AI Technical Summary
Existing video coding methods fail to effectively allocate bitrates based on the spatial differences in the content of the video, resulting in insufficient coding quality in key target areas or excessive coding bits consumed in background areas, making it difficult to meet the high-quality coding requirements in practical applications.
By maintaining the budget state within a continuous time window, and performing closed-loop correction on the initial quantization parameter offset mapping based on the budget state and preset target priority, differentiated rate recovery and protection enhancement are achieved, ensuring that the actual number of encoded bits tends to the target value and prioritizing the quality of high-priority regions.
It improves the long-term stability and coding quality of bitrate control, rationally allocates coding resources, and ensures that key target areas achieve higher coding quality under bitrate constraints.
Smart Images

Figure CN123123576A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of video coding technology, specifically to a video coding bitrate control method, apparatus, system, and storage medium. Background Technology
[0002] Currently, in video encoding applications such as video surveillance, vehicle recording, and edge computing, end users typically pay more attention to key targets in the frame, while paying relatively less attention to ordinary background areas. However, traditional video encoding bitrate control methods usually allocate bits on a frame-by-frame basis, without tailoring the bitrate to the spatial differences in the scene content. This results in insufficient encoding quality for key target areas or excessive unnecessary bit consumption in background areas, making it difficult to meet the high-quality encoding requirements for key targets in practical applications.
[0003] To address the aforementioned issues, various video coding bitrate control schemes based on regions of interest (ROIs) have been proposed. These include differentiating quantization parameter allocation by defining ROIs based on motion and texture information according to the human eye's selective attention mechanism, performing hierarchical quantization of ROIs based on a minimum perceptible distortion model, performing pixel-level semantic segmentation on the video frame sequence to extract foreground ROIs for bitrate redistribution, and image coding with bitrate redistribution based on prior target information obtained according to the machine vision task type.
[0004] In the process of implementing the embodiments of this application, at least the following problems were found in the related technology: Related technologies have achieved bitrate skew allocation based on regions of interest to some extent, but the deviation between the actual number of encoded bits and the target value persists across multiple frames. This deviation is further aggravated when the area of the region of interest, the number of targets, or the texture complexity changes; and regions of different importance are adjusted in the same way when the bitrate deviates. Summary of the Invention
[0005] This application provides a video encoding bitrate control method, apparatus, system, and storage medium.
[0006] A first aspect of this application provides a video encoding bitrate control method, comprising: generating an initial quantization parameter offset mapping for the current frame based on target region information of the video frame to be encoded; maintaining a budget state within a continuous time window, and correcting the initial quantization parameter offset mapping based on the budget state and a preset target priority; wherein the correction includes: performing differentiated bitrate reclamation for different priority regions when the budget state indicates that the bitrate budget is exceeded; performing differentiated protection enhancement for different priority regions when the budget state indicates that there is a bitrate margin; transmitting the corrected quantization parameter offset mapping to the encoder, and obtaining the actual number of bits after encoding to update the budget state.
[0007] A second aspect of this application provides a video encoding bitrate control device, including a processor and a memory storing program instructions. The processor is configured to execute the video encoding bitrate control method as described in the first aspect of this application when running the program instructions.
[0008] A third aspect of the present application provides a video encoding bitrate control system, comprising: a video encoding bitrate control system body; and a video encoding bitrate control device as described in the second aspect of the present application, which is installed on the video encoding bitrate control system body.
[0009] A fourth aspect of the embodiments of this application provides a computer-readable storage medium storing program instructions, which, when executed, cause a computer to perform a video encoding bitrate control method as described in the first aspect of the embodiments of this application.
[0010] The video encoding bitrate control method, apparatus, system, and storage medium provided in the embodiments of this application have the following beneficial effects: This application embodiment maintains a budget state, including the cumulative deviation between the actual number of encoded bits and the target number of encoded bits, within a continuous time window. Based on this, it performs closed-loop correction on the initial quantization parameter offset mapping. This ensures that the actual number of encoded bits tends towards the target value within the time window, avoiding long-term bitrate deviations caused by independent decisions in a single frame, and effectively improving the long-term stability of bitrate control. By performing differentiated bitrate recovery or protection enhancement operations on different priority regions according to preset target priorities, targeted adjustments can be made according to the importance of the regions when bitrate fluctuates. When the bitrate exceeds the limit, the quality of high-priority regions is prioritized; when there is bitrate margin, the protection of high-priority regions is enhanced. This allows for a more rational allocation of limited coding resources under bitrate constraints, improving the coding quality of critical target regions. Attached Figure Description
[0011] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram of a video encoding bitrate control method provided in an embodiment of this application; Figure 2 This is a schematic diagram of another video encoding bitrate control method provided in an embodiment of this application; Figure 3 This is a schematic diagram of a video encoding bitrate control device provided in an embodiment of this application.
[0012] Figure label: 800: Video encoding bitrate control device; 801: Processor; 802: Memory; 803: Communication interface; 804: Bus. Detailed Implementation
[0013] To make the technical solutions and advantages of the embodiments of this application clearer, the exemplary embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.
[0014] This application proposes a video coding bitrate control system, including a video coding bitrate control system body and a video coding bitrate control device 800 described below. The video coding bitrate control system body includes a target detection module, an initial quantization parameter offset mapping generation module, a budget state statistics module, a budget feedback control module, a timing stabilization module, and an encoder interface module. The target detection module detects targets in video frames and outputs target region information. The initial quantization parameter offset mapping generation module generates the initial quantization parameter offset mapping for the current frame based on the target region information. The budget state statistics module maintains the budget state within a continuous time window, which includes the cumulative deviation between the actual number of encoded bits and the target number of encoded bits. The budget feedback control module performs closed-loop correction of the initial quantization parameter offset mapping based on the budget state and a preset target priority. When the cumulative deviation indicates that the bitrate budget is exceeded, differentiated bitrate recovery is performed for different priority regions; when the cumulative deviation indicates that there is a bitrate margin, differentiated protection enhancement is performed for different priority regions. The timing stabilization module performs inter-frame smoothing and variation amplitude limitation on the corrected quantization parameter offset mapping. The encoder interface module is used to transmit the quantization parameter offset mapping processed by the timing stabilization module to the encoder. The video encoding rate control device 800 includes a processor 801, electrically connected to the aforementioned modules, for controlling the modules to perform corresponding actions. The video encoding rate control device 800 is installed within the main body of the video encoding rate control system. The installation relationship described herein is not limited to placement within the video encoding rate control system, but also includes installation connections with other components of the video encoding rate control system, including but not limited to physical connections, electrical connections, or signal transmission connections. Those skilled in the art will understand that the video encoding rate control device 800 can be adapted to any feasible video encoding rate control system main body to achieve other feasible embodiments.
[0015] Figure 1 and Figure 2 This is a schematic diagram of the video encoding bitrate control method provided in the embodiments of this application. Any of the following methods can be executed in the video encoding bitrate control system, or in a server or terminal device communicatively connected to the video encoding bitrate control system. Based on the structure of the video encoding bitrate control system described above, as follows... Figure 1 As shown, this application provides a video encoding bitrate control method, including: S01, Generate the initial quantization parameter offset mapping for the current frame based on the target region information of the video frame to be encoded.
[0016] S02, maintain the budget state within a continuous time window, and correct the initial quantization parameter offset mapping according to the budget state and the preset target priority; wherein, the correction includes: when the budget state indicates that the bitrate budget is exceeded, perform differentiated bitrate recovery for different priority regions; when the budget state indicates that there is bitrate margin, perform differentiated protection enhancement for different priority regions.
[0017] S03, pass the corrected quantization parameter offset mapping to the encoder, and obtain the actual number of bits after encoding to update the budget state.
[0018] In this embodiment, the target region information in step S01 is obtained by performing target detection on the current video frame to be encoded using a target detection model. The target detection model outputs the target category, target bounding box coordinates, and detection confidence of each target in the current video frame. After obtaining the target region information, the initial quantization parameter offset mapping generation module generates an initial quantization parameter offset mapping that matches the encoder's coding block grid size based on the spatial position of the target bounding box in the video frame. This initial quantization parameter offset mapping is a two-dimensional array, where each element corresponds to the quantization parameter offset value of a coding block. This offset value represents the increase or decrease in the basic quantization parameter output by the encoder's original bitrate control module. For coding blocks covered by target bounding boxes, the initial quantization parameter offset mapping generation module assigns a negative offset value, i.e., reduces the quantization parameter to improve coding quality; for background coding blocks not covered by any target bounding boxes, the initial quantization parameter offset mapping generation module assigns a zero offset value or a positive offset value. The continuous time window in step S02 includes a preset number of frames, which can be flexibly set according to the application scenario and coding frame rate. The budget status statistics module continuously collects and updates various statistical data within the continuous time window, including but not limited to the cumulative deviation between the actual number of encoded bits and the target number of encoded bits. The budget feedback control module, based on the budget status information provided by the budget status statistics module, determines whether the bitrate usage within the current window exceeds the budget or has a margin, and corrects the initial quantization parameter offset mapping according to the preset target priority. When the budget status indicates that the bitrate budget is exceeded, the budget feedback control module gradually recovers the bitrate in order of priority from low to high; when the budget status indicates that there is a bitrate margin, the budget feedback control module gradually strengthens the protection in order of priority from high to low. In step S03, the encoder interface module transmits the corrected quantization parameter offset mapping to the encoder in a format supported by the encoder. The encoder superimposes this quantization parameter offset mapping on the basic quantization parameters output by the original bitrate control module, and encodes each coding block using the final quantization parameters. After encoding, the encoder interface module obtains the actual number of encoded bits for the current frame and feeds this value back to the budget status statistics module, which updates the budget status, providing data support for closed-loop control of subsequent frames.
[0019] The video coding bitrate control method provided in this application maintains a budgeted state, such as the cumulative deviation between the actual number of encoded bits and the target number of encoded bits, within a continuous time window. Based on this budgeted state, the initial quantization parameter offset mapping is corrected in a closed loop. This ensures that the actual number of encoded bits tends towards the target value within the time window, avoiding long-term bitrate deviations caused by independent decisions in a single frame, and effectively improving the long-term stability of bitrate control. By performing differentiated bitrate recovery or protection enhancement operations on different priority regions according to preset target priorities, targeted adjustments can be made according to the importance of the region when bitrate fluctuates. When the bitrate exceeds the limit, the quality of high-priority regions is prioritized; when there is bitrate margin, the protection of high-priority regions is enhanced. This allows for a more reasonable allocation of limited coding resources under bitrate constraints, improving the coding quality of critical target regions.
[0020] In an optional embodiment of this application, the budget state includes a cumulative deviation amount; the cumulative deviation amount of the current frame is determined based on the difference between the actual number of encoded bits in the current frame and the target number of encoded bits in the current frame, and the cumulative deviation amount of the previous frame.
[0021] In this embodiment of the application, the cumulative deviation of the current frame is determined by the following formula:
[0022] in, Cumulative deviation for the current frame The cumulative deviation of the previous frame The actual number of encoded bits in the current frame The number of bits encoded for the target in the current frame.
[0023] In the embodiments of this application, the cumulative deviation amount It is a scalar value that accumulates recursively over time. Its physical meaning is the cumulative difference between the number of coded bits actually consumed and the target number of coded bits allocated up to the current frame. The initial value of the cumulative deviation. Set to 0. Target number of encoded bits. It is calculated based on the preset target bitrate and video frame rate. After encoding each frame, the encoder interface module will output the actual number of encoded bits. The feedback is sent to the budget status statistics module, which updates the cumulative deviation based on the above formula. This cumulative deviation is a signed number; a positive value indicates that the cumulative number of bits consumed up to the current frame exceeds the cumulative number of allocated bits, i.e., the bitrate is over-limited; a negative value indicates that the cumulative number of bits consumed is less than the cumulative number of allocated bits, i.e., there is bitrate margin. The budget feedback control module uses this cumulative deviation as one of the core criteria for judging the bitrate usage status when correcting the quantization parameter offset mapping for subsequent frames.
[0024] In this way, by recursively calculating the cumulative deviation, a quantitative memory of bitrate usage can be formed between consecutive frames. This allows the budget feedback control module to perceive the overall trend of bitrate deviation over a period of time, thereby providing a clear direction and magnitude basis for adjusting the quantization parameters of subsequent frames, avoiding control lag or overreaction caused by relying solely on information from a single frame.
[0025] In an optional embodiment of this application, the budget state further includes a compensation budget difference; the compensation budget difference for the current frame is determined based on the difference between the estimated additional bit consumption of the target region and the estimated bit saving of the background region in the current frame, and the compensation budget difference of the previous frame.
[0026] In this embodiment of the application, the compensation budget difference of the current frame is determined by the following formula:
[0027] in, Compensation budget difference for the current frame Compensation budget difference for the previous frame Additional bit consumption for estimating the target region in the current frame Saves estimated bits for the background region in the current frame.
[0028] In this embodiment of the application, the compensation budget difference This is used to characterize the cumulative difference between the number of bits theoretically consumed due to improved target region quality and the number of bits saved due to reduced background region quality after differential quantization parameter allocation for the target and background regions. Estimated additional bit consumption for the target region. Rate-distortion estimation is performed using a rate-distortion model. This is calculated based on the texture complexity, motion intensity, and assigned quantization parameter offsets of all coded blocks in the target region, using a pre-fitted rate-distortion curve. One estimation method involves, for each coded block within the target region, querying a pre-generated rate-distortion lookup table based on its initial quantization parameter offset and texture features to obtain the bit count change before and after the offset. Then, the bit count changes of all coded blocks within the target region are summed to obtain the result. Estimated bit savings in the background region The estimation method and The same, but it applies to the sum of bit reductions across all coded blocks in the background region after applying a positive quantization parameter offset. Initial value for compensating for budget differences. Set to 0. Compensate for budget difference. A positive value indicates that the cumulative estimated additional bit consumption is greater than the cumulative estimated bit saving, suggesting that the target enhancement strategy generally tends to consume more bits; a negative value indicates that the cumulative estimated bit saving is greater than the cumulative estimated additional bit consumption, suggesting that the background compensation strategy generally tends to save more bits. This compensation budget difference is related to the cumulative bias. They complement each other. The cumulative deviation reflects the actual number of encoded bits deviating from the target, while the compensation budget difference reflects the degree of budget bias determined by the differentiated quantization parameter allocation strategy itself. The combination of the two provides the budget feedback control module with more comprehensive decision-making information.
[0029] In this way, by maintaining the compensation budget difference, the budget feedback control module can distinguish whether the bitrate deviation is caused by excessive enhancement of the target area or insufficient compensation of the background area, so as to determine the correction direction more effectively and avoid blindly weakening the target area and enhancing the background area at the same time when the bitrate exceeds the limit, thereby improving the accuracy of bitrate adjustment.
[0030] In an optional embodiment of this application, the continuous time window includes a preset number of frames; the difference between the actual number of encoded bits in each frame within the preset number of frames and the target number of encoded bits in the corresponding frame is accumulated to obtain the cumulative budget debt of the window; the difference between the estimated additional bit consumption of the target region and the estimated bit saving of the background region in each frame within the preset number of frames is accumulated to obtain the cumulative compensation budget difference of the window.
[0031] In this embodiment of the application, the cumulative budgetary debt of the window and the cumulative compensation budgetary difference of the window are determined by the following formulas:
[0032]
[0033] in, Accumulated budget debt for the window Accumulated compensation for budget difference for the window The number of frames contained in a continuous time window Frame index within the window For the first Actual number of encoded bits per frame For the first Number of target encoding bits per frame For the first Estimated extra bit consumption of target region in frame For the first Estimated bit savings for background regions in the frame.
[0034] In this embodiment of the application, the window accumulates budgetary debt. The difference between the cumulative compensation budget and the window It is obtained by accumulating the above differences within a sliding window of fixed length. The value is set according to the actual needs of the application scenario, with a typical value between 15 and 30 frames. The advantage of using a fixed window length for accumulation is that the accumulated budget debt of the window will not accumulate indefinitely, but only reflects the most recent budget. The system monitors bitrate deviations within frames, enabling the budget feedback control module to respond promptly to changes in the current encoding state. After processing each frame, the sliding window moves forward one frame, removing the oldest frame and adding the newest, recalculating the window's cumulative budget debt and cumulative budget compensation difference. In the initial stage before the window is full, only existing frames are accumulated. A first and second preset threshold are proportional to the number of frames contained in the continuous time window. Specifically, the first preset threshold is set as a percentage of the product of the target bitrate and the number of frames in the window, and the second preset threshold is set as the opposite of the first preset threshold.
[0035] In this way, by calculating the cumulative budget debt and the cumulative compensation budget difference of the window through sliding window accumulation, the smoothness of multi-frame statistics is preserved, the interference of single-frame noise on control decisions is avoided, and the recent bitrate trend can be reflected in a timely manner. This allows the budget feedback control module to make decisions based on the overall state over a recent period of time, thereby improving the robustness and timeliness of bitrate control.
[0036] In an optional embodiment of this application, the correction further includes: determining a global quantization parameter correction offset based on the window's cumulative budget debt; setting the global quantization parameter correction offset to a positive value when the window's cumulative budget debt exceeds a first preset threshold; and setting the global quantization parameter correction offset to a negative value when the window's cumulative budget debt is lower than a second preset threshold; wherein the first preset threshold and the second preset threshold are proportional to the number of frames contained in the continuous time window; and the global quantization parameter correction offset is a uniform quantization parameter correction offset applied to all coding blocks.
[0037] In this embodiment, the global quantization parameter correction offset is determined by the following formula:
[0038] in, Correct offset for global quantization parameters Accumulated budget debt for the window For the budget feedback control function, when When the value exceeds the first preset threshold, a positive value is output. When the value is below the second preset threshold, a negative value is output.
[0039] In the embodiments of this application, the budget feedback control function Multiple implementation methods are adopted. In one implementation method, For a piecewise linear function, when When it is within the dead zone between the first preset threshold and the second preset threshold, The value is 0; when When the first preset threshold is exceeded, The offset is gradually increased at a slope proportional to the excess amount until it reaches the preset upper limit of the global quantization parameter correction; when When it is below the second preset threshold, The offset is gradually decreased according to a slope proportional to the margin until it reaches the preset lower limit of the global quantization parameter correction offset. In another implementation, For a control function with hysteresis characteristics, when When the value exceeds the first preset threshold from within the dead zone in the positive direction. Jump to a preset positive value to quickly respond to bitrate exceeding limits; when When the price falls from a positive value to below the first preset threshold but has not yet entered the dead zone... Maintain this positive value until The threshold is lowered to below a first preset threshold to prevent frequent switching. In another implementation, For integral control functions or proportional-integral control functions, based on window cumulative budget debt Calculate using its rate of change .
[0040] In this way, by determining the global quantization parameter correction offset based on the cumulative budget debt of the window, and applying the offset uniformly to all coding blocks, it is possible to achieve rapid coarse adjustment of the whole frame bitrate with a small computational cost, so that the bitrate can quickly return to the vicinity of the target. Combining global correction with subsequent regional differential adjustment forms a two-level bitrate control architecture that combines coarse and fine adjustment, which ensures both response speed and adjustment accuracy.
[0041] In an optional embodiment of this application, differential bitrate recovery for different priority regions includes: progressively increasing the quantization parameter offset value in the order of background region, low-priority target region, and high-priority target region; and differential protection enhancement for different priority regions includes: progressively decreasing the quantization parameter offset value in the order of high-priority target region, low-priority target region, and background region.
[0042] In this embodiment, the high-priority target area includes the license plate area, and the low-priority target area includes the face area or the human body area; the category priority of the license plate area is higher than that of the face area, and the category priority of the face area is higher than that of the human body area.
[0043] In this embodiment of the application, differentiated bitrate recovery refers to the cumulative budgeted debt within a window. When the bitrate is positive and exceeds the first preset threshold, the budget feedback control module needs to reclaim bitrate from certain regions to reduce the coded bit consumption of the current frame or subsequent frames. The specific operation for reclaiming bitrate is to increase the quantization parameter offset value of the corresponding region. The order of increasing the quantization parameter offset value is from low to high priority; that is, first increase the quantization parameter offset value of all coded blocks in the background region. If simply increasing the quantization parameter offset value of the background region is insufficient... If the priority level falls back to an acceptable range, the quantization parameter offset values for face and body regions in low-priority target areas are further increased, and only lastly are the quantization parameter offset values for license plate regions in high-priority target areas considered for increase. Within each level, further granular prioritization is applied. For example, in low-priority target areas, the priority of the body region is lower than that of the face region; therefore, the quantization parameter offset value for the body region is increased first, followed by the face region. Differential protection enhancement refers to the cumulative budget debt within the window. When the value is negative and below the second preset threshold, the budget feedback control module utilizes the bitrate margin to enhance the encoding quality of certain areas. The specific operation of this enhanced protection is to reduce the quantization parameter offset value of the corresponding area. The order of reducing the quantization parameter offset value follows a priority order from high to low: first, the quantization parameter offset value of the license plate area is reduced; then, the quantization parameter offset value of the face area is reduced; and finally, the quantization parameter offset values of low-priority target areas and background areas are considered.
[0044] In this way, by performing bitrate reclamation and protection enhancement step by step according to a clear priority order, high-value targets can be protected to the greatest extent when the bitrate exceeds the limit. The cost of bitrate reduction is preferentially allocated to background areas and low-priority targets with lower perceived importance. When there is a bitrate margin, the bitrate gain is preferentially allocated to high-priority targets with higher perceived importance, thereby achieving the optimal allocation of coding quality under the target bitrate constraint.
[0045] In an optional embodiment of this application, the method further includes: obtaining downstream task quality feedback, which includes at least one of license plate optical character recognition confidence, face recognition confidence, target detection confidence, or target tracking stability; when the downstream task quality feedback is lower than a preset quality threshold, adding a negative vectorization parameter correction value to the corresponding target category; when the budget status indicates that the bitrate budget is exceeded and the downstream task quality feedback is lower than the preset quality threshold, prioritizing the recovery of bitrate from the background area or low-priority target area.
[0046] In this embodiment, downstream task quality feedback is a quality metric output by the downstream visual task module from the decoding result after video encoding. License plate optical character recognition confidence refers to the reliability of the recognition result after performing optical character recognition on the license plate area. Face recognition confidence refers to the reliability of the matching result after performing face recognition or face verification on the face area. Target detection confidence refers to the detection score output by the target detection model for each target in the current frame. Target tracking stability refers to whether the tracking state of the same target is stable across multiple consecutive frames. After obtaining the above downstream task quality feedback, the budget feedback control module compares each quality feedback with its corresponding preset quality threshold. When the downstream task quality feedback for a certain target category is lower than the corresponding preset quality threshold, it indicates that the quality currently allocated to that category of targets is insufficient to support the normal operation of downstream tasks. Therefore, a negative quantization parameter correction value needs to be added to that category of targets, i.e., the quantization parameter offset value of that category of targets is further reduced to improve its encoding quality. When the budget status indicates that the bitrate budget has been exceeded and the downstream task quality feedback is below a preset quality threshold, the budget feedback control module prioritizes recovering bitrate from background regions or low-priority target regions, rather than directly reducing the quality of target categories with insufficient quality. This ensures that the encoding quality of critical targets in downstream tasks is maintained even when the bitrate budget is tight. Specifically, the budget feedback control module first attempts to recover bitrate by increasing the quantization parameter offset value in the background region. If the recovery space in the background region is insufficient, it further attempts to increase the quantization parameter offset value of non-critical categories in low-priority target regions. Only when the above measures still cannot meet the budget requirements will it consider moderately recovering bitrate from categories whose downstream task quality feedback is below the threshold.
[0047] In this way, by introducing downstream task quality feedback, a dual closed-loop control between encoding quality and task availability is formed, so that the goal of bit rate control is no longer limited to the precise matching of bit count, but extends to the actual support capability of the encoding result for downstream vision tasks.
[0048] In an optional embodiment of this application, the corrected target region quantization parameter offset value is determined based on the initial quantization parameter offset value of the target region, the global quantization parameter correction offset, and the additional quantization parameter correction based on downstream task quality feedback; the corrected background region quantization parameter offset value is determined based on the initial quantization parameter offset value of the background region, the global quantization parameter correction offset, and the additional correction based on the texture complexity or motion intensity of the background region; the corrected target region quantization parameter offset value and the corrected background region quantization parameter offset value together constitute the corrected quantization parameter offset mapping, which is used to perform differentiated bitrate reclamation for regions of different priorities or differentiated protection enhancement for regions of different priorities.
[0049] In this embodiment of the application, the corrected target region quantization parameter offset value is determined by the following formula:
[0050] in, For the revised first quantization parameter offset value of each target region For the initial quantization parameter offset mapping, the first Initial quantization parameter offset value for each target region Correct offset for global quantization parameters To address the issue based on downstream task quality feedback Additional quantization parameter correction for the category to which each target belongs.
[0051] In this embodiment, the corrected background region quantization parameter offset value is determined by the following formula:
[0052] in, For the corrected grid coordinates Quantization parameter offset of the background block For the initial quantization parameter offset mapping located in the grid coordinates The initial quantization parameter offset value of the background block Correct offset for global quantization parameters Based on grid coordinates Additional corrections to the texture complexity or motion intensity of the background blocks.
[0053] In this embodiment, the corrected quantization parameter offset value of the target region is composed of three superimposed parts. The first part is the initial quantization parameter offset value. The first part reflects the basic offset allocated to the target region by the target detection module and the initial quantization parameter offset mapping generation module based on information such as target category and target size. The second part is the global quantization parameter correction offset. This applies uniformly to all target and background regions to achieve coarse adjustment of the overall frame bitrate. The third part is the additional quantization parameter correction based on downstream task quality feedback. It makes differentiated adjustments based on specific target categories, and when the quality of downstream tasks is insufficient... Taking negative values enhances protection when downstream task quality is sufficient and bitrate is tight. Take a positive value to appropriately weaken the protection. The value of is determined through a proportional control method, that is This equals the difference between the downstream task quality feedback and the preset quality threshold multiplied by a preset scaling factor. The corrected quantization parameter offset value for the background region is also composed of three superimposed parts, among which... This is an additional correction based on the characteristics of the background block itself. Texture complexity is measured by the contrast, variance, or mean gradient magnitude of the gray-level co-occurrence matrix of the coded block, while motion intensity is measured by the sum of the absolute values of the pixel differences between corresponding coded blocks in consecutive frames. For coded blocks classified as low-value backgrounds, i.e., background blocks with low texture complexity and low motion intensity, their initial quantization parameter offset is set to a higher positive value; for high-motion or high-texture background blocks, their initial quantization parameter offset is set to a lower value, such as 0.
[0054] In this way, by decomposing the corrected quantization parameter offset value into three independent components—initial offset, global correction, and additional correction—the adjustment target and basis of each component are clear and explicit, facilitating independent debugging and optimization. At the same time, the superposition relationship between the components ensures the interpretability and predictability of the adjustment.
[0055] In an optional embodiment of this application, the preset target priority is determined by the target importance score; the target importance score is determined based on the target category weight, the target detection confidence weight, the target size weight, and the target temporal stability weight.
[0056] In this embodiment of the application, the target importance score is determined by the following formula:
[0057] in, For the first Importance score of each goal For the first Category weights of the target categories to which each target belongs For the first Detection confidence weights for each target For the first Target size weight of each target For the first Temporal stability weights for each objective.
[0058] In the embodiments of this application, the target importance score This is a dimensionless positive number used to prioritize all targets detected in the current frame. A higher score indicates that the target should receive a higher protection priority in bitrate allocation. (Class weight) This reflects the inherent importance of different target categories in the application scenario. In security monitoring scenarios, license plates are assigned the highest category weight because they carry key information for vehicle identification; faces are assigned the second highest category weight because they carry key information for personnel identification. The specific values of the category weights are flexibly adjusted according to the different levels of attention given to each category of targets in different application scenarios. Detection confidence weight. The value of is directly derived from the detection confidence score output by the object detection model. Size weight The function reflecting the impact of target size on importance is set to be inversely or directly proportional to the target size. In one implementation, since small targets are more prone to losing details during encoding, a specific setting is used. The weight is inversely proportional to the square root of the target bounding box area, giving higher weight to smaller targets; in another implementation, for targets like license plates that require high optical character recognition accuracy, a setting is used... Proportional to the target size. Temporal stability weight. The determination is based on the target's appearance and positional stability across multiple consecutive frames. For targets that can be stably tracked for more than a preset number of frames using the target tracking algorithm... Take the higher value; for newly emerging or unstable targets, Take the lower value.
[0059] In this way, the quantification and configurability of target priority are achieved through target importance scores, enabling the budget feedback control module to make priority judgments based on a unified calculation framework in different scenarios. This ensures both the universality of the method and the flexibility to customize configurations for specific scenarios.
[0060] Based on the structure of the video encoding bitrate control system described above, such as Figure 2 As shown in the embodiment of this application, another video encoding bitrate control method is provided, including: S01, Generate the initial quantization parameter offset mapping for the current frame based on the target region information of the video frame to be encoded.
[0061] S02, maintain the budget state within a continuous time window, and correct the initial quantization parameter offset mapping according to the budget state and the preset target priority; wherein, the correction includes: when the budget state indicates that the bitrate budget is exceeded, perform differentiated bitrate recovery for different priority regions; when the budget state indicates that there is bitrate margin, perform differentiated protection enhancement for different priority regions.
[0062] S21, perform timing smoothing on the corrected quantization parameter offset mapping.
[0063] S03, pass the corrected quantization parameter offset mapping to the encoder, and obtain the actual number of bits after encoding to update the budget state.
[0064] In this embodiment of the application, temporal smoothing of the corrected quantization parameter offset mapping includes: weighted summation of the quantization parameter offset values at each spatial position in the corrected quantization parameter offset mapping of the current frame and the quantization parameter offset values at the same spatial position in the quantization parameter offset mapping of the previous frame to obtain the quantization parameter offset values at each spatial position in the temporally smoothed quantization parameter offset mapping of the current frame; and limiting the change in the quantization parameter offset values at the same spatial position between adjacent frames to not exceed a preset change threshold.
[0065] In this embodiment of the application, the quantization parameter offset mapping after temporal smoothing of the current frame is located in the grid coordinates. The quantization parameter offset value is determined by the following formula:
[0066] in, The time-smoothed grid coordinates The Frame quantization parameter offset value This is the smoothing coefficient, and its value range is... For the corrected grid coordinates Current frame quantization parameter offset value For grid coordinates The previous frame's quantization parameter offset value; and, the variation range of the quantization parameter offset value at the same spatial location between adjacent frames is limited to not exceeding a preset variation threshold. ,in This represents the maximum allowable variation in the inter-frame quantization parameter offset value.
[0067] In this embodiment, the purpose of temporal smoothing is to suppress drastic changes in the quantization parameter offset mapping over time caused by inter-frame jitter in the target detection box, unstable target detection results, or fluctuations in the budget feedback control output. The temporal stabilization module performs first-order exponential smoothing filtering on the corrected quantization parameter offset mapping; that is, the smoothed offset value of the current frame is obtained by a weighted average of the corrected offset value of the current frame and the smoothed offset value of the previous frame. Smoothing coefficient. The weight of the current frame correction value in the smoothing result is controlled. The larger the value, the more the smoothing result tends to reflect the actual correction value of the current frame; the faster the response, but the lower the smoothness. The smaller the value, the more the smoothing result tends to historical values; the higher the smoothing degree, the slower the response speed. The value can range from 0.3 to 0.7, and the specific value should be adjusted according to the dynamism of the scene. A larger value should be used in scenes with intense movement or rapidly changing targets. Take the smaller value in a static scene After weighted smoothing, the temporal stabilization module also imposes hard limits on the inter-frame variation amplitude at each spatial location, i.e. and The absolute value of the difference must not exceed the preset change threshold. . The typical value is one or two quantization parameter steps. When the change after weighted smoothing exceeds... At that time, Limited to Within the range.
[0068] The video encoding bitrate control method provided in this application, through a combination of first-order exponential smoothing and inter-frame variation amplitude limitation, can effectively suppress the drastic jumps in quantization parameter offset mapping between frames, significantly reduce quality flicker caused by detection box jitter and feedback control fluctuations, and improve the subjective quality stability of encoded video.
[0069] In an optional embodiment of this application, the timing smoothing further includes: for the target region that appears for the first time, directly using the corrected quantization parameter offset value of the current frame; for the target region that disappears in consecutive frames, gradually returning the quantization parameter offset value of the target region to the quantization parameter offset value of the background region frame by frame within a preset number of gradual frames.
[0070] In this embodiment, the first-appearing target region refers to the target first identified by target detection in the current frame. This target did not exist in the previous frame, therefore there is no quantization parameter offset value from the previous frame for reference. In this case, the timing stabilization module directly uses the corrected quantization parameter offset value of the current frame as the quantization parameter offset value of the target region, without performing weighted smoothing. The disappearance of a target in consecutive frames means that a target existed in the previous frame but is no longer detected in the current frame. To avoid a quality jump caused by abrupt changes in the quantization parameter offset value from the target region's offset value to the background region's offset value, the timing stabilization module adopts a gradual regression strategy. That is, starting from the frame where the target disappears, the quantization parameter offset values of each grid coordinate corresponding to the target region are gradually decreased or increased frame by frame within a preset gradual frame number, so that it smoothly transitions from the target region's offset value to the background region's offset value. The preset gradual frame number can be set to 5 to 10 frames. In each gradual frame, the quantization parameter offset value of the region approaches the background region's offset value in a linear interpolation or exponential decay manner, until it is completely equal to the background region's offset value after the preset gradual frame number. Specifically, if in the first If the frame target disappears for the first time, then the offset of that region in subsequent frames is set to [value]. ,in The frame number that starts counting from when the target disappears. To preset the number of gradient frames, This is the offset value before the target disappeared. This is the offset value for the background area.
[0071] In this way, by directly assigning values to targets that appear for the first time and performing gradual regression on targets that disappear, the time-series stabilization module can make reasonable handling when there is a lack of historical reference or when the target state changes abruptly. This avoids smoothing errors caused by a lack of historical data and eliminates quality mutations caused by the sudden disappearance of targets.
[0072] In an optional embodiment of this application, the initial quantization parameter offset mapping is generated by any of the following methods: a mapping generation method based on the negative offset of the target region and the positive offset of the background region of the target detection box; a differential mapping generation method based on pixel-level semantic segmentation of the foreground region and the background region; a fixed region mapping generation method based on manual configuration; a prediction generation method based on the quantization parameter offset mapping of historical frames; and a correction step for correcting the initial quantization parameter offset mapping that is independent of the generation method of the initial quantization parameter offset mapping.
[0073] In this embodiment, the initial quantization parameter offset mapping generation module employs various implementation methods to generate the initial quantization parameter offset mapping. These implementation methods can be flexibly switched without modifying the processing logic of the budget feedback control module and the timing stabilization module. Based on the target detection box generation method, after the target detection module outputs the target bounding box, the initial quantization parameter offset mapping generation module assigns negative quantization parameter offset values to the area covered by the target box and positive quantization parameter offset values to the background area. Based on pixel-level semantic segmentation, the generation method uses a semantic segmentation network to classify video frames pixel by pixel, assigning negative offset values to pixel areas classified as foreground targets and positive offset values to background areas. Based on manually configured fixed region generation, the method is suitable for applications with relatively fixed scenes, where the user directly selects a fixed region in the image and configures the corresponding quantization parameter offset values. Based on historical frame quantization parameter offset mapping, the prediction generation method uses the quantization parameter offset mapping of historical frames and motion vector information to predict the quantization parameter offset mapping of the current frame through motion compensation. The initial quantization parameter offset mapping output by the above various generation methods has a unified data format, namely a two-dimensional array, where each element corresponds to the quantization parameter offset value of a coding block. The budget feedback control module and the timing stability module rely solely on this unified data format for subsequent processing, without caring about the specific method by which the initial mapping was generated.
[0074] In this way, by decoupling the generation process of the initial quantization parameter offset mapping from the budget feedback control process, the budget feedback control module provided in this application can be compatible with a variety of different initial mapping sources. Users can flexibly choose the most suitable initial mapping generation method according to the computing resources, accuracy requirements and real-time requirements of specific application scenarios without modifying the feedback control logic.
[0075] In an optional embodiment of this application, when the target region information is empty, the initial quantization parameter offset mapping is reverted to a uniform offset value for the entire frame or an adaptive offset value based on image texture features and motion features; the correction step of correcting the initial quantization parameter offset mapping continues to perform bitrate reclamation or protection enhancement operations in the background region based on the budget state.
[0076] In this embodiment, the situation where the target region information is empty occurs when there is no target of interest in the video frame, or when the target detection module is temporarily shut down due to computational resource limitations, or when the target detection model does not detect any target in the current frame. In these cases, since there is no target region, the initial quantization parameter offset mapping generation module cannot generate a differentiated mapping based on the target region information, and therefore falls back to one of two backup strategies. The first backup strategy is to use a uniform offset value across the entire frame, that is, to assign the same quantization parameter offset value to all coding blocks in the current frame, which is 0, or to take a global offset value based on the budget status fed back by the budget status statistics module. The second backup strategy is to generate an adaptive offset value based on image texture features and motion features, that is, to assign positive offset values to coding blocks with low texture complexity or low motion intensity to save bitrate, and to assign zero or negative offset values to coding blocks with high texture complexity or high motion intensity to maintain quality. Regardless of which backup strategy is used to generate the initial quantization parameter offset mapping, the budget feedback control module continues to work, and continues to perform bitrate reclamation or protection enhancement operations based on the budget status. Specifically, when the target region information is empty, differentiated bitrate recovery is simplified to uniformly increasing the quantization parameter offset value in the background region to recover bitrate, and differentiated protection enhancement is simplified to uniformly decreasing the quantization parameter offset value in the background region to enhance overall quality. At this time, the budget feedback control module still accumulates budget debt based on the window. Output global quantization parameter correction offset And then superimpose it onto the initial quantization parameter offset map.
[0077] In this way, by providing a fallback strategy when the target area information is empty, the bitrate control method provided in this application can still work normally in scenarios where target detection is temporarily unavailable or there is no target, and the budget feedback control module still maintains closed-loop operation.
[0078] In an optional embodiment of this application, the target region information is obtained through a target detection model, which outputs the target category, target bounding box coordinates, and detection confidence. The conversion from target bounding box coordinates to quantization parameter offset mapping includes: rounding the target bounding box coordinates down to the grid coordinates according to the coding block size; when multiple target boxes cover the same grid coordinates, taking the offset value corresponding to the target with the highest preset target priority among the covered targets; extending a preset number of coding blocks outward from the edge of the target bounding box as a transition band, where the quantization parameter offset value of the transition band is the interpolation between the target region offset value and the background region offset value.
[0079] In this embodiment, the target bounding box coordinates output by the target detection model are represented in pixels, while the quantization parameter offset mapping is a two-dimensional grid with coding blocks as the basic unit. Therefore, it is necessary to convert the pixel-level target bounding box coordinates to grid coordinates. The size of each coding block is... Pixels multiplied pixels, then any pixel coordinates The corresponding grid coordinates are For the top-left pixel coordinates of the target bounding box and bottom right pixel coordinates The corresponding grid coverage area is to , to All coded blocks within the coverage area of this grid are considered to be covered by the target. When multiple target bounding boxes cover the same grid coordinate, i.e., the grid coordinate belongs to the bounding boxes of multiple different targets simultaneously, the target importance score is selected from all targets covering the grid coordinate. For the highest target, the quantization parameter offset value corresponding to that target is used as the offset value for that grid coordinate. At the edge of the target bounding box, to prevent abrupt changes in the quantization parameter offset value between the target region and the background region that could lead to noticeable blockiness, a predetermined number of coded blocks are extended outwards from the edge of the target bounding box as a transition zone. The predetermined number is set to 1 to 3 coded blocks. The quantization parameter offset value of each coded block in the transition zone is the interpolation between the target region offset value and the background region offset value. For each coded block in the transition zone, the shortest distance to the target bounding box and the shortest distance to the background region are calculated, and linear interpolation is performed according to the distance ratio. The closer to the target bounding box, the closer to the target region offset value; the closer to the background region, the closer to the background region offset value.
[0080] In this way, by rounding down the target bounding box coordinates and mapping them to the coding block grid, the consistency between the quantization parameter offset mapping and the coding block division within the encoder is ensured; by taking the value of the highest priority target when multiple targets overlap, the protection requirements of high-priority targets in the target overlap area are ensured not to be covered by low-priority targets; by setting a transition band for interpolation, the sudden drop or rise of quantization parameters at the boundary between the target area and the background area is eliminated, effectively reducing the visual block effect at the coding block boundary.
[0081] Based on the detailed description of the various optional embodiments of the video encoding bitrate control method provided in this application above, a complete embodiment is given below for ease of understanding: Taking a security camera scenario as an example, the input video resolution is... Pixels, encoder uses H.265 constant bitrate mode, target bitrate is 2 megabits per second, video frame rate is 30 frames per second, feedback window The frame rate is set to 30 frames. The object detection module performs object detection on each frame of the input image, outputting face bounding boxes and license plate bounding boxes along with their detection confidence scores. The initial quantization parameter offset mapping generation module generates an initial quantization parameter offset mapping based on the object detection results. Specifically, the initial quantization parameter offset value is set to -3 for the face region, -6 for the license plate region, +1 for low-value background regions, and 0 for high-motion or high-texture background regions. The budget state statistics module obtains the actual number of encoded bits after each frame is encoded, calculates the difference between the actual number of encoded bits and the target number of encoded bits, and updates the cumulative deviation. and window accumulated budget debt .when When the target is exceeded, the budget feedback control module determines the target importance score. The priority of each target in the current frame is determined, and then the quantization parameter offset value of the corresponding region is increased in the following order: background region, low-priority target region, face region, and license plate region, in order to recover the bitrate. When the value falls below the second preset threshold, the budget feedback control module progressively reduces the quantization parameter offset value of the corresponding region in the order of license plate area, face area, and low-priority target area to enhance protection. The corrected quantization parameter offset mapping, after being processed by the timing stabilization module through first-order exponential smoothing and inter-frame variation amplitude limiting, is transmitted to the encoder by the encoder interface module. The encoder superimposes the corrected quantization parameter offset mapping onto the basic quantization parameters to complete the encoding of the current frame and output the actual number of encoded bits. This actual number of encoded bits is fed back to the budget status statistics module to update the budget status, thus forming a complete closed-loop control.
[0082] Combination Figure 3As shown, this application provides a video encoding bitrate control device 800, including a processor 801 and a memory 802. Optionally, the device may further include a communication interface 803 and a bus 804. The processor 801, communication interface 803, and memory 802 can communicate with each other via the bus 804. The communication interface 803 can be used for information transmission. The processor 801 can call logical instructions in the memory 802 to execute the video encoding bitrate control method of the above embodiment. Furthermore, the logical instructions in the memory 802 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. The memory 802, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as the program instructions / modules corresponding to the methods in the embodiments of this application. The processor 801 executes functional applications and data processing by running the program instructions / modules stored in the memory 802, thereby implementing the video encoding bitrate control method of the above embodiment. The memory 802 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and application programs required for at least one function; and the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 802 may include high-speed random access memory and may also include non-volatile memory.
[0083] This application provides a computer-readable storage medium storing computer-executable instructions configured to execute the above-described video encoding bitrate control method.
[0084] The technical solutions of this application embodiment can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this application embodiment. The aforementioned storage medium can be a non-transitory storage medium, including: USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, and other media capable of storing program code.
[0085] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A video encoding bitrate control method, characterized in that, include: Based on the target region information of the video frame to be encoded, generate the initial quantization parameter offset mapping for the current frame; Maintain the budget status within a continuous time window, and correct the initial quantization parameter offset mapping based on the budget status and preset target priority; the correction includes: when the budget status indicates that the bitrate budget is exceeded, perform differentiated bitrate recovery for different priority regions; when the budget status indicates that there is bitrate margin, perform differentiated protection enhancement for different priority regions. The corrected quantization parameter offset mapping is passed to the encoder, and the actual number of bits after encoding is obtained to update the budget state.
2. The method according to claim 1, characterized in that, The budget status includes the cumulative deviation; the cumulative deviation of the current frame is determined based on the difference between the actual number of encoded bits and the target number of encoded bits in the current frame, as well as the cumulative deviation of the previous frame.
3. The method according to claim 1, characterized in that, The budget status also includes the compensation budget difference; the compensation budget difference for the current frame is determined based on the difference between the estimated additional bit consumption of the target region and the estimated bit saving of the background region in the current frame, as well as the compensation budget difference of the previous frame.
4. The method according to claim 1, characterized in that, The continuous time window contains a preset number of frames; the difference between the actual number of encoded bits and the target number of encoded bits in each frame within the preset number of consecutive frames is accumulated to obtain the cumulative budget debt of the window; the difference between the estimated additional bit consumption in the target region and the estimated bit saving in the background region in each frame within the preset number of consecutive frames is accumulated to obtain the cumulative compensation budget difference of the window.
5. The method according to claim 1, characterized in that, The revisions also include: Based on the cumulative budget debt in the window, determine the global quantitative parameter correction offset; When the cumulative budget debt of the window exceeds the first preset threshold, the global quantification parameter correction offset will be set to a positive value; When the cumulative budget debt of the window is lower than the second preset threshold, the global quantitative parameter correction offset will be set to a negative value. The first preset threshold and the second preset threshold are proportional to the number of frames contained in the continuous time window; the global quantization parameter correction offset is a uniform quantization parameter correction offset applied to all coding blocks.
6. The method according to claim 1, characterized in that, Differentiated bitrate recovery for different priority regions includes progressively increasing the quantization parameter offset value in the order of background region, low-priority target region, and high-priority target region. Differential protection enhancements for different priority regions include progressively reducing the quantization parameter offset value in the order of high-priority target region, low-priority target region, and background region.
7. The method according to claim 1, characterized in that, The preset target priority is determined by the target importance score; the target importance score is determined based on the target category weight, target detection confidence weight, target size weight and / or target temporal stability weight.
8. A video encoding bitrate control device, comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to execute the video encoding bitrate control method as described in any one of claims 1 to 7 when running the program instructions.
9. A video encoding bitrate control system, characterized in that, include: The video encoding bitrate control system itself; and, The video encoding bitrate control device as described in claim 8 is installed on the main body of the video encoding bitrate control system.
10. A computer-readable storage medium storing program instructions, characterized in that, When the program instructions are executed, they cause the computer to perform the video encoding bitrate control method as described in any one of claims 1 to 7.
