A method and apparatus for adaptive variable bit rate control

Abstract

The application discloses a kind of self-adapting variable code rate's code rate control method.Motion detection method is used to identify the motion, still feature of image block in video frame, and then the motion intensity of video frame is calculated.The motion intensity of video frame is compared with the upper limit threshold of motion intensity, the lower limit threshold of motion intensity set by user, and the pre-allocation encoding bit number of video frame is calculated according to the comparison result and the maximum code rate, minimum code rate, trigger code rate, code rate fall-back smooth step length set by user.The pre-allocation encoding bit number of video frame is corrected according to the encoding state information of video sequence, and the target encoding bit number of video frame is obtained.The target bit control operation is carried out on video frame according to the target encoding bit number of video frame, and the Lagrange multiplier and quantization parameter of video frame are obtained, which are used in the actual encoding process of video frame.The application can flexibly configure different variable code rate control effect through intuitive and easy-to-understand code rate control parameters.

Description

Technical Field

[0001] This invention relates to a video coding technology, and more particularly to an adaptive variable bitrate control method suitable for video surveillance scenarios. Background Technology

[0002] Video coding is a technique that compresses redundant components in video images and represents video information using as little data as possible. Common video coding standards include HEVC (High Efficiency Video Coding, also known as H.265) and AVC (Advanced Video Coding, also known as H.264).

[0003] Video coding technology uses image blocks as the most basic coding unit. For example, in HEVC, the basic unit of video coding is the CU (Coding Unit). A CU can be an image block of 64 pixels × 64 pixels, 32 pixels × 32 pixels, 16 pixels × 16 pixels, or 8 pixels × 8 pixels. The 64-pixel × 64-pixel image block is also called the LCU (Largest Coding Unit).

[0004] To maximize video compression, common video coding algorithms often employ lossy compression, resulting in differences between the reconstructed video and the original (distortion). HEVC and AVC are both lossy compression algorithms. The coding performance of lossy compression algorithms is jointly measured by the coding bit rate (Rate) and the distortion introduced by the coding. In video coding, the coding bit rate R and coding distortion D are mutually restrictive and contradictory; reducing the coding bit rate R often increases coding distortion D, and reducing coding distortion D often increases coding bit rate R. Rate-Distortion Optimization (RDO) aims to minimize coding distortion D at a given coding bit rate R, or minimize coding bit rate R at a given coding distortion D. Common video coding algorithms employ rate-distortion optimization methods based on Lagrange multipliers λ, which aim to achieve the minimum coding rate-distortion cost (RD Cost) J, where J = D + λ × R.

[0005] In video encoding, each encoding mode has a coding distortion D and a coding bit rate R, and the coding rate-distortion cost J of each encoding mode can be calculated. During video encoding, selecting the encoding mode with the lowest coding rate-distortion cost J yields optimal encoding performance. The process by which the encoder selects the optimal coding mode for each image block to be encoded using rate-distortion optimization methods is called mode decision for the image block to be encoded.

[0006] To remove information redundancy in the spatial and temporal domains, video encoding uses intra-frame coding and inter-frame coding techniques to encode the input video frames. I-frames are encoded using only intra-frame coding. P-frames and B-frames are encoded using a combination of intra-frame and inter-frame coding techniques. The type of frame to be encoded is determined by the user-defined Group of Pictures (GOP) parameter.

[0007] In video surveillance scenarios, I-frames are typically encoded periodically, while most other video frames are encoded as P-frames. This is because I-frames help prevent error propagation, and P-frames have higher encoding efficiency. B-frames, due to their bidirectional reference technology, have greater encoding and decoding latency and are generally not used in scenarios requiring real-time transmission and display, such as video surveillance. Figure 1 It shows the image group structure of a typical video surveillance scene, which only contains I-frames and P-frames, and does not contain B-frames.

[0008] In real-world scenarios, the bandwidth capacity of a video transmission channel is limited. If the video encoding bitrate is too high, exceeding the channel bandwidth capacity, it will cause video transmission congestion or even packet loss. Conversely, if the video encoding bitrate is too low, the channel bandwidth will not be fully utilized, and higher video quality cannot be achieved. Therefore, it is necessary to use bitrate control technology to control the output bitrate of the video encoder to match it with the channel bandwidth capacity.

[0009] Rate control adjusts the encoding parameters of a video encoder to make the encoder's output bitrate equal to a pre-set target bitrate, while minimizing encoding distortion to improve video encoding quality. Common rate control algorithms typically achieve this through two stages: target bit allocation and target bit control.

[0010] Target bit allocation typically occurs at three levels: GOP (Group of Pictures, a set of time-sequential video frames) level, video frame level, and image block level. At the GOP level, target bit allocation is determined by the target bitrate of the video to obtain the target number of coded bits for the GOP. At the video frame level, target bit allocation is determined by the target number of coded bits for each video frame within the GOP. At the image block level, target bit allocation is determined by the target number of coded bits for each video frame and the bit allocation weights for each image block within that video frame.

[0011] Target bit control typically occurs at the video frame level and the image block level. At the video frame level, the Lagrange multiplier λ and quantization parameter QP are calculated based on the target number of coded bits for the video frame. At the image block level, the image block-level Lagrange multiplier λ and quantization parameter QP are calculated based on the target number of coded bits for a given image block, the video frame-level Lagrange multiplier λ and quantization parameter QP of the video frame to which the image block belongs, and the image block-level Lagrange multiplier λ and quantization parameter QP of the previously encoded image block. These parameters are then used in the video coding process for that image block.

[0012] Both the Lagrange multiplier λ and the quantization parameter QP are used at the video frame level and the image block level. The video frame-level Lagrange multiplier λ and quantization parameter QP are used in the target bit allocation and control stages (i.e., video frame-level bitrate control), while the image block-level Lagrange multiplier λ and quantization parameter QP are used in the target bit allocation and control stages (i.e., image block-level bitrate control). The video frame-level Lagrange multiplier is sometimes simply referred to as the video frame-level Lagrange multiplier, frame-level Lagrange multiplier, or video frame Lagrange multiplier. The image block-level Lagrange multiplier is sometimes simply referred to as the image block-level Lagrange multiplier, block-level Lagrange multiplier, or image block Lagrange multiplier. The abbreviations for the quantization parameter follow the same pattern.

[0013] In video surveillance scenarios, users typically only focus on video frames where motion events occur, and generally ignore frames without motion events. In this case, using a constant bitrate to encode the video surveillance scene not only wastes a significant amount of encoding bits on the user-uninterested frames without motion events, but also, because the bitrate is fixed, it cannot be increased promptly when motion events occur that users are interested in, thus failing to improve the encoding quality of these user-focused frames containing motion events.

[0014] Therefore, it is necessary to develop a variable bitrate control method that can automatically detect the presence of motion events based on the content of video frames and adjust the target bitrate of the video frames accordingly. When there are no motion events within a video frame, a smaller target bitrate is allocated to the video frame, keeping the bitrate (i.e., the number of encoded bits) at a lower level to save bandwidth for video storage and transmission. When there are motion events within a video frame, a larger target bitrate can be allocated to the video frame based on the intensity of the motion, keeping the bitrate at a higher level to improve the encoding quality of the video. In this way, both bandwidth is saved and the encoding quality of the video content that users are concerned about is improved.

[0015] Currently, there are several technical solutions available for variable bitrate encoding of video surveillance scenarios.

[0016] A common approach involves users setting MaxBitrate, ChangePos, MinStillPercent, MaxQp, and MinQp, while the video encoder detects the motion / static state of the current scene. MaxBitrate represents the maximum bitrate in a moving scene. ChangePos controls the bitrate threshold at which adjustments to the quantization parameters of video frames begin; it's a percentage relative to MaxBitrate. MinStillPercent represents the minimum bitrate in a static state as a percentage of the bitrate threshold, and MaxBitrate × ChangePos × MinStillPercent represents the minimum bitrate in a static scene. MaxQp and MinQp represent the maximum and minimum quantization parameters that can be used during video frame encoding, respectively. When the encoding bitrate exceeds MaxBitrate × ChangePos, the quantization parameters of the video frame gradually adjust towards MaxQp. If the quantization parameters of a video frame have reached MaxQp, they are fixed at MaxQp, at which point the limiting effect of MaxBitrate is lost, and the encoding bitrate may exceed MaxBitrate. If the encoding bitrate is less than MaxBitrate × ChangePos, the quantization parameters of the video frame gradually adjust towards MinQp. If the quantization parameter of the video frame has reached MinQp, the encoding bitrate has reached its maximum value, and the image quality is the best.

[0017] This scheme achieves variable bitrate encoding, using a lower bitrate when there is no motion and a higher bitrate when there is motion. However, this scheme suffers from drawbacks such as unintuitive user configuration parameters and insufficient flexibility. Furthermore, in this scheme, the configuration parameters MaxQp and MinQp are related to MaxBitrate. If MaxQp and MinQp are not configured appropriately, it is easy for the actual encoded bitrate to exceed the maximum bitrate (MaxBitrate) in motion scenes, or for the actual encoded bitrate to fail to reach the minimum bitrate (MaxBitRate × ChangePos × MinStillPercent) in static scenes. Summary of the Invention

[0018] The technical problem to be solved by this invention is: how to flexibly configure different variable bitrate control effects through intuitive and easy-to-understand bitrate control parameters.

[0019] To address the aforementioned technical problems, this invention proposes an adaptive variable bitrate control method, comprising the following steps: Step S1: Using motion detection to identify the motion and stillness features of image blocks within a video frame, and then calculating the motion intensity of the video frame. Step S2: Comparing the motion intensity of the video frame with the user-set upper limit threshold MovStrength. high MovStrength (lower limit of exercise intensity) low A comparison is made based on the comparison results and the user-set maximum bitrate (BitRate). max Minimum Bitrate min Trigger bitrate (BitRate) trig The pre-allocated number of coding bits for a video frame is calculated using the BitRateStep smoothing step. Step S3: If the motion intensity of the video frame is lower than the lower motion intensity threshold or higher than the upper motion intensity threshold, the pre-allocated number of coding bits for the video frame is corrected according to the coding state information of the video sequence to obtain the target number of coding bits for the video frame. If the motion intensity of the video frame is between the lower and upper motion intensity thresholds, the pre-allocated number of coding bits for the video frame is directly used as the target number of coding bits for the video frame. Step S4: Based on the target number of coding bits for the video frame, target bit control operations are performed on the video frame to obtain the Lagrange multipliers and quantization parameters of the video frame, which are used in the actual coding process of the video frame.

[0020] Furthermore, in step S1, any one or more of the following motion detection methods are used to determine the motion and stillness features of image blocks within a video frame: frame difference method, background subtraction method, optical flow method, Gaussian mixture model (GMM) method, and motion detection method based on machine learning.

[0021] For example, in step S1, the frame difference method for determining the motion and stillness features of image blocks within a video frame specifically includes the following: First, the video frame is divided into multiple image blocks of the same size, and each image block is further divided into multiple image sub-blocks of the same size. Each image sub-block contains the same number of pixels. The frame difference is calculated between the image blocks within the video frame and the image blocks at the same position within the reference frame. The brightness difference value of each pixel in the image blocks within the video frame is calculated and compared with a motion pixel determination threshold. If the brightness difference value of a pixel is greater than the motion pixel determination threshold, the pixel is marked as a moving pixel; otherwise, it is marked as a still pixel. If the number of moving pixels in an image sub-block is greater than the motion image sub-block determination threshold, the image sub-block is marked as a moving image sub-block; otherwise, it is marked as a still image sub-block. If the number of moving image sub-blocks in an image block is greater than the motion image block determination threshold, the image block is marked as a moving image block; otherwise, it is marked as a still image block.

[0022] Furthermore, in step S1, after determining the motion and stillness features of each image block within the video frame, the motion strength (MovStrength) of the video frame is calculated. Block mov Block represents the number of moving image blocks in a video frame. total This indicates the total number of image blocks in a video frame.

[0023] Furthermore, in step S2, the lower limit threshold of motion intensity is MovStrength. low The physical meaning is: when the motion intensity of a video frame is greater than or equal to the lower limit threshold of motion intensity, it is determined that a motion event has occurred in the video frame; otherwise, it is determined that no motion event has occurred in the video frame; when no motion event has occurred in the video frame, the video encoding bitrate needs to be maintained at the minimum bitrate set by the user (BitRate). min MovStrength (Upper Threshold of Exercise Intensity) high The physical meaning is: when the motion intensity of a video frame exceeds the upper limit threshold of motion intensity, the video encoding bitrate needs to be maintained at the maximum bitrate set by the user (BitRate). max Maximum bitrate (BitRate) max The physical meaning is: the maximum coding bit rate that can be achieved when there is motion in a video frame; the minimum bit rate (BitRate). min The physical meaning is: the minimum coding bit rate that needs to be maintained when no motion events occur in a video frame; it needs to satisfy BitRate. min <BitRate max Trigger bitrate (BitRate) trigThe physical meaning is: the minimum coding bit rate required when motion events occur in a video frame; it needs to satisfy the BitRate. min ≤BitRate trig ≤BitRate max The physical meaning of BitRateStep, the bitrate fallback smoothing step, is: when motion events disappear in a video frame, the video coding bitrate of the subsequent video frames needs to gradually decrease from the bitrate when motion events occurred to the minimum bitrate BitRate when no motion events occurred. min The value of BitRateStep is the number of video frames used to smooth the video encoding quality.

[0024] Further, step S2 specifically includes the following: (1) If MovStrength < MovStrength low If no motion events occur in the video frame, the target encoding bitrate (BitRate) is maintained at BitRate. min (2) If MovStrength low ≤MovStrength≤MovStrength high The system determines that motion events have occurred in the video frame, and the target encoding bitrate (BitRate) is adjusted from BitRate. min Upgrade to BitRate trig and BitRate max Between these values, the specific value is directly proportional to the value of MovStrength. (3) If MovStrength > MovStrength high If motion events are detected in a video frame, the target encoding bitrate is maintained at BitRate. max (4) If a motion event exists in the previous video frame but disappears in the current video frame, the target bitrate (BitRate) of the current video frame and the subsequent several consecutive video frames needs to be calculated from the bitrate (BitRate) of the previous video frame. rec Gradually decrease to BitRate min This stage is called the transition stage after the motion event disappears; if the current video frame is the first frame used to smooth the video coding quality, then the target coding bitrate (BitRate) for the m-th frame used to smooth the video coding quality is: (5) During the transition phase after the motion event disappears, if a motion event occurs again in a video frame, the transition phase is immediately terminated. The target coding bitrate (BitRate) for the video frame is calculated according to (2) or (3) based on the MovStrength of the video frame. After calculating the target coding bitrate (BitRate) based on the motion intensity of the video frame, the pre-allocated coding bit count (FrameBits) for the video frame is then calculated. pre ; Use FrameRate to represent the encoding frame rate.

[0025] Furthermore, in step S2, MovStrength low and MovStrength high The smaller the value, the more sensitive the bitrate control is to motion events; MovStrength low and MovStrength high The higher the value, the less sensitive the bitrate control is to motion events.

[0026] Further, in step S2, when BitRate min <BitRate> trig <BitRate> max When a motion event occurs, the video frame encoding bitrate changes from BitRate. min Step to BitRate trig Then, based on the motion intensity of the video frames in the BitRate... trig and BitRate max The bitrate is adjusted linearly between BitRate and BitRate; when the motion event disappears, the video frame coding bitrate changes from BitRate. rec Gradually decrease to BitRate min When BitRate min =BitRate trig <BitRate> max When a motion event occurs, the video frame encoding bitrate is adjusted according to the motion intensity of the video frame in the BitRate. min and BitRate max The bitrate is adjusted linearly between BitRate and BitRate; when the motion event disappears, the video frame coding bitrate changes from BitRate. rec Gradually decrease to BitRate min When BitRate min <BitRate> trig =BitRate max When a motion event occurs, the video frame encoding bitrate immediately changes from BitRate. min Step to BitRate maxAnd it remained at BitRate until the motion event disappeared. max The bitrate remains unchanged when the motion event disappears, changing from BitRate. max Gradually decrease to BitRate min .

[0027] Further, in step S3, when the motion intensity of a video frame is lower than the lower threshold of motion intensity, or when the motion intensity of a video frame is higher than the upper threshold of motion intensity, the error between the actual number of encoded bits of the video frame and the target number of encoded bits calculated before encoding is compensated within a sliding window consisting of multiple subsequent consecutive video frames, so that the actual encoding bitrate of the video sequence is close to the target encoding bitrate set by the user; the number of consecutive video frames contained in this sliding window is called the size of the sliding window, denoted by N. sw Indicated by Bits coded N represents the actual number of encoded bits for all video frames encoded at the current target coding bitrate in the video sequence. coded FrameBits represents the number of video frames encoded at the current target coding bitrate in the video sequence, and FrameBits represents the target coding bitrate of the video frame after correction. FrameBits is calculated using the following formula:

[0028] This invention also proposes an adaptive variable bitrate control device, comprising a video frame motion intensity calculation module, a video frame bit pre-allocation module, a video frame bit correction module, and a video frame bit control module. The video frame motion intensity calculation module uses motion detection methods to identify the motion and stillness features of image blocks within a video frame, thereby calculating the motion intensity of the video frame. The video frame bit pre-allocation module is used to compare the motion intensity of the video frame with a user-set upper limit threshold MovStrength. high MovStrength (lower limit of exercise intensity) low A comparison is made based on the comparison results and the user-set maximum bitrate (BitRate). max Minimum Bitrate min Trigger bitrate (BitRate) trigThe bitrate fallback smoothing step size (BitRateStep) is used to calculate the pre-allocated number of encoded bits for a video frame. The video frame bit correction module is used to correct the pre-allocated number of encoded bits for a video frame when the motion intensity is below the lower threshold or above the upper threshold, based on the encoding state information of the video sequence, to obtain the target number of encoded bits for the video frame. It is also used to directly use the pre-allocated number of encoded bits as the target number of encoded bits for a video frame when the motion intensity is between the lower and upper thresholds. The video frame bit control module is used to perform target bit control operations on the video frame based on the target number of encoded bits, obtaining the Lagrange multipliers and quantization parameters for use in the actual encoding process of the video frame.

[0029] The technical advantages achieved by this invention are: the proposed adaptive variable bitrate control is particularly suitable for video surveillance scenarios; by setting MovStrength... low and MovStrength high The size of the bitrate is adjusted to control sensitivity to motion events; by setting the BitRate... trig The value of is used to achieve different bitrate control effects; ultimately, it realizes automatic detection of motion events in video frames and calculation of motion intensity, and adjusts the target encoding bit number of video frames accordingly. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of a typical image group structure in a video surveillance scenario.

[0031] Figure 2 This is a flowchart illustrating the adaptive variable bitrate control method proposed in this invention.

[0032] Figures 3-5 It is BitRate trig A schematic diagram showing three different bitrate control processes resulting from different bitrate values.

[0033] Figure 6 This is a schematic diagram of the adaptive variable bit rate control device proposed in this invention.

[0034] The following are the labels in the figure: 1 is the video frame motion intensity calculation module, 2 is the video frame bit pre-allocation module, 3 is the video frame bit correction module, and 4 is the video frame bit control module. Detailed Implementation

[0035] Please see Figure 2 The adaptive variable bit rate bit rate control method proposed in this invention includes the following steps.

[0036] Step S1: Use motion detection to identify the motion and stillness features of image blocks within a video frame, and then calculate the motion intensity of the video frame.

[0037] In this step, there are many methods for motion detection of image patches and determination of their motion and stillness features. Common and widely used motion detection methods include the frame diff method, background subtraction method, optical flow method, Gaussian mixture model (GMM), and machine learning-based motion detection methods, among others.

[0038] This invention provides a detailed description of determining the motion and stillness features of image blocks using the frame difference method. First, the video frame is divided into multiple image blocks of the same size, and each image block is further divided into multiple image sub-blocks of the same size. Each image sub-block contains the same number of pixels. Then, the motion and stillness features of each image block within the video frame are determined, as follows: (1) The frame difference is calculated between the image blocks within the video frame and the image blocks at the same position within the reference frame. The brightness difference of each pixel in the image blocks within the video frame is calculated and compared with the motion pixel determination threshold. If the brightness difference of a pixel is greater than the motion pixel determination threshold, the pixel is marked as a motion pixel; otherwise, the pixel is marked as a still pixel. The reference frame refers to the reconstructed video frame of the previously encoded video frame, which can be used for motion estimation of image blocks within the subsequent video frame that employ inter-frame prediction technology to remove temporal information redundancy. (2) After marking the motion and stillness features of each pixel within the image block, dilation and erosion operations based on the image sub-blocks are required to highlight the motion features of the image and eliminate image noise, reducing the impact of missed and false judgments. Image dilation and erosion are two basic morphological operations, mainly used to find maxima and minima in an image. Dilation is similar to "neighborhood expansion", which expands the bright or white areas of the image, and the resulting image is larger than the bright areas of the original image. Erosion is similar to "neighborhood erosion", which reduces and refines the bright or white areas of the image, and the resulting image is smaller than the bright areas of the original image. Specifically, the number of moving pixels in each sub-block within the image block is counted. If the number of moving pixels in the sub-block is greater than the threshold for determining moving sub-blocks, the sub-block is marked as a moving sub-block; otherwise, it is marked as a stationary sub-block. (3) After marking the motion and stationary features of each sub-block within the image block, the number of moving sub-blocks in the image block is counted. If the number of moving sub-blocks in the image block is greater than the threshold for determining moving sub-blocks, the sub-block is marked as a moving sub-block; otherwise, it is marked as a stationary sub-block. The specific values ​​of the motion pixel determination threshold, the motion image sub-block determination threshold, and the motion image block determination threshold are derived from experimental experience. Their specific values ​​are related to the sensitivity of the frame difference method to motion, the total number of pixels contained in the image sub-block, and the total number of pixels contained in the image block.

[0039] After determining the motion and stillness features of each image block within a video frame, the motion intensity of the video frame is then calculated. MovStrength represents the motion intensity of the video frame, and Block... mov Block represents the number of moving image blocks in a video frame. totalLet represent the total number of image blocks in a video frame. Then, the motion intensity of the video frame is: As can be seen, the value of MovStrength is 0 ≤ MovStrength ≤ 100.

[0040] Step S2: Compare the motion intensity of the video frames with the user-defined upper limit threshold MovStrength. high MovStrength (lower limit of exercise intensity) low A comparison is made based on the comparison results and the user-set maximum bitrate (BitRate). max Minimum Bitrate min Trigger bitrate (BitRate) trig The bitrate is calculated using the BitRateStep method to determine the pre-allocated number of encoded bits for a video frame. The bitrate is the encoding bitrate.

[0041] MovStrength (lower limit of exercise intensity) low The physical meaning is: when the motion intensity of a video frame is equal to or greater than the lower threshold of motion intensity, it is determined that a motion event has occurred in the video frame; otherwise, it is determined that no motion event has occurred in the video frame. When no motion event has occurred in a video frame, the video encoding bitrate needs to be maintained at the minimum bitrate set by the user (BitRate). min MovStrength low The range of values ​​for is 0 ≤ MovStrength low <100. MovStrength low The smaller the value, the more sensitive the bitrate control is to motion events; even small motion events can trigger an increase in the video encoding bitrate. MovStrength low The larger the value, the less sensitive the bitrate control is to motion events; only larger motion events will trigger an increase in the video encoding bitrate.

[0042] MovStrength (Upper Threshold of Exercise Intensity) high The physical meaning is: when the motion intensity of a video frame exceeds the upper limit threshold of motion intensity, the video encoding bitrate needs to be maintained at the maximum bitrate set by the user (BitRate). max The bitrate cannot exceed the maximum bitrate set by the user. (MovStrength) high The range of values ​​for MovStrength is low <MovStrength high ≤100. MovStrength highThe smaller the value, the more sensitive the bitrate control is to motion events; even small motion events will cause the video encoding bitrate to reach the maximum bitrate set by the user. MovStrength high The larger the value, the less sensitive the bitrate control is to motion events; only larger motion events will cause the video encoding bitrate to reach the maximum bitrate set by the user.

[0043] Maximum Bitrate max The physical meaning of bitrate is: the maximum bitrate (in bps, bits per second) that can be achieved when motion events occur in a video frame. Minimum bitrate (BitRate) min The physical meaning of BitRate is: the minimum coding bit rate (in bps, bits per second) that needs to be maintained when no motion events occur in a video frame. Users must ensure that BitRate meets these requirements when setting it. min <BitRate max .

[0044] Trigger BitRate trig The physical meaning of BitRate is: the minimum coding bit rate (unit: bps, bits per second) that must be achieved when motion events occur in a video frame. Users must ensure that BitRate meets these requirements when setting it. min ≤BitRate trig ≤BitRate max In this way, when a motion event occurs in a video frame, it will trigger the video encoding bitrate to change from BitRate. min Jump to BitRate trig Or BitRate trig Above.

[0045] The physical meaning of the bitrate fallback smoothing step size BitRateStep is: when a motion event disappears in a video frame, the video coding bitrate for the subsequent video frames needs to be reduced from the coding bitrate when the motion event occurred (greater than or equal to BitRate). trig Gradually decrease to the minimum bitrate when no motion events occur (BitRate). min This is to prevent the video frame encoding bitrate from dropping too quickly, causing a sharp drop in video encoding quality. The value of BitRateStep is the number of video frames used to smooth the video encoding quality.

[0046] Step S2 specifically includes the following:

[0047] (1) If MovStrength < MovStrength lowIf no motion events occur in the video frame, the target encoding bitrate (BitRate) needs to be maintained at BitRate. min That is, BitRate = BitRate min .

[0048] (2) If MovStrength low ≤MovStrength≤MovStrength high The system determines that motion events have occurred in the video frame, and the target encoding bitrate (BitRate) is adjusted from BitRate. min Upgrade to BitRate trig and BitRate max Its specific value is directly proportional to the value of MovStrength, and is calculated by the following formula:

[0049] (3) If MovStrength > MovStrength high If motion events are detected in a video frame, the target encoding bitrate needs to be maintained at BitRate. max That is, BitRate = BitRate max .

[0050] (4) If there is a motion event in the previous video frame (i.e., the previous video frame satisfies MovStrength ≥ MovStrength) low The motion events in the current video frame disappear (i.e., the current video frame satisfies MovStrength < MovStrength). low The target bitrate (BitRate) for the current video frame and several subsequent consecutive video frames needs to be derived from the bitrate (BitRate) of the previous video frame. rec Gradually decrease to BitRate min This is to avoid a sudden drop in video coding quality. This stage is simply referred to as the transition stage after the motion event disappears. The number of video frames used to smooth video coding quality is determined by the value of BitRateStep. If the current video frame is the first frame used to smooth video coding quality, then the target coding bitrate BitRate for the m-th frame used to smooth video coding quality is: Where 1≤m≤BitRateStep.

[0051] (5) In the transition phase after the motion event disappears, if a motion event occurs again in a certain video frame, the transition phase is immediately terminated, and the target coding bitrate BitRate corresponding to the video frame is calculated according to (2) or (3) based on the MovStrength corresponding to the video frame.

[0052] After calculating the target coding bitrate (BitRate) based on the motion intensity of the video frames, the pre-allocated coding bit count (FrameBits) for each video frame is then calculated. pre FrameRate represents the encoding frame rate.

[0053] Step S3: If the motion intensity of the video frame is lower than the lower motion intensity threshold or higher than the upper motion intensity threshold, the pre-allocated number of coding bits for the video frame is corrected according to the coding state information of the video sequence to obtain the target number of coding bits for the video frame. If the motion intensity of the video frame is between the lower and upper motion intensity thresholds, the pre-allocated number of coding bits for the video frame is directly used as the target number of coding bits for the video frame. In this step, correction is only performed when the motion intensity of the video frame is lower than the lower motion intensity threshold or higher than the upper motion intensity threshold; otherwise, no correction is performed.

[0054] When the motion intensity of a video frame is below the lower threshold of motion intensity, the target bitrate (BitRate) needs to be maintained at the minimum bitrate set by the user. min When the motion intensity of a video frame exceeds the upper limit threshold for motion intensity, the target bitrate (BitRate) needs to remain at the maximum bitrate set by the user. max In both cases, the target bitrate of the video sequence needs to be maintained at a constant value. However, during video encoding, when a video frame is encoded, there is often an error between the actual number of encoded bits and the target number of encoded bits calculated before encoding. If compensation is not made during the encoding of subsequent video frames, it is difficult to maintain a constant target bitrate for the video sequence.

[0055] In this step, the error between the actual number of encoded bits in a video frame and the target number of encoded bits calculated before encoding is compensated within a sliding window consisting of multiple consecutive subsequent video frames. This ensures that the actual bitrate of the video sequence is close to the user-set target bitrate. The number of consecutive video frames contained within the sliding window is called the window size, denoted by N. sw Represented by Bits coded N represents the actual number of encoded bits for all video frames encoded at the current target coding bitrate in the video sequence. codedThis indicates the number of video frames encoded in the video sequence at the current target coding bitrate. (Bits) coded and N coded The value of is continuously updated as the video sequence encoding process progresses, reflecting the encoding status of the video sequence. Let FrameBits represent the target number of encoded bits for a corrected video frame. FrameBits can be calculated using the following formula:

[0056] Step S4: Based on the target number of coding bits for the video frame, perform target bit control operations on the video frame to obtain the Lagrange multipliers and quantization parameters of the video frame, which are used in the actual encoding process of the video frame. In this step, there are several algorithms for target bit control, such as the JCTVC-K0103 proposal, which will not be elaborated here.

[0057] In step S2, MovStrength low MovStrength high BitRate max BitRate min BitRate trig BitRateStep is a user-configurable parameter. By modifying these parameters, users can achieve different bitrate control effects, flexibly adapting to various use cases.

[0058] For example, if a user wants bitrate control to be more sensitive to motion events, they can set a lower MovStrength. low and MovStrength high This allows even small movements to trigger an increase in the video encoding bitrate and reach the maximum bitrate more quickly. If the user wants bitrate control to be less sensitive to motion events, a larger MovStrength setting can be used. low and MovStrength high This means that only larger movements can trigger an increase in the video encoding bitrate, and the maximum bitrate is reached more slowly.

[0059] For example, users can adjust BitRate. trig The value of BitRate is used to achieve different bitrate control effects. The following shows an example of BitRate. trig The different values ​​produce three different bitrate control effects.

[0060] User configuration BitRate trig The first possible value: BitRate min <BitRate> trig <BitRate> maxIn this case, when a motion event occurs, the video frame encoding bitrate will change from BitRate. min Step to BitRate trig Then, based on the motion intensity of the video frames, in BitRate... trig and BitRate max The bitrate is adjusted linearly between BitRate and BitRate. When the motion event disappears, the video frame encoding bitrate will change from BitRate to BitRate. rec Gradually decrease to BitRate min This avoids a sudden drop in encoding quality. This configuration mode is suitable for users with a moderate level of concern about motion events and a moderate level of concern about bandwidth conservation. The bitrate control process in this configuration is as follows: Figure 3 As shown.

[0061] User configuration BitRate trig The second possible value: BitRate min =BitRate trig <BitRate> max In this case, when a motion event occurs, the video frame encoding bitrate does not change abruptly, but rather adjusts according to the motion intensity of the video frame within the BitRate range. min and BitRate max The bitrate is adjusted linearly between BitRate and BitRate. When the motion event disappears, the video frame encoding bitrate will change from BitRate to BitRate. rec Gradually decrease to BitRate min This is to avoid a sudden drop in encoding quality. However, due to BitRate... min =BitRate trig The transition phase after the motion event disappears is not obvious. This configuration mode is suitable for situations where users have low attention to motion events and high attention to bandwidth conservation. The bitrate control process in this configuration is as follows: Figure 4 As shown.

[0062] User configuration BitRate trig The third possible value: BitRate min <BitRate> trig =BitRate max In this case, when a motion event occurs, the video frame encoding bitrate will immediately change from BitRate. min Step to BitRate max And it remained at BitRate until the motion event disappeared. max The bitrate remains unchanged. When the motion event disappears, the video frame encoding bitrate will change from BitRate. max Gradually decrease to BitRate minThis avoids a sudden drop in encoding quality. This configuration mode is suitable for users who are highly focused on motion events and less concerned about bandwidth savings. The bitrate control process in this configuration is as follows: Figure 5 As shown.

[0063] Depend on Figures 3-5 As can be seen, by modifying the configuration parameters of the bitrate control method in this invention, users can achieve different variable bitrate control effects, flexibly adapting to different usage scenarios. These bitrate configuration parameters are intuitive and easy to understand and configure for users.

[0064] Please see Figure 6 The adaptive variable bitrate bitrate control device proposed in this invention includes a video frame motion intensity calculation module 1, a video frame bit pre-allocation module 2, a video frame bit correction module 3, and a video frame bit control module 4. Figure 6 The device shown corresponds to Figure 2 The method shown.

[0065] The video frame motion intensity calculation module 1 is used to identify the motion and stillness features of image blocks within a video frame using motion detection methods, and then calculate the motion intensity of the video frame.

[0066] The video frame bit pre-allocation module 2 is used to match the motion intensity of the video frame with the user-set upper limit threshold MovStrength. high MovStrength (lower limit of exercise intensity) low A comparison is made based on the comparison results and the user-set maximum bitrate (BitRate). max Minimum Bitrate min Trigger bitrate (BitRate) trig The bitrate is calculated using the BitRateStep method to determine the pre-allocated number of encoded bits for a video frame. The bitrate is the encoding bitrate.

[0067] The video frame bit correction module 3 is used to correct the pre-allocated number of coding bits of the video frame according to the coding state information of the video sequence when the motion intensity of the video frame is lower than the lower threshold or higher than the upper threshold, so as to obtain the target number of coding bits of the video frame; it is also used to directly use the pre-allocated number of coding bits of the video frame as the target number of coding bits of the video frame when the motion intensity of the video frame is between the lower threshold and the upper threshold.

[0068] The video frame bit control module 4 is used to perform target bit control operations on the video frame according to the target number of encoding bits of the video frame, to obtain the Lagrange multipliers and quantization parameters of the video frame, which are used in the actual encoding process of the video frame.

[0069] This invention proposes an adaptive variable bitrate control method suitable for video surveillance scenarios. The proposed method automatically detects the occurrence and intensity of motion events based on the content of video frames and adjusts the target bitrate of the video frames accordingly. When no motion events occur within a video frame, fewer target bitrates are allocated, resulting in a lower bitrate and saving bandwidth for video storage and transmission. When motion events occur within a video frame, more target bitrates are allocated based on the intensity of the motion, resulting in a higher bitrate and improved video encoding quality. This approach achieves both bandwidth savings and improved encoding quality for the video content of interest to the user. Compared to other variable bitrate control methods, the proposed method offers advantages such as intuitive and easy-to-understand bitrate control parameters, flexible configuration for different variable bitrate control effects, and wider applicability across various scenarios.

[0070] The above are merely preferred embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method of adaptive variable bit rate control, characterized in that, The method comprises the following steps; Step S1: using a motion detection method to identify the motion and static features of the image blocks in the video frame, i.e., determining whether each image block in the video frame is a moving image block or a static image block, and then calculating the motion intensity of the video frame; Step S2: comparing the motion intensity of the video frame with a user-set upper threshold of motion intensity , a user-set lower threshold of motion intensity , calculating the pre-allocated encoding bit number of the video frame according to the comparison result and a user-set maximum bit rate , a user-set minimum bit rate , a trigger bit rate , a bit rate fallback smoothing step ; When < < the motion event occurs, the video frame encoding code rate is stepped from to , and then is linearly adjusted between and according to the motion intensity of the video frame; when the motion event disappears, the video frame encoding code rate is gradually decreased from to ; When = < When the motion event occurs, the video frame coding rate is linearly adjusted according to the motion intensity of the video frame between and ; when the motion event disappears, the video frame coding rate gradually decreases from to ; when < = When a motion event occurs, the video frame encoding bitrate immediately changes from... Step to And it remained at that level until the movement event disappeared. Unchanged; when the motion event disappears, the video frame coding bitrate changes from... Gradually decrease to ; Step S3: if the motion intensity of the video frame is lower than the lower threshold of the motion intensity or higher than the upper threshold of the motion intensity, correcting the pre-allocated encoding bits of the video frame according to the encoding state information of the video sequence to obtain the target encoding bits of the video frame; the correction is that the error between the actual encoding bits of the video frame and the target encoding bits calculated before encoding is compensated in a sliding window composed of a plurality of subsequent continuous video frames, so that the actual encoding code rate of the video sequence approaches the target encoding code rate set by the user; if the motion intensity of the video frame is between the lower threshold of the motion intensity and the upper threshold of the motion intensity, directly using the pre-allocated encoding bits of the video frame as the target encoding bits of the video frame; Step S4: performing target bit control operation on the video frame according to the target encoding bits of the video frame to obtain the Lagrange multiplier and the quantization parameter of the video frame, which are used in the actual encoding process of the video frame.

2. The adaptive variable bit rate code rate control method of claim 1, wherein, In the step S1, any one or more of the following motion detection methods is used to determine the motion and static features of the image blocks in the video frame: frame difference method, background subtraction method, optical flow method, Gaussian mixture model (GMM) method, and motion detection method based on machine learning.

3. The method of claim 2, wherein the step of determining the target bit rate is performed by: determining a target bit rate for the current frame based on the target bit rate for the previous frame and the target bit rate for the next frame. In the step S1, the frame difference method is used to determine the motion and static features of the image blocks in the video frame, which specifically includes the following contents: First, the video frame is divided into a plurality of image blocks of the same size, and each image block is further divided into a plurality of image sub-blocks of the same size, each image sub-block containing a plurality of pixel points of the same number; frame difference is performed between the image blocks in the video frame and the image blocks at the same positions in the reference frame, the brightness difference value of each pixel point in the image blocks in the video frame is calculated, and the brightness difference value is compared with the motion pixel determination threshold value; if the brightness difference value of the pixel point is greater than the motion pixel determination threshold value, the pixel point is marked as a motion pixel point; otherwise, the pixel point is marked as a static pixel point; if the number of motion pixel points in the image sub-block is greater than the determination threshold value of the motion image sub-block, the image sub-block is marked as a motion image sub-block; otherwise, the image sub-block is marked as a static image sub-block; if the number of motion image sub-blocks in the image block is greater than the determination threshold value of the motion image block, the image block is marked as a motion image block; otherwise, the image block is marked as a static image block.

4. The method of adaptive variable bit rate control according to claim 1, wherein, In step S1, after determining the motion and static characteristics of each image block within the video frame, the motion intensity of the video frame is calculated ; ; wherein represents the number of moving image blocks in the video frame, represents the total number of image blocks in the video frame.

5. The method of adaptive variable bit rate control according to claim 1, wherein, In step S2, the lower limit threshold of exercise intensity The physical meaning is: when the motion intensity of a video frame is greater than or equal to the lower limit threshold of motion intensity, it is determined that a motion event has occurred in the video frame; otherwise, it is determined that no motion event has occurred in the video frame; when no motion event has occurred in the video frame, the video encoding bitrate needs to be maintained at the minimum bitrate set by the user. ; motion intensity upper threshold The physical meaning is: when the motion intensity of the video frame > the motion intensity upper threshold, the video encoding code rate needs to be kept as the maximum code rate set by the user ; Maximum bit rate The physical meaning of the maximum bit rate is: the maximum encoding bit rate that can be achieved when there is a motion event in the video frame; Minimum code rate The physical meaning is: the minimum code bit rate required to maintain when there is no motion event in the video frame; need to meet ; Triggered code rate The physical meaning is: the minimum code bit rate that needs to be reached when there is a motion event in the video frame; needs to meet ;​​ Code rate fall-back smoothing step The physical meaning is: when the motion event in the video frame disappears, the video encoding code rate of the subsequent several video frames needs to be gradually reduced from the encoding code rate when the motion event occurs to the minimum code rate when there is no motion event , The value of is the number of these several video frames used to smooth the video encoding quality.

6. The method of adaptive variable bit rate control according to claim 5, wherein, In the step S2, the following contents are specifically included; (1) If < , it is determined that no motion event occurs in the video frame, and the target encoding code rate is maintained at ; (2) If ≤ ≤ , it is determined that a motion event occurs in the video frame, and the target coding rate is raised from to and , and the specific value is proportional to the value of ; (3) If , it is determined that a motion event occurs in the video frame, and the target coding rate is maintained at ;​ (4) If there is a motion event in the previous video frame and the motion event disappears in the current video frame, the target encoding code rate of the current video frame and the subsequent continuous several video frames needs to be gradually reduced from the encoding code rate of the previous video frame , This stage is called the transition stage after the disappearance of the motion event; if the current video frame is the first frame for smoothing the video encoding quality, the target encoding code rate of the mth frame for smoothing the video encoding quality is : ; 1≤m≤ ; (5) in the transition stage after the motion event disappears, if a motion event occurs again in a video frame, the transition stage is immediately stopped, and the target coding rate corresponding to the video frame is calculated according to (2) or (3) According to the target coding rate corresponding to the video frame , the calculation is performed according to (2) or (3). According to the motion intensity of the video frame, a target coding code rate is calculated Then, the pre-allocated coding bit number of the video frame is calculated ; use to represent the coding frame rate, .

7. The method of adaptive variable bit rate control according to claim 5, wherein, The step S2, and The smaller, the more sensitive the rate control is to motion events; and The larger, the more insensitive the rate control is to motion events.

8. The method of adaptive variable bit rate control according to claim 6, wherein, In the step S3, the number of the continuous video frames contained in the sliding window is called the size of the sliding window, denoted as The actual number of the coded bits of all the video frames in the video sequence that have been coded at the current target coding rate is denoted as The number of the video frames in the video sequence that have been coded at the current target coding rate is denoted as The target coding bits of the corrected video frame is denoted as Then the target coding bits of the corrected video frame is calculated by the following formula: .​ 9. A rate control apparatus for adaptive variable bit rate, characterized by, It comprises a video frame motion intensity calculation module, a video frame bit pre-allocation module, a video frame bit correction module, and a video frame bit control module; The video frame motion intensity calculation module is used to identify the motion and static features of the image blocks in the video frame by using a motion detection method, and then calculate the motion intensity of the video frame; The video frame bit pre-assignment module is used to compare the motion intensity of a video frame with a user-set upper threshold of motion intensity , a user-set lower threshold of motion intensity , and calculate the pre-assignment encoding bit number of the video frame according to the comparison result and a user-set maximum code rate , a user-set minimum code rate , a code rate trigger , and a code rate fallback smoothing step length . When < < motion event occurs, the video frame encoding code rate is stepped from to and then linearly adjusted according to the motion intensity of the video frame between and ; when the motion event disappears, the video frame encoding code rate is gradually decreased from to ; When = < When the motion event occurs, the video frame encoding code rate is linearly adjusted according to the motion intensity of the video frame between and ; when the motion event disappears, the video frame encoding code rate gradually decreases from to ; When < = When a motion event occurs, the video frame coding rate is immediately stepped up from to and remains unchanged at until the motion event disappears; when the motion event disappears, the video frame coding rate is gradually decreased from to ; The video frame bit correction module is used to correct the pre-allocated encoding bits of the video frame according to the encoding state information of the video sequence when the motion intensity of the video frame is lower than the lower limit threshold of the motion intensity or higher than the upper limit threshold of the motion intensity, to obtain the target encoding bits of the video frame; the correction is that the error between the actual encoding bits of the video frame and the target encoding bits calculated before encoding is compensated in a sliding window composed of a plurality of subsequent continuous video frames, so that the actual encoding code rate of the video sequence is close to the target encoding code rate set by the user; and the video frame bit correction module is also used to directly use the pre-allocated encoding bits of the video frame as the target encoding bits of the video frame when the motion intensity of the video frame is between the lower limit threshold of the motion intensity and the upper limit threshold of the motion intensity. The video frame bit control module is used to perform target bit control operation on the video frame according to the target encoding bits of the video frame, to obtain the Lagrange multiplier and the quantization parameter of the video frame, which are used in the actual encoding process of the video frame.

Images