A method for AVC bit rate control based on human visual system

Abstract

The present invention belongs to the field of video coding and decoding technology, and specifically is an AVC rate control method based on the human visual system. The present invention includes: setting the target bit rate and frame rate of the encoder, initializing the parameters of the rate control model, and allocating the target number of bits to each GOP; calculating the complexity of each frame of the image, and allocating the target number of bits of each frame of the image in each GOP according to the image complexity; calculating the characteristic information of each macroblock in each frame of the image, classifying the macroblocks, and allocating the target number of bits of each macroblock according to the macroblock type; calculating the encoding quantization parameter of each macroblock. The present invention can reasonably allocate the target number of bits according to the characteristics of the macroblocks, so that the actual encoding bit rate is close to the set target bit rate, while obtaining a higher image quality, and providing users with a better experience under a given bandwidth.

Description

Technical Field

[0001] The present invention relates to the field of computer information technology, and in particular to the field of image compression. Background Art

[0002] In video communication, the video source must first be effectively compressed so that the amount of data transmitted is within the channel bandwidth. A bit rate that is too high will exceed the maximum capacity of the channel, resulting in data loss; a bit rate that is too low will cause bandwidth waste. Therefore, the goal of bit rate control is to obtain the best image quality while meeting the channel bandwidth constraint.

[0003] The AVC video codec standard is widely used for its high compression rate and image quality, and has become a research hotspot in the current image communication field. Traditional AVC rate control has achieved a good balance between image compression rate and image quality. However, traditional AVC rate control uses a unified strategy to allocate target bit rates for each macroblock without considering the visual characteristics of the human eye. When there is a large global motion in the video sequence or the image content is more complex, the traditional AVC rate control method cannot objectively and accurately allocate the number of bits that conform to the visual sensitivity characteristics of the human eye, resulting in a certain degree of degradation of the subjective video quality of the human eye. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides an AVC bit rate control method based on the human visual system.

[0005] To achieve the purpose of the present invention, an AVC bit rate control method based on the human visual system is provided, comprising the following steps:

[0006] Set the target bit rate and frame rate of the encoder, initialize the rate control model parameters, and allocate the target number of bits to each GOP;

[0007] Calculate the complexity of each frame of the image and allocate the target number of bits of each frame of the image in each GOP according to the image complexity, including the following steps:

[0008] Calculate the remaining target number of bits when encoding the jth frame of the i-th GOP, denoted as T r (n i,j ), in, represents the jth frame of the i-th GOP, N gop Indicates the number of frames contained in a GOP, u(n i,j ) represents the available channel bandwidth when encoding the jth frame of the i-th GOP, F r represents the frame rate, A(n i,j ) represents the number of bits actually generated by encoding the jth frame of the i-th GOP;

[0009] Calculate the target number of bits allocated to the jth frame of the ith GOP in the first round, denoted as

[0010] Calculate the target buffer level of the jth frame of the ith GOP, denoted as Tbl(n i,j ), Among them, B s Indicates the initial buffer filling degree, Tbl(n i,2 )=B c (n i,2 ), B c (n i,j ) represents the actual buffer fullness after encoding the jth frame of the i-th GOP;

[0011] Calculate the target number of bits allocated to the jth frame of the ith GOP in the second round, denoted as Among them, γ is a constant;

[0012] Calculate the target number of bits allocated to the jth frame of the ith GOP in the third round, denoted as f'(n i,j ), Among them, β is a constant;

[0013] Calculate the image complexity, denoted as C m , Where m represents the sequence number of the current frame image in the entire video sequence, W and H represent the image width and height respectively, and I C (x,y) and I P (x, y) represents the brightness value of the reconstructed frame of the current frame image and the previous frame image at the coordinate position (x, y);

[0014] Calculate the target number of bits allocated to the jth frame of the ith GOP in the fourth round, denoted as f(n i,j ),

[0015] Calculate the feature information of each macroblock in each frame, classify the macroblocks, and allocate the target number of bits for each macroblock according to the macroblock type;

[0016] Calculate the encoding quantization parameter for each macroblock.

[0017] In one embodiment, setting a target bit rate and frame rate of an encoder, initializing rate control model parameters, and allocating a target number of bits to each GOP includes the following steps:

[0018] Set the target bit rate and frame rate of the encoder according to the application scenario requirements;

[0019] Initialize rate control model parameters;

[0020] The target number of bits allocated to the i-th GOP in the video sequence is denoted as T r (n i，0 ), Among them, n i，j (i=0,1,2...,j=0,1,2...,N gop -1) represents the jth frame of the ith GOP, N gop Indicates the number of frames contained in a GOP, B indicates the target bit rate, and F r Indicates the frame rate, B s Indicates the initial buffer filling degree, B c (n i，j ) represents the actual buffer fullness after encoding the jth frame of the ith GOP.

[0021] In one embodiment, the actual buffer filling degree is Update, where A(n i，j ) represents the number of bits actually generated when encoding the jth frame of the ith GOP, u(n i，j ) represents the channel bandwidth available when encoding the j-th frame of the ith GOP.

[0022] In one embodiment, the characteristic information of each macroblock in each frame of the image is calculated, the macroblocks are classified, and the target number of bits of each macroblock is allocated according to the macroblock type, including the following steps:

[0023] Calculate the target number of bits for the kth macroblock of the jth frame of the ith GOP in the first round, denoted as f k (n i，j ), f k (n i，j ) = f k-1 (n i，j )-A k-1 (n i，j ), where k = 1, 2, 3, ... N mb -1, N mb Indicates the number of macroblocks contained in the jth frame, A k (n i，j ) represents the number of bits actually generated by encoding the k-th macroblock of the j-th frame of the ith GOP;

[0024] Calculate the MAD prediction value of the kth macroblock of the jth frame of the ith GOP, denoted as PMAD k (n i，j ), PMAD k (n i，j )=F1*MAD k (n i，j-1)+F2, where F1 and F2 represent the two coefficients of the linear model, MAD k (n i，j ) represents the MAD value of the k-th macroblock of the j-th frame of the i-th GOP;

[0025] Calculate the target number of bits for the kth macroblock of the jth frame of the ith GOP in the second round, denoted as Where N = N mb -1;

[0026] Calculate the brightness feature information of the kth macroblock of the jth frame of the i-th GOP, denoted as Int k (n i，j ), Among them, M and N represent the width and height of the macroblock respectively;

[0027] If Int k (n i，j ) is less than the threshold value T1, the macroblock is judged to be a brightness-sensitive macroblock; otherwise, it is judged to be a non-brightness-sensitive macroblock;

[0028] If the macroblock is a non-brightness-sensitive macroblock, the motion feature information of the kth macroblock of the jth frame of the ith GOP is calculated, denoted as Mov k (n i，j ),

[0029] If MOV k (n i，j ) is greater than the threshold T2, the macroblock is judged as a motion block; otherwise, it is judged as a background block;

[0030] If the macroblock is a background block, the texture feature information of the kth macroblock of the jth frame of the ith GOP is calculated and recorded as Tex k (n i，j ), Among them, D1 represents the set of pixels in the macroblock whose brightness value is greater than or equal to the average brightness value of the macroblock, and N D1 represents the number of pixels contained in the D1 set, D2 represents the set of pixels in the macroblock whose brightness value is less than the average brightness value of the macroblock, and N D2 Indicates the number of pixels contained in the D2 set;

[0031] If Tex k (n i，j ) is greater than the threshold T3, the macroblock is judged as a detail block; otherwise, it is judged as a smooth block;

[0032] Set the macroblock weight, denoted as α k (n i，j ), if the macroblock is a brightness-sensitive macroblock, set αk (n i，j )=1.5; if the macroblock is a motion block, set α k (n i，j )=1.3; if the macroblock is a detail block, set αk(ni,j)=0.35; if the macroblock is a smooth block, set α k (n i，j )=0.65;

[0033] Calculate the target number of bits for the kth macroblock of the jth frame of the ith GOP in the third round, denoted as g k (n i，j ),

[0034] In one embodiment, the target number of bits allocated to the 0th macroblock of the jth frame of the ith GOP in the first round is initialized to f0(n i，j )=f(n i，j ).

[0035] In one of the embodiments, F1 is initialized to 1, F2 is initialized to 0, and F1 and F2 are updated after each macroblock is encoded.

[0036] In one embodiment, calculating the encoding quantization parameter of each macroblock includes the following steps:

[0037] The encoding quantization parameter of the macroblock is calculated by the quadratic equation, denoted as QStep k (n i，j ), Among them, HB k (n i，j ) represents the average proportion of header information of all macroblocks before the kth macroblock of the jth frame of the ith GOP, and X1 and X2 represent the two coefficients of the quadratic model.

[0038] In one of the embodiments, X1 is initialized to the target bit rate, X2 is initialized to 0, and X1 and X2 are updated after each macroblock is encoded.

[0039] The beneficial effects of the above-mentioned AVC rate control method based on the human visual system are as follows:

[0040] Set the target bit rate and frame rate of the encoder, initialize the parameters of the bit rate control model, and allocate the target number of bits to each GOP; calculate the complexity of each frame of the image, and allocate the target number of bits for each frame of the image in each GOP according to the image complexity; calculate the characteristic information of each macroblock in each frame of the image, classify the macroblocks, and allocate the target number of bits for each macroblock according to the macroblock type; calculate the encoding quantization parameter of each macroblock. The present invention adaptively adjusts different types of macroblocks, allocates more bits in the visually sensitive area of ​​the human eye, and allocates less bits in the visually insensitive area of ​​the human eye. In the process of video compression and transmission, the limited bandwidth is fully utilized, and the subjective and objective quality of the video is effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of a specific embodiment of the method according to the present invention.

[0042] Figure 2 This is a subjective effect chart encoded by AVC reference software JM19.0.

[0043] Figure 3 A subjective effect diagram according to a specific embodiment of the method of the present invention.

[0044] Figure 4 This is the effect diagram of the bit rate control encoded by AVC reference software JM19.0.

[0045] Figure 5 It is a diagram showing the effect of bit rate control according to a specific embodiment of the method of the present invention.

[0046] Figure 6 It is a PSNR comparison diagram between the encoding of AVC reference software JM19.0 and the encoding according to a specific embodiment of the method of the present invention. DETAILED DESCRIPTION

[0047] The present invention will be further described below in conjunction with the accompanying drawings.

[0048] Figure 1 is a flow chart of a specific embodiment of the method according to the present invention;

[0049] In step 101, the target bit rate and frame rate of the encoder are set, the rate control model parameters are initialized, and the target number of bits is allocated to each GOP.

[0050] In step 102, the complexity of each frame of the image is calculated, and the target number of bits of each frame of the image in each GOP is allocated according to the image complexity.

[0051] In step 103, characteristic information of each macroblock in each frame of image is calculated, the macroblocks are classified, and a target number of bits is allocated to each macroblock according to the macroblock type.

[0052] In step 104, the encoding quantization parameter of each macroblock is calculated.

[0053] Next, a detailed description is given of an AVC bit rate control method based on the human visual system according to an embodiment of the present invention.

[0054] In GOP layer rate control, the target bit rate and frame rate of the encoder are set according to the application scenario and bandwidth capacity.

[0055] Initialize the linear model coefficients, F1=1, F2=0; initialize the quadratic model coefficients, X1 is initialized to the target bit rate, and X2 is initialized to 0; initialize the encoding quantization parameters.

[0056] Determine the current GOP number and frame number. When i=0 and j=0, set the actual buffer fullness. Otherwise press Calculate the actual buffer filling.

[0057] The target number of bits allocated to the i-th GOP in the video sequence is denoted as T r (n i，0 ),

[0058] In frame-level rate control, the target number of remaining bits when encoding the jth frame of the ith GOP is calculated, denoted as T r (n i，j ), T r (n i，j )=Tr(n i，j-1 )-(N gop -j)*(u(n i，j )-u(n i，j-1 )) / FA(n i，j-1 ).

[0059] Calculate the target number of bits allocated to the jth frame of the ith GOP in the first round, denoted as

[0060] Calculate the target buffer level of the jth frame of the ith GOP, denoted as Tbl(n i，j ),

[0061] Determine the current frame number, and when j=2, set the target buffer level Tbl(n i，2 )=B c (n i，2 ).

[0062] Calculate the target number of bits allocated to the jth frame of the ith GOP in the second round, denoted as Here, γ is a constant, and here, γ=0.5.

[0063] Calculate the target number of bits allocated to the jth frame of the ith GOP in the third round, denoted as f′(n i，j ), Here, β is a constant, and here, β=0.5.

[0064] Calculate the image complexity, denoted as C m ,

[0065] Calculate the target number of bits allocated to the jth frame of the ith GOP in the fourth round, denoted as f(n i，j ),

[0066] In the macroblock layer rate control, assume that the current frame image is the jth frame of the i-th GOP, and assume that the current frame image contains four macroblocks A, B, C and D, and the previous frame image contains four macroblocks AP, BP, CP and DP. Assume that the MAD values ​​corresponding to the four macroblocks AP, BP, CP and DP are recorded as MAD0(n i，j-1 )、MAD1(n i，j-1 )、MAD2(n i，j-1 ) and MAD3(n i，j-1 ), the MAD prediction values ​​of the four macroblocks A, B, C and D are recorded as PMAD0(n i，j )、PMAD1(n i，j )、PMAD2(n i，j ) and PMAD3(n i，j ), then PMAD k (n i，j )=F1*MAD k (n i，j-1 )+F2,0≤k≤3.

[0067] Determine the macroblock number of the current coding block. When the macroblock number k=0, that is, macroblock A is currently encoded, the target bit number f0 (n i，j )=f(n i，j ), directly set the encoding quantization parameter of macroblock A to the initialized encoding quantization parameter, encode macroblock A according to the encoding quantization parameter, and obtain the actual number of bits generated after encoding macroblock A, recorded as A0(n i，j ), and at the same time obtain the number of bits generated by the header information of the coded macroblock A, recorded as HB0(n i，j ).

[0068] Calculate the target number of bits f1(n i，j )=f0(n i，j )-A0(ni，j ). Calculate the target number of bits allocated to macroblock B in the second round Calculate the brightness feature information of macroblock B And compare it with the threshold T1. Assume Int1(n i，j ) is greater than or equal to T1, then macroblock B is judged to be a non-brightness sensitive block. Calculate the motion feature information of macroblock B And compare it with the threshold T2. Assume MOV1(n i，j ) is less than or equal to T2, then macroblock B is determined to be a background block. Calculate the average brightness value of the macroblock corresponding to macroblock B, compare the brightness values ​​of all pixels in macroblock B with the average brightness value of the macroblock, and put the pixels with brightness values ​​greater than or equal to the average brightness value of the macroblock into set D1, and put the pixels with brightness values ​​less than the average brightness value of the macroblock into set D2. Assume that set D1 contains N1 pixels and set D2 contains N2 pixels. Calculate the texture feature information of macroblock B And compare it with the threshold T3. Assume that Tex1(n i，j ) is greater than T3, then macroblock B is judged to be a detail block. Set the macroblock weight α1(n i，j )=0.35. Calculate the target number of bits allocated to macroblock B in the third round Through the quadratic equation Calculate the encoding quantization parameter QStep1(n i，j After encoding macroblock B according to the coding quantization parameter, the actual number of bits generated after encoding macroblock B is obtained, which is recorded as A1(n i，j ), and at the same time obtain the average number of bits generated by the header information of the coded macroblocks A and B, recorded as HB1(n i，j ). The linear model coefficients F1 and F2 are updated by the linear regression method; the quadratic model coefficients X1 and X2 are updated by the linear regression method.

[0069] Similarly, macroblock C and macroblock D are encoded according to the above process method to complete the encoding of the current frame.

[0070] In summary, the present invention can reasonably allocate the target bit number according to the macroblock characteristics, so that the actual encoding bit rate is close to the set target bit rate, while obtaining higher image quality, and providing users with a better experience under a given bandwidth. The AVC reference software JM19.0 and the method of the present invention are respectively used for encoding, and the same parameters are configured to encode 720P images. Under the condition that the number of bits generated by encoding is basically the same, see Figure 2 and Figure 3The image encoded by the method of the present invention has richer details and no blurred blocks, while the image encoded by the AVC reference software JM19.0 is not clear enough in the details of the forest and has blurred blocks in some parts; when the same parameters are configured to encode 1080P images, under the condition that the number of bits generated by encoding is basically the same, see Figure 4 and Figure 5 The image encoded by the method of the present invention has a clearer visual effect and no block effect, while the cloud part of the image encoded by the AVC reference software JM19.0 has obvious block effect, and the street lights are integrated into the clouds, resulting in serious visual distortion; from the PSNR point of view, setting the same configuration parameters, see Figure 6 The PSNR of encoding using the method of the present invention is higher.

[0071] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. An AVC bit rate control method based on the human visual system, comprising the following steps: Set the target bit rate and frame rate of the encoder, initialize the rate control model parameters, and allocate the target number of bits to each GOP; Calculate the complexity of each frame of the image and allocate the target number of bits for each frame of the image in each GOP according to the image complexity; The steps include: Calculate the remaining target number of bits when encoding the jth frame of the i-th GOP, denoted as T r (n i,j ), T r (n i,j )=T r (n i,j-1 )-(N gop -j)*(u(n i,j )-u(n i,j-1 )) / F r -A(n i,j-1 ); in, n i,j (i=0,1,2...,j=0,1,2...,N gop -1) represents the jth frame of the ith GOP, N gop Indicates the number of frames contained in a GOP, u(n i,j ) represents the available channel bandwidth when encoding the jth frame of the i-th GOP, F r represents the frame rate, A(n i,j ) represents the number of bits actually generated by encoding the jth frame of the i-th GOP; Calculate the target number of bits allocated to the jth frame of the ith GOP in the first round, denoted as Calculate the target buffer level of the jth frame of the ith GOP, denoted as Tbl(n i,j ), Among them, Tbl(n i,2 )=B c (n i,2 );B c (n i,j ) represents the actual buffer fullness after encoding the jth frame of the i-th GOP, B s Indicates the initial buffer filling degree; Calculate the target number of bits allocated to the jth frame of the ith GOP in the second round, denoted as Among them, γ is a constant; Calculate the target number of bits allocated to the jth frame of the ith GOP in the third round, denoted as f′(n i,j ), Among them, β is a constant; Calculate the image complexity, denoted as C m , Where m represents the sequence number of the current frame image in the entire video sequence, W and H represent the image width and height respectively, and I c (x,y) and I p (x, y) represents the brightness value of the reconstructed frame of the current frame image and the previous frame image at the coordinate position (x, y); Calculate the target number of bits allocated to the jth frame of the ith GOP in the fourth round, denoted as f(n i,j ), Calculate the feature information of each macroblock in each frame, classify the macroblocks, and allocate the target number of bits for each macroblock according to the macroblock type; Calculate the encoding quantization parameter for each macroblock.

2. The method according to claim 1, characterized in that: Setting the target bit rate and frame rate of the encoder, initializing the rate control model parameters, and allocating the target number of bits to each GOP includes the following steps: Set the target bit rate and frame rate of the encoder according to the application scenario requirements; Initialize rate control model parameters; The target number of bits allocated to the i-th GOP in the video sequence is denoted as T r (n i,0 ), Among them, n i,j (i=0,1,2…,h=0,1,2…,N gop -1) represents the hth frame of the i-th GOP, N gop Indicates the number of frames contained in a GOP, B indicates the target bit rate, and F r Indicates the frame rate, B s Indicates the initial buffer filling degree, B c (n i,j ) represents the actual buffer fullness before encoding the jth frame of the ith GOP.

3. The method according to claim 2, characterized in that: Actual buffer filling degree Update, where A(n i,j ) represents the number of bits actually generated when encoding the jth frame of the ith GOP, u(n i,j ) represents the channel bandwidth available when encoding the j-th frame of the ith GOP.

4. The method according to claim 1, characterized in that: Calculate the characteristic information of each macroblock in each frame of the image, classify the macroblocks, and allocate the target number of bits for each macroblock according to the macroblock type, including the following steps: Calculate the target number of bits for the kth macroblock of the jth frame of the ith GOP in the first round, denoted as f k (n i,j ), f k (n i,k ) = f k-1 (n i,j )-A k-1 (n i,j ), where k = 1, 2, 3…N mb -1, N mb Indicates the number of macroblocks contained in the jth frame, A k (n i,j ) represents the number of bits actually generated by encoding the k-th macroblock of the j-th frame of the ith GOP; Calculate the MAD prediction value of the kth macroblock of the jth frame of the i-th GOP, denoted as PMAD k (n i,j ), PMAD k (n i,j )=F1*MAD k (n i,j-1 )+F2, where F1 and F2 represent the two coefficients of the linear model, MAD k (n i,j ) represents the MAD value of the k-th macroblock of the j-th frame of the i-th GOP; Calculate the target number of bits for the kth macroblock of the jth frame of the ith GOP in the second round, denoted as Where N = N mb -1; Calculate the brightness feature information of the kth macroblock of the jth frame of the i-th GOP, denoted as Int k (n i,j ), Among them, M and N represent the width and height of the macroblock respectively; If Int k (n i,j ) is less than the threshold value T1, the macroblock is judged to be a brightness-sensitive macroblock; otherwise, it is judged to be a non-brightness-sensitive macroblock; If the macroblock is a non-brightness-sensitive macroblock, the motion feature information of the kth macroblock of the jth frame of the ith GOP is calculated, denoted as Mov k (n i,j ), If Mov k (n i,j ) is greater than the threshold T2, the macroblock is judged as a motion block; otherwise, it is judged as a background block; If the macroblock is a background block, the texture feature information of the kth macroblock of the jth frame of the ith GOP is calculated and recorded as Tex k (n i,j ), Among them, D1 represents the set of pixels in the macroblock whose brightness value is greater than or equal to the average brightness value of the macroblock, and N D1 represents the number of pixels contained in the D1 set, D2 represents the set of pixels in the macroblock whose brightness value is less than the average brightness value of the macroblock, and N D2 Indicates the number of pixels contained in the D2 set; If Tex k (n i,j ) is greater than the threshold T3, the macroblock is judged as a detail block; otherwise, it is judged as a smooth block; Set the macroblock weight, denoted as α k (n i,j ), if the macroblock is a brightness-sensitive macroblock, set α k (n i,j )=1.5; if the macroblock is a motion block, set α k (n i,j )=1.3; if the macroblock is a detail block, set α k (n i,j )=0.35; if the macroblock is a smooth block, set α k (n i,j )=0.65; Calculate the target number of bits for the kth macroblock of the jth frame of the ith GOP in the third round, denoted as g k (n i,j ), 5. The method according to claim 4, characterized in that: The target number of bits allocated to the 0th macroblock of the jth frame of the ith GOP in the first round is initialized to f0(n i,j )=f(n i,j ).

6. The method according to claim 4, characterized in that: F1 is initialized to 1, F2 is initialized to 0, and F1 and F2 are updated after each macroblock is encoded.

7. The method according to claim 4, characterized in that: Calculating the encoding quantization parameter of each macroblock includes the following steps: The encoding quantization parameter of the macroblock is calculated by the quadratic equation, denoted as QStep k (n i,j ), Among them, HB k (n i,j ) represents the average proportion of header information of all macroblocks before the kth macroblock of the jth frame of the ith GOP, and X1 and X2 represent the two coefficients of the quadratic model.

8. The method according to claim 7, characterized in that: X1 is initialized to the target bit rate, X2 is initialized to 0, and X1 and X2 are updated after each macroblock is encoded.

Images