Video processing methods, apparatus, devices and computer-readable storage media

Abstract

This application provides a video processing method, apparatus, device, and computer-readable storage medium. The method includes: acquiring the pre-allocated bitrate and the preset target bitrate of the current frame to be encoded in the video to be processed; when the adjustment conditions for quantization parameters are determined based on the pre-allocated bitrate and the target bitrate, acquiring the encoding complexity and quantization parameters of the current frame to be encoded; acquiring the sum of absolute errors of macroblocks between frames during motion estimation of the previous encoded frame in the video to be processed; adjusting the quantization parameters based on the encoding complexity and the sum of absolute errors of macroblocks between frames to obtain the adjusted quantization parameters; and encoding the current frame to be encoded based on the adjusted quantization parameters to obtain the encoding result. By determining that quantization parameters need adjustment based on the bitrate, complexity analysis is performed on the frame to be encoded to adjust the quantization parameters, thereby achieving quality control at the video frame level and optimizing video quality.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and to, but is not limited to, a video processing method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] With the rapid development of digital technology, people have increasingly higher requirements for video quality. The first problem to be solved when using digital transmission and computer processing of video signals is digitizing the video signal. A key step in video signal digitization is quantization, which is the process of mapping continuous signal values ​​into multiple discrete amplitude values, achieving a many-to-one mapping of signal values. Quantization inevitably introduces distortion, leading to a decrease in video quality. Summary of the Invention

[0003] In view of the above, embodiments of this application provide a video processing method, apparatus, device, and computer-readable storage medium.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a video processing method, the method comprising:

[0006] Obtain the pre-allocated bitrate and the preset target bitrate of the current frame to be encoded in the video to be processed;

[0007] When the adjustment conditions for satisfying the quantization parameters are determined based on the pre-allocated bitrate and the target bitrate, the encoding complexity and quantization parameters of the current frame to be encoded are obtained.

[0008] The sum of absolute errors of macroblocks between frames during motion estimation of the previous encoded frame in the video to be processed is obtained.

[0009] The quantization parameters are adjusted based on the coding complexity and the sum of absolute errors of the inter-frame macroblocks to obtain the adjusted quantization parameters.

[0010] The current frame to be encoded is encoded according to the adjusted quantization parameters to obtain the encoding result.

[0011] In some embodiments, the coding complexity includes frame complexity, vertical complexity, and horizontal complexity;

[0012] The step of obtaining the encoding complexity of the current frame to be encoded includes:

[0013] Obtain the reference frame of the current frame to be encoded from the encoding device;

[0014] The frame complexity of the current frame to be encoded is determined based on the current frame to be encoded and the reference frame.

[0015] The vertical and horizontal complexity of the current frame to be encoded are obtained from the encoding device.

[0016] In some embodiments, determining the frame complexity of the current frame to be encoded based on the current frame to be encoded and the reference frame includes:

[0017] The current frame to be encoded is divided into multiple encoding blocks;

[0018] Obtain the original pixel value of each coded block and the reconstructed pixel value of the corresponding block in the reference frame for each coded block;

[0019] The absolute error of each coding block is determined based on the original pixel value of each coding block and the reconstructed pixel value of the corresponding position block of each coding block;

[0020] The frame complexity of the current frame to be encoded is determined based on the absolute error of each coded block.

[0021] In some embodiments, adjusting the quantization parameters based on the coding complexity and the sum of absolute errors of the inter-frame macroblocks to obtain adjusted quantization parameters includes:

[0022] The first control factor is determined based on the vertical complexity, the horizontal complexity, and the sum of the absolute errors of each inter-frame macroblock.

[0023] Obtain the quantization step size of the current encoding and the number of encoding blocks obtained by dividing the current frame to be encoded;

[0024] The second control factor is determined based on the vertical complexity, horizontal complexity, quantization step size, number of coding blocks, absolute error of each inter-frame macroblock, and frame complexity.

[0025] The adjustment value is determined based on the first control factor and the second control factor;

[0026] The quantization parameters are adjusted according to the adjustment value to obtain the adjusted quantization parameters.

[0027] In some embodiments, determining the first control factor based on the vertical complexity, the horizontal complexity, and the sum of absolute errors of each inter-frame macroblock includes:

[0028] The absolute errors of each inter-frame macroblock are summed to obtain the cumulative sum of absolute errors;

[0029] The sum of the vertical complexity and the horizontal complexity is determined as the directional complexity.

[0030] The quotient of the directional complexity and the sum of the absolute errors is determined as the first control factor.

[0031] In some embodiments, determining the second control factor based on the vertical complexity, horizontal complexity, quantization step size, number of coding blocks, frame complexity, and the sum of absolute errors of the inter-frame macroblocks includes:

[0032] The control sub-factor is determined based on the vertical complexity, horizontal complexity, quantization step size, and number of coding blocks.

[0033] The second control factor is determined based on the sum of the absolute errors of each inter-frame macroblock, the cumulative sum of absolute errors, the control sub-factor, and the frame complexity.

[0034] In some embodiments, determining the adjustment value based on the first control factor and the second control factor includes:

[0035] When the first control factor is greater than the first threshold and less than or equal to the second threshold, the adjustment value is determined as the preset first adjustment value;

[0036] When the first control factor is greater than the second threshold and the second control factor is less than or equal to the third threshold, the adjustment value is determined as the preset second adjustment value;

[0037] When the first control factor is greater than the second threshold, and the second control factor is greater than the third threshold and less than or equal to the fourth threshold, the adjustment value is determined as the preset third adjustment value;

[0038] When the first control factor is greater than the second threshold and the second control factor is greater than the fourth threshold, the adjustment value is determined as the preset fourth adjustment value.

[0039] This application provides a video processing apparatus, the apparatus comprising:

[0040] The first acquisition module is used to acquire the pre-allocated bitrate and the preset target bitrate of the current frame to be encoded in the video to be processed;

[0041] The second acquisition module is used to acquire the encoding complexity and quantization parameters of the current frame to be encoded when the adjustment conditions for satisfying the quantization parameters are determined based on the pre-allocated bitrate and the target bitrate.

[0042] The third acquisition module is used to acquire the sum of absolute errors of macroblocks between frames during motion estimation of the previous encoded frame in the video to be processed;

[0043] The adjustment module is used to adjust the quantization parameters according to the encoding complexity and the sum of absolute errors of the inter-frame macroblocks to obtain the adjusted quantization parameters.

[0044] The encoding module is used to encode the current frame to be encoded according to the adjusted quantization parameters to obtain the encoding result.

[0045] This application provides an electronic device, including:

[0046] Processor; and

[0047] Memory for storing computer programs that can run on the processor;

[0048] The computer program, when executed by the processor, implements the steps of the above-described video processing method.

[0049] This application provides a computer-readable storage medium storing computer-executable instructions configured to perform the steps of the video processing method described above.

[0050] This application provides a video processing method, apparatus, device, and computer-readable storage medium. The method includes: acquiring the pre-allocated bitrate and a preset target bitrate of the current frame to be encoded in the video to be processed; determining whether the adjustment conditions for quantization parameters are met based on the pre-allocated bitrate and target bitrate; if met, acquiring the encoding complexity and quantization parameters of the current frame to be encoded; acquiring the sum of absolute errors of macroblocks between frames during motion estimation of the previous encoded frame in the video to be processed; then adjusting the quantization parameters based on the encoding complexity and the sum of absolute errors of macroblocks between frames, and using the adjusted quantization parameters to encode the current frame to be encoded, obtaining the encoding result. By determining that quantization parameters need adjustment based on the bitrate, complexity analysis is performed on the frame to be encoded to adjust the quantization parameters, thereby improving the encoding effect of the frame to be encoded, achieving quality control at the video frame level, and optimizing video quality. Attached Figure Description

[0051] In the accompanying drawings (which are not necessarily drawn to scale), similar reference numerals may describe similar parts in different views. The drawings illustrate, by way of example and not limitation, the various embodiments discussed herein.

[0052] Figure 1 This is a schematic diagram illustrating an implementation flow of the video processing method provided in an embodiment of this application;

[0053] Figure 2 A schematic diagram illustrating one implementation flow of the step of obtaining encoding complexity in the video processing method provided in this application embodiment;

[0054] Figure 3 This is a schematic diagram illustrating one implementation process of the step of adjusting quantization parameters in the video processing method provided in this application embodiment;

[0055] Figure 4 A schematic diagram illustrating the principle of a subjective quality optimization system for video conferencing scenarios based on frame complexity analysis, provided in an embodiment of this application.

[0056] Figure 5a This is a schematic diagram obtained by processing the example image using the original quantization parameters;

[0057] Figure 5b This is a schematic diagram showing the example image processed using the quantization parameters obtained through the video processing method provided in this embodiment.

[0058] Figure 6 This is a schematic diagram of the composition structure of a video processing apparatus provided in an embodiment of this application;

[0059] Figure 7 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0062] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0064] Before providing a more detailed description of the embodiments of this application, the nouns and terms used in the embodiments of this application shall be explained as follows.

[0065] Quantization is the process of mapping continuous values ​​of a signal into multiple discrete amplitude values, achieving a many-to-one mapping of signal values.

[0066] The quantization step size (Qstep), also known as the quantization matrix, is applied during the scaling process (which occurs between transform and quantization). Using different quantization steps for coefficients at different positions can improve the subjective quality of the video.

[0067] The quantization parameter (QP) is the quantization step size, reflecting the degree of spatial detail compression. A smaller value results in finer quantization, higher image quality, and a longer bitstream. For example, a smaller QP preserves more detail, leading to finer quantization; a larger QP loses more detail, resulting in coarser quantization. While this reduces the bitrate, it increases image distortion and degrades quality. QP and Qstep are linearly correlated; Qstep increases with QP, doubling in value for every 6 increases in QP.

[0068] Bit rate, or bit rate, represents the number of bits transmitted per unit of time.

[0069] Motion estimation, in inter-frame predictive coding, refers to the process of dividing a moving image into blocks or macroblocks due to the correlation between objects in neighboring frames. The goal is to find the position of each block or macroblock in neighboring frames and determine the relative spatial offset between them. This relative offset is commonly referred to as the motion vector, and the process of obtaining the motion vector is called motion estimation.

[0070] In related technologies, the quantization parameters used for encoding all frames within one second are the same, which affects video quality. To solve this problem, embodiments of this application provide a video processing method and apparatus. The method provided in the embodiments of this application will be described below with reference to the apparatus for implementing the embodiments of this application. Figure 1 This is a schematic diagram illustrating an implementation flow of the video processing method provided in an embodiment of this application, such as... Figure 1 As shown, the method provided in this application embodiment includes the following steps:

[0071] Step S101: Obtain the pre-allocated bitrate and the preset target bitrate of the current frame to be encoded in the video to be processed.

[0072] This application embodiment can be executed by a video processing device in an electronic device. When video encoding is required, after acquiring the video to be processed, the current video frame to be encoded is determined from the video to be processed, and the bitrate pre-allocated to the frame to be encoded by the encoder is obtained, thus obtaining the pre-allocated bitrate of the current frame to be encoded. Furthermore, the target bitrate pre-set by the user is obtained. In practical applications, the user can set the target bitrate by second, meaning that the target bitrate set for multiple video frames within a second is the same bitrate.

[0073] Step S102: When the adjustment conditions for quantization parameters are met based on the pre-allocated bitrate and the target bitrate, the encoding complexity and quantization parameters of the current frame to be encoded are obtained.

[0074] Based on the pre-allocated bitrate and the target bitrate, determine whether the quantization parameters of the current frame to be encoded in the encoder need to be adjusted. If the pre-allocated bitrate and the target bitrate satisfy the preset relationship, that is, when the conditions for adjusting the quantization parameters are met based on the pre-allocated bitrate and the target bitrate, it is determined that the quantization parameters need to be adjusted. At this time, the encoding complexity and quantization parameters of the current frame to be encoded are obtained.

[0075] In this embodiment, the adjustment condition for the quantization parameter can be: perFrameBits > (target bitrate / frame rate) * a, where perFrameBits is the pre-allocated bitrate and a is a coefficient, for example, a can be 1.2. When perFrameBits > (target bitrate / frame rate) * 1.2, the adjustment condition for the quantization parameter is met, and it is determined that the quantization parameter needs to be adjusted. At this time, the encoding complexity and quantization parameter of the current frame to be encoded are obtained, and step S103 is continued to adjust the quantization parameter before executing the encoding step. When perFrameBits ≤ (target bitrate / frame rate) * 1.2, the adjustment condition for the quantization parameter is not met, and there is no need to adjust the quantization parameter of the current frame to be encoded. That is, at this time, the quantization parameter of the current frame to be encoded is obtained, and the current frame to be encoded is encoded according to the quantization parameter to obtain the encoding result, and the encoding step of the next frame continues.

[0076] The encoding complexity here includes frame complexity, vertical complexity, and horizontal complexity, and the quantization parameter is the value predicted by the encoder.

[0077] In some embodiments, it may be based on Figure 2 The following steps are shown to obtain the coding complexity:

[0078] Step S1021: Obtain the reference frame of the current frame to be encoded from the encoding device.

[0079] A reference frame is a frame that is referenced when encoding the current frame to be encoded; it can be an I-frame or a P-frame. The encoding device here can be an encoder.

[0080] Step S1022: Determine the frame complexity of the current frame to be encoded based on the current frame to be encoded and the reference frame.

[0081] In one implementation, the frame complexity of the current frame to be encoded can be determined by the following steps: dividing the current frame to be encoded into multiple coding blocks; obtaining the original pixel values ​​of each coding block and the reconstructed pixel values ​​of the corresponding block in the reference frame; determining the sum of absolute errors of each coding block based on the original pixel values ​​of each coding block and the reconstructed pixel values ​​of the corresponding block; and determining the frame complexity of the current frame to be encoded based on the sum of absolute errors of each coding block.

[0082] In implementation, the frame can be divided into blocks of 4*4, 8*8, 16*16, 32*32, etc. For example, the current frame to be encoded can be divided into 16*16 blocks. If the current frame to be encoded is 1280*720, then the horizontal division is 1280 / 16=80, and the vertical division is 720 / 16=45, for a total of 80*45=3600 encoding blocks. The absolute error between the current frame to be encoded and the reference frame is calculated at the pixel level, and then accumulated block by block to obtain the frame complexity curFrameComplex of the current frame to be encoded. The calculation formula of Sad is as follows (1):

[0083]

[0084] Where f(x,y) is the original pixel value at (x,y), and g(x,y) is the reconstructed pixel value at (x,y).

[0085] Step S1023: Obtain the vertical and horizontal complexity of the current frame to be encoded from the encoding device.

[0086] Based on the above steps, the frame complexity, vertical complexity, and horizontal complexity of the current frame to be encoded are obtained, and the frame complexity, vertical complexity, and horizontal complexity are determined as the encoding complexity of the current frame to be encoded.

[0087] Step S103: Obtain the sum of absolute errors of macroblocks between frames during motion estimation of the previous encoded frame in the video to be processed.

[0088] Let the current frame to be encoded be frame k, and the previous encoded frame be frame (k-1), where k > 1. Calculate the Sad value during motion estimation of inter-frame macroblocks in frame (k-1) to obtain the sum of absolute errors of each inter-frame macroblock.

[0089] Step S104: Adjust the quantization parameters according to the coding complexity and the sum of absolute errors of macroblocks between frames to obtain the adjusted quantization parameters.

[0090] The cumulative sum of absolute errors for macroblocks between frames is calculated to obtain the cumulative sum of absolute errors, denoted as meSadAll. Then, based on the coding complexity and meSadAll, the amount of adjustment to the quantization parameters of the current frame to be encoded is determined, i.e., the adjustment value. The quantization parameters are adjusted using the adjustment value to obtain the adjusted quantization parameters.

[0091] Step S105: Encode the current frame to be encoded according to the adjusted quantization parameters to obtain the encoding result.

[0092] After determining the adjusted quantization parameters, the current frame to be encoded is encoded using the adjusted quantization parameters to obtain the encoding result. The encoding process then continues to the next frame.

[0093] The method provided in this application embodiment obtains the pre-allocated bitrate and the preset target bitrate of the current frame to be encoded in the video to be processed through a video processing device; determines whether the adjustment conditions for quantization parameters are met based on the pre-allocated bitrate and the target bitrate; if met, obtains the encoding complexity and quantization parameters of the current frame to be encoded; and obtains the sum of absolute errors of macroblocks between frames during motion estimation of the previous encoded frame in the video to be processed; then, adjusts the quantization parameters based on the encoding complexity and the sum of absolute errors of macroblocks between frames, and uses the adjusted quantization parameters to encode the current frame to be encoded to obtain the encoding result. By determining that quantization parameters need to be adjusted based on the bitrate, complexity analysis is performed on the frame to be encoded to adjust the quantization parameters, thereby improving the encoding effect of the frame to be encoded, achieving quality control at the video frame level, and optimizing video quality.

[0094] In some embodiments, step S104, "adjusting the quantization parameters based on the coding complexity and the sum of absolute errors of macroblocks between frames, to obtain the adjusted quantization parameters," can be achieved through... Figure 3 The following steps are shown to achieve this:

[0095] Step S1041: Determine the first control factor based on the vertical complexity, horizontal complexity, and the sum of absolute errors of macroblocks between frames.

[0096] The absolute errors of each macroblock in each frame are summed to obtain the cumulative sum of absolute errors; the sum of the vertical and horizontal complexity is determined as the directional complexity; the quotient of the directional complexity and the cumulative sum of absolute errors is determined as the first control factor.

[0097] Let the first control factor be denoted as ctrFactor, the cumulative absolute error be meSadAll, the horizontal complexity be hFrameComplex, and the vertical complexity be vFrameComplex. The first control factor ctrFactor can be calculated according to the following formula (2):

[0098] ctrFactor = (hFrameComplex + vFrameComplex) / meSadAll (2);

[0099] Step S1042: Obtain the quantization step size of the current encoding and the number of encoding blocks obtained by dividing the current frame to be encoded.

[0100] Obtain the quantization step size Qstep of the current encoding from the current encoder. If the current frame to be encoded is 1280*720, divide it horizontally into 1280 / 16=80 blocks and vertically into 720 / 16=45 blocks, resulting in 80*45=3600 blocks.

[0101] Step S1043: Determine the second control factor based on the vertical complexity, horizontal complexity, quantization step size, number of coding blocks, absolute error of macroblocks between frames, and frame complexity.

[0102] First, the control factor is determined based on the vertical complexity, horizontal complexity, quantization step size, and number of coding blocks. Then, the second control factor is determined based on the absolute error of each inter-frame macroblock, the sum of absolute errors, the control factor, and the frame complexity.

[0103] Let the control factor be denoted as Cursi and the second control factor as ctrFactor_plus. In one implementation, the control factor Cursi can be calculated according to the following equation (3), and the second control factor ctrFactor_plus can be calculated according to the following equation (4):

[0104] Cursi=100*(hFrameComplex+vFrameComplex) / (Qstep*MBH*MBW) (3);

[0106] ctrFactor_plus = meSadAll / (1000*Cursi*curFrameComplex) (4);

[0107] Among them, MBH = 1280 / 16 = 80, MBW = 720 / 16 = 45.

[0108] Step S1044: Determine the adjustment value based on the first control factor and the second control factor.

[0109] In implementation, different values ​​can be preset for each interval based on the first control factor, the second control factor, and multiple preset intervals, thereby determining different adjustment values.

[0110] Specifically, when the first control factor is greater than the first threshold and less than or equal to the second threshold, the adjustment value is determined as the preset first adjustment value; when the first control factor is greater than the second threshold and the second control factor is less than or equal to the third threshold, the adjustment value is determined as the preset second adjustment value; when the first control factor is greater than the second threshold and the second control factor is greater than the third threshold and less than or equal to the fourth threshold, the adjustment value is determined as the preset third adjustment value; and when the first control factor is greater than the second threshold and the second control factor is greater than the fourth threshold, the adjustment value is determined as the preset fourth adjustment value.

[0111] For example, taking a first threshold value of 0.01, a second threshold value of 0.02, a third threshold value of 0.65, and a fourth threshold value of 0.95 as an example.

[0112] When 0.01 < ctrFactor ≤ 0.02, the adjustment value is set to -1, which means the current quantization parameter is reduced by 1.

[0113] When ctrFactor > 0.02 and ctrFactor_plus ≤ 0.65, the adjustment value is set to -2, which means subtracting 2 from the current quantization parameter;

[0114] When ctrFactor > 0.02 and 0.65 < ctrFactor_plus ≤ 0.95, the adjustment value is set to -3, which means subtracting 3 from the current quantization parameter;

[0115] When ctrFactor > 0.02 and ctrFactor_plus > 0.95, the adjustment value is set to -4, which means subtracting 4 from the current quantization parameter.

[0116] Step S1045: Adjust the quantization parameters according to the adjustment value to obtain the adjusted quantization parameters.

[0117] After determining the adjustment value, the quantization parameters of the current frame to be encoded are adjusted according to the adjustment value to obtain the adjusted quantization parameters. The adjusted quantization parameters are smaller than the original quantization parameters, making the quantization more refined. This reduces the distortion of the current frame to be encoded, improves the quality of the image frame, and thus improves the video quality.

[0118] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0119] With the rapid development of digital technology, people have increasingly higher requirements for video quality. In existing technologies, fixed quantization parameters are used when encoding video frames, which affects the subjective evaluation of video quality.

[0120] To address the aforementioned problems in the prior art, this application provides a method and apparatus for optimizing the subjective quality of video conferencing scenarios based on frame complexity analysis. It employs the principles of image complexity analysis and calculates empirical data using formulas based on the analysis, thereby adjusting frame-level quantization parameters to improve the subjective quality of the video.

[0121] The solution provided in this application is described in detail below.

[0122] Sad represents the pixel-level absolute error of the image. Through encoder analysis and encoder characteristic analysis, the quality control of the video frame level is achieved, and the subjective quality optimization of the video is realized, without increasing the complexity as much as possible.

[0123] Video conferencing encoded frame structure: IPPPP P......PIPPPP P......

[0124] Figure 4 This is a schematic diagram illustrating the principle of a subjective quality optimization system for video conferencing scenarios based on frame complexity analysis, provided in an embodiment of this application. Figure 4 The detailed process of the solution provided in the embodiments of this application is as follows:

[0125] 1) Starting from the kth frame (k>1, i.e. from the 2nd frame), divide the kth frame into 16x16 segments, calculate Sad with its reference frame, and sum the results block by block to obtain the complexity curFrameCompl ex of the current frame to be encoded. The Sad calculation formula is as follows (5):

[0126]

[0127] Where f(x,y) is the original pixel value at (x,y), and g(x,y) is the reconstructed pixel value at (x,y).

[0128] 2) Obtain the quantization step size Qstep of the current encoding process.

[0129] In some embodiments, to further improve the subjective quality of the video, if Qstep is greater than 0, its reciprocal can be used to obtain reverQstep, and the adjustment value can be calculated using the reciprocal reverQstep.

[0130] 3) Obtain the inter-frame macroblock in the (k-1)th frame, perform motion estimation, and sum the Sad values, denoted as meSadAll;

[0131] 4) Obtain the bitrate pre-allocated to each frame, defined as perFrameBits, to determine whether to perform current algorithm adjustment. If perFrameBits > (target bitrate / frame rate) * 1.2, enter the encoding algorithm adjustment and execute 5); otherwise, do not enter the algorithm adjustment and execute 8).

[0132] 5) Customize the first control factor, ctrFactor

[0133] ctrFactor = (hFrameComplex + vFrameComplex) / meSadAll, where hFrameComplex and vFrameComplex are the vertical and horizontal complexity, respectively, and can be obtained directly from the current encoder.

[0134] 6) Custom second control factor ctrFactor_plus

[0135] Cursi=100*(hFrameComplex+vFrameComplex) / (Qstep*MBH*MBW), ctrFactor_plus=meSadAll / (1000*Cursi*curFrameComplex), where MBH=1280 / 16=80 and MBW=720 / 16=45.

[0136] 7) Based on condition 4), and in conjunction with ctrFactor and ctrFactor_plus, adjust the quantization parameters of the current encoded frame. The adjustment strategy is as follows:

[0137] ① If 0.01 < ctrFactor ≤ 0.02, the quantization parameter iLumaQp of the current frame = iLumaQp-1;

[0138] ②If ctrFactor > 0.02 and ctrFactor_plus ≤ 0.65, then set iLumaQp = iLumaQp - 2;

[0139] ③ If ctrFactor > 0.02 and 0.65 < ctrFactor_plus ≤ 0.95, then set iLumaQp = iLumaQp - 3;

[0140] ④ If ctrFactor > 0.02 and ctrFactor_plus > 0.95, then set iLumaQp = iLumaQp - 4.

[0141] 8) Continue with the subsequent coding process.

[0142] The following tests were conducted using objective data BD-RATE and subjective quality, and the results are shown in the table below:

[0143] Table 1 Test results of video frames at different resolutions

[0144] sequence resolution BD-RATE Test Sequence 1 1920x1080 -1.52％ Test Sequence 2 1280x720 -2.11％ Test sequence 3 720x1280 -0.87％ AVG / -1.50％

[0145] BD-RATE represents the percentage reduction in bitrate for the same quality.

[0146] Figure 5a This is a schematic diagram obtained by processing the example image using the original quantization parameters. Figure 5b This is a schematic diagram obtained by processing an example image using the quantization parameters obtained through the video processing method provided in this application embodiment, for comparison. Figure 5a and Figure 5b As can be seen, by using the method provided in the embodiments of this application, the subjective quality of the video is improved while maintaining a basically consistent bitrate.

[0147] Based on the foregoing embodiments, this application provides a video processing device. The various modules and units included in the device can be implemented by a processor in a computer device; of course, they can also be implemented by specific logic circuits. In the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPG A), etc.

[0148] This application embodiment further provides a video processing apparatus. Figure 6 This is a schematic diagram of the composition structure of a video processing apparatus provided in an embodiment of this application, such as... Figure 6 As shown, the video processing device 600 includes:

[0149] The first acquisition module 601 is used to acquire the pre-allocated bitrate and the preset target bitrate of the current frame to be encoded in the video to be processed;

[0150] The second acquisition module 602 is used to acquire the encoding complexity and quantization parameters of the current frame to be encoded when the adjustment conditions for satisfying the quantization parameters are determined based on the pre-allocated bitrate and the target bitrate.

[0151] The third acquisition module 603 is used to acquire the sum of absolute errors of macroblocks between frames during motion estimation of the previous encoded frame in the video to be processed;

[0152] The adjustment module 604 is used to adjust the quantization parameters according to the coding complexity and the sum of absolute errors of the inter-frame macroblocks to obtain the adjusted quantization parameters.

[0153] The encoding module 605 is used to encode the current frame to be encoded according to the adjusted quantization parameters to obtain the encoding result.

[0154] In some embodiments, the coding complexity includes frame complexity, vertical complexity, and horizontal complexity;

[0155] The second acquisition module 602 is further configured to:

[0156] Obtain the reference frame of the current frame to be encoded from the encoding device;

[0157] The frame complexity of the current frame to be encoded is determined based on the current frame to be encoded and the reference frame.

[0158] The vertical and horizontal complexity of the current frame to be encoded are obtained from the encoding device.

[0159] In some embodiments, the second acquisition module 602 is further configured to:

[0160] The current frame to be encoded is divided into multiple encoding blocks;

[0161] Obtain the original pixel value of each coded block and the reconstructed pixel value of the corresponding block in the reference frame for each coded block;

[0162] The absolute error of each coding block is determined based on the original pixel value of each coding block and the reconstructed pixel value of the corresponding position block of each coding block;

[0163] The frame complexity of the current frame to be encoded is determined based on the absolute error of each coded block.

[0164] In some embodiments, the adjustment module 604 is further configured to:

[0165] The first control factor is determined based on the vertical complexity, the horizontal complexity, and the sum of the absolute errors of each inter-frame macroblock.

[0166] Obtain the quantization step size of the current encoding and the number of encoding blocks obtained by dividing the current frame to be encoded;

[0167] The second control factor is determined based on the vertical complexity, horizontal complexity, quantization step size, number of coding blocks, absolute error of each inter-frame macroblock, and frame complexity.

[0168] The adjustment value is determined based on the first control factor and the second control factor;

[0169] The quantization parameters are adjusted according to the adjustment value to obtain the adjusted quantization parameters.

[0170] In some embodiments, the adjustment module 604 is further configured to:

[0171] The absolute errors of each inter-frame macroblock are summed to obtain the cumulative sum of absolute errors;

[0172] The sum of the vertical complexity and the horizontal complexity is determined as the directional complexity.

[0173] The quotient of the directional complexity and the sum of the absolute errors is determined as the first control factor.

[0174] In some embodiments, the adjustment module 604 is further configured to:

[0175] The control sub-factor is determined based on the vertical complexity, horizontal complexity, quantization step size, and number of coding blocks.

[0176] The second control factor is determined based on the sum of the absolute errors of each inter-frame macroblock, the cumulative sum of absolute errors, the control sub-factor, and the frame complexity.

[0177] In some embodiments, the adjustment module 604 is further configured to:

[0178] When the first control factor is greater than the first threshold and less than or equal to the second threshold, the adjustment value is determined as the preset first adjustment value;

[0179] When the first control factor is greater than the second threshold and the second control factor is less than or equal to the third threshold, the adjustment value is determined as the preset second adjustment value;

[0180] When the first control factor is greater than the second threshold, and the second control factor is greater than the third threshold and less than or equal to the fourth threshold, the adjustment value is determined as the preset third adjustment value;

[0181] When the first control factor is greater than the second threshold and the second control factor is greater than the fourth threshold, the adjustment value is determined as the preset fourth adjustment value.

[0182] It should be noted that the descriptions of the above video processing apparatus embodiments are similar to the descriptions of the methods described above, and have the same beneficial effects as the method embodiments. For technical details not disclosed in the video processing apparatus embodiments of this application, those skilled in the art should refer to the descriptions of the method embodiments of this application for understanding.

[0183] It should be noted that, in the embodiments of this application, if the above methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0184] Accordingly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps in the video processing method provided in the above embodiments.

[0185] This application provides an electronic device. Figure 7 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application. Figure 7 The exemplary structure of the electronic device 700 shown can be used to deduce other exemplary structures of the electronic device 700. Therefore, the structure described herein should not be regarded as a limitation. For example, some components described below may be omitted, or components not described below may be added to suit the specific needs of certain applications.

[0186] Figure 7 The illustrated electronic device 700 includes: a processor 701, at least one communication bus 702, a user interface 703, at least one external communication interface 704, and a memory 705. The communication bus 702 is configured to enable communication between these components. The user interface 703 may include a display screen, and the external communication interface 704 may include standard wired and wireless interfaces. The processor 701 is configured to execute a program of a video processing method stored in the memory to implement the steps of the video processing method provided in the above embodiments.

[0187] The descriptions of the above embodiments of the electronic devices and storage media are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the embodiments of the electronic devices and storage media of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0188] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0189] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0190] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0191] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0192] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0193] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0194] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0195] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A video processing method, characterized in that, The method includes: Obtain the pre-allocated bitrate and the preset target bitrate of the current frame to be encoded in the video to be processed; When the adjustment conditions for satisfying the quantization parameters are determined based on the pre-allocated bitrate and the target bitrate, the encoding complexity and quantization parameters of the current frame to be encoded are obtained. The sum of absolute errors of macroblocks between frames during motion estimation of the previous encoded frame in the video to be processed is obtained. The quantization parameters are adjusted based on the coding complexity and the sum of absolute errors of the inter-frame macroblocks to obtain the adjusted quantization parameters. The current frame to be encoded is encoded according to the adjusted quantization parameters to obtain the encoding result.

2. The method according to claim 1, characterized in that, The encoding complexity includes frame complexity, vertical complexity, and horizontal complexity; The step of obtaining the encoding complexity of the current frame to be encoded includes: Obtain the reference frame of the current frame to be encoded from the encoding device; The frame complexity of the current frame to be encoded is determined based on the current frame to be encoded and the reference frame. The vertical and horizontal complexity of the current frame to be encoded are obtained from the encoding device.

3. The method according to claim 2, characterized in that, The step of determining the frame complexity of the current frame to be encoded based on the current frame to be encoded and the reference frame includes: The current frame to be encoded is divided into multiple encoding blocks; Obtain the original pixel value of each coded block and the reconstructed pixel value of the corresponding block in the reference frame for each coded block; The absolute error of each coding block is determined based on the original pixel value of each coding block and the reconstructed pixel value of the corresponding position block of each coding block; The frame complexity of the current frame to be encoded is determined based on the absolute error of each coded block.

4. The method according to claim 2, characterized in that, The step of adjusting the quantization parameters based on the coding complexity and the sum of absolute errors of the inter-frame macroblocks to obtain the adjusted quantization parameters includes: The first control factor is determined based on the vertical complexity, the horizontal complexity, and the sum of the absolute errors of each inter-frame macroblock. Obtain the quantization step size of the current encoding and the number of encoding blocks obtained by dividing the current frame to be encoded; The second control factor is determined based on the vertical complexity, horizontal complexity, quantization step size, number of coding blocks, absolute error of each inter-frame macroblock, and frame complexity. The adjustment value is determined based on the first control factor and the second control factor; The quantization parameters are adjusted according to the adjustment value to obtain the adjusted quantization parameters.

5. The method according to claim 4, characterized in that, The step of determining the first control factor based on the vertical complexity, the horizontal complexity, and the sum of the absolute errors of each inter-frame macroblock includes: The absolute errors of each inter-frame macroblock are summed to obtain the cumulative sum of absolute errors; The sum of the vertical complexity and the horizontal complexity is determined as the directional complexity. The quotient of the directional complexity and the cumulative sum of the absolute errors is determined as the first control factor.

6. The method according to claim 4, characterized in that, The determination of the second control factor based on the vertical complexity, horizontal complexity, quantization step size, number of coding blocks, frame complexity, and the sum of absolute errors of the macroblocks between frames includes: The control sub-factor is determined based on the vertical complexity, horizontal complexity, quantization step size, and number of coding blocks. The second control factor is determined based on the sum of the absolute errors of each inter-frame macroblock, the cumulative sum of absolute errors, the control sub-factor, and the frame complexity.

7. The method according to claim 4, characterized in that, The step of determining the adjustment value based on the first control factor and the second control factor includes: When the first control factor is greater than the first threshold and less than or equal to the second threshold, the adjustment value is determined as the preset first adjustment value; When the first control factor is greater than the second threshold and the second control factor is less than or equal to the third threshold, the adjustment value is determined as the preset second adjustment value; When the first control factor is greater than the second threshold, and the second control factor is greater than the third threshold and less than or equal to the fourth threshold, the adjustment value is determined as the preset third adjustment value; When the first control factor is greater than the second threshold and the second control factor is greater than the fourth threshold, the adjustment value is determined as the preset fourth adjustment value.

8. A video processing apparatus, characterized in that, The device includes: The first acquisition module is used to acquire the pre-allocated bitrate and the preset target bitrate of the current frame to be encoded in the video to be processed; The second acquisition module is used to acquire the encoding complexity and quantization parameters of the current frame to be encoded when the adjustment conditions for satisfying the quantization parameters are determined based on the pre-allocated bitrate and the target bitrate. The third acquisition module is used to acquire the sum of absolute errors of macroblocks between frames during motion estimation of the previous encoded frame in the video to be processed; The adjustment module is used to adjust the quantization parameters according to the encoding complexity and the sum of absolute errors of the inter-frame macroblocks to obtain the adjusted quantization parameters. The encoding module is used to encode the current frame to be encoded according to the adjusted quantization parameters to obtain the encoding result.

9. An electronic device, characterized in that, include: processor; as well as Memory for storing computer programs that can run on the processor; When the computer program is executed by a processor, it implements the steps of the video processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The device stores computer-executable instructions configured to perform the steps of the video processing method according to any one of claims 1 to 7.

Images