Video encoding method, device and electronic equipment

Abstract

The present application provides a video encoding method, apparatus, and electronic device, wherein the method includes: determining the frame level of a video frame according to a reference relationship of video frames in a picture group of a video stream; assigning an initial quantization parameter (QP) to the video frame according to the frame level of the video frame; taking a video frame whose frame level is lower than a preset frame level threshold as a target video frame, obtaining the image quality of the target video frame, and adjusting the initial QP of the target video frame to an optimized QP according to the image quality; encoding other video frames in the picture group except the target video frame according to the initial QP of the other video frames; and encoding the target video frame in the picture group according to the optimized QP of the target video frame. The present application can consider both the reference relationship of the video frames and the image quality of the video frames when assigning QPs to video frames before video encoding, adjust the QP according to the image quality, and reduce the bit rate waste of video frames with poor image quality.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a video encoding method, device and electronic device. Background Art

[0002] In video coding, entire video sequences are typically encoded using GOPs (Group of Pictures). Within each GOP, video frames are divided into different levels based on reference relationships, and each level is assigned a different frame-level Quantizer Parameter (QP) value. Current QP allocation strategies use empirical values: the more references a particular frame has within a GOP, the lower its level and the lower the QP value assigned. Alternatively, QP values ​​are allocated based on a rough reference relationship derived from pre-analysis for actual encoding.

[0003] However, while some frames may serve as reference frames for many frames within a GOP, if their image quality is poor, they are of little use as reference for subsequent frames. Setting a small quantization parameter (QP) for these frames wastes significant bitrate. Existing techniques allocate bitrate based solely on reference relationships, resulting in bitrate waste for low-quality images. Summary of the Invention

[0004] The purpose of this application is to provide a video encoding method, device and electronic device to solve the above technical problems.

[0005] In a first aspect, an embodiment of the present application provides a video encoding method, the method comprising: determining a frame level of a video frame according to a reference relationship of video frames in a picture group of a video stream; assigning an initial quantization parameter QP to the video frame according to the frame level of the video frame; taking a video frame whose frame level is lower than a preset frame level threshold as a target video frame, obtaining an image quality of the target video frame, and adjusting the initial QP of the target video frame to an optimized QP according to the image quality; encoding other video frames in the picture group except the target video frame according to the initial QP of the other video frames; and encoding the target video frame in the picture group according to the optimized QP of the target video frame.

[0006] Furthermore, each of the above-mentioned frame levels corresponds to a preset QP; the step of assigning an initial quantization parameter QP to the video frame according to the frame level of the video frame includes: determining the preset QP corresponding to the frame level as the initial QP corresponding to the video frame according to the frame level of the video frame; or determining the optimal QP corresponding to each frame level according to the preset QP corresponding to each frame level and the attribute information of the video frame contained in each frame level; determining the optimal QP corresponding to the frame level as the initial QP corresponding to the video frame according to the frame level of the video frame; wherein the attribute information includes: the number of header information bits of the video frame, the video frame coding complexity and macroblock information.

[0007] Furthermore, the above-mentioned step of obtaining the image quality of the target video frame includes: calculating a mean square error value corresponding to the target video frame based on a first pixel value corresponding to the target video frame and a second pixel value of the target video frame; wherein the first pixel value is an initial pixel value before encoding the target video frame, and the second pixel value is a pixel value after encoding based on an initial QP corresponding to the target video frame; or, the first pixel value is a pixel value after encoding based on a preset QP corresponding to the target video frame, and the second pixel value is a pixel value after encoding based on the initial QP corresponding to the target video frame; and determining the image quality of the target video frame based on the mean square error value.

[0008] Furthermore, the step of calculating the mean square error value corresponding to the target video frame based on the first pixel value corresponding to the target video frame and the second pixel value of the target video frame includes: calculating the mean square error value corresponding to the target video frame according to the following formula:

[0009]

[0010] Among them, MSE represents the mean square error value corresponding to the target video frame; mn represents a video frame image of size m*n, I(i, j) and K(i, j) respectively represent the first pixel value and the second pixel value of the target video frame at the pixel position (i, j).

[0011] Furthermore, the above step of determining the optimal QP corresponding to each frame level according to the preset QP corresponding to each frame level and the attribute information of the video frame contained in each frame level includes: setting the preset QP corresponding to each frame level as Q l , the optimal QP is Q l ', l represents the frame level, l = 0, 1, 2...L-1; L represents the number of frame levels; according to the header information bit rate and video frame coding complexity of each frame level, and the Q corresponding to each frame level l and Q l ', determine the sum of the bit rate changes corresponding to each frame level; according to the macroblock information of the video frame corresponding to each frame level, the Q corresponding to each frame level l and Q l', determine the sum of coding losses corresponding to each frame level; find the Q when the sum of code rate changes corresponding to each frame level is less than or equal to 0 and the sum of coding losses corresponding to each frame level is the smallest l ', and get the optimal QP corresponding to each frame level.

[0012] Furthermore, the header information bit rate and video frame coding complexity of the video frame corresponding to each frame level, as well as the Q corresponding to each frame level are l and Q l ', the step of determining the sum of the bit rate changes corresponding to each frame level includes: for each video frame corresponding to each frame level, the following operations are performed: the video frame is calculated in Q according to the following formula l The first bit rate under the effect of Q l 'The second code rate under action:

[0013]

[0014] Where R represents the bit rate of the video frame under the effect of QP; H represents the number of header information bits of the video frame; S represents the encoding complexity of the video frame; a and b represent adjustment parameters respectively; the difference between the first bit rate and the second bit rate is calculated to obtain the bit rate change corresponding to the video frame; the bit rate changes of all video frames corresponding to each frame level are summed to obtain the bit rate change corresponding to each frame level; the bit rate changes corresponding to each frame level are summed to obtain the total bit rate change corresponding to each frame level.

[0015] Furthermore, the above-mentioned macroblock information of the video frame corresponding to each frame level, the Q corresponding to each frame level l and Q l ', the step of determining the sum of coding losses corresponding to each frame level includes: for each video frame corresponding to each frame level, the following operations are performed: the video frame is calculated in Q according to the following formula l The first coding loss is affected, and the video frame is in Q l 'The second coding loss under the action:

[0016]

[0017] Where D represents the coding loss of the video frame under the effect of QP; N represents the number of macroblocks contained in the video frame; A i Indicates the loss ratio of the i-th macroblock in the video frame; Q i Indicates the quantization parameter value corresponding to the i-th macroblock, which is based on A i The QP is adjusted; the difference between the first coding loss and the second coding loss is calculated to obtain the coding loss corresponding to the video frame; the average coding loss of all video frames corresponding to each frame level is calculated; and the coding loss corresponding to each frame level is obtained according to the following formula:

[0018]

[0019] Where ΔD tot，l represents the coding loss corresponding to frame level l, ΔD l represents the average value of the coding loss of all video frames corresponding to frame level l, δ = 0.5 represents the reference coefficient between different frame levels; the coding loss corresponding to each frame level is summed to obtain the total coding loss corresponding to each frame level.

[0020] Furthermore, the sum of the bit rate changes corresponding to each frame level is less than or equal to 0, and the sum of the coding losses corresponding to each frame level is the smallest Q l ', the step of obtaining the optimal QP corresponding to each frame level includes: solving the optimal QP corresponding to each frame level by Lagrange multiplier method according to the following formula:

[0021]

[0022] Where l represents the frame level, l = 0, 1, 2...L-1; ΔD tot，l represents the coding loss corresponding to frame level l, ΔR tot，l Indicates the bit rate change corresponding to frame level l; Q l ' indicates the optimal QP corresponding to each frame level when the sum of the bit rate changes corresponding to each frame level is less than or equal to 0 and the coding loss corresponding to each frame level is minimized.

[0023] Furthermore, the step of adjusting the initial QP of the target video frame to the optimized QP according to the image quality includes: if the image quality exceeds a preset threshold, increasing the initial QP to obtain the optimized QP corresponding to the target video frame.

[0024] Furthermore, the step of increasing the initial QP to obtain an optimized QP corresponding to the target video frame includes: determining a QP increase amplitude value corresponding to the target video frame based on the image quality of the target video frame and a preset threshold; and adding the initial QP corresponding to the target video frame to the QP increase amplitude value to obtain the optimized QP corresponding to the target video frame.

[0025] Furthermore, the step of determining the QP increase amplitude value corresponding to the target video frame based on the image quality of the target video frame and the preset threshold includes: determining the QP increase amplitude value corresponding to the target video frame according to the following formula:

[0026]

[0027] Among them, ΔQ represents the QP increase amplitude value corresponding to the target video frame; IQA l Indicates the image quality corresponding to the target video frame; IQA yu Indicates the preset threshold corresponding to the image quality; in, Represents the average quantization step size.

[0028] In a second aspect, an embodiment of the present application further provides a video encoding device, comprising: a frame level determination module, configured to determine the frame level of a video frame according to a reference relationship of video frames in a picture group of a video stream; an initial QP allocation module, configured to allocate an initial quantization parameter QP to the video frame according to the frame level of the video frame; a QP adjustment module, configured to take a video frame whose frame level is lower than a preset frame level threshold as a target video frame, obtain the image quality of the target video frame, and adjust the initial QP of the target video frame to an optimized QP according to the image quality; an encoding module, configured to encode other video frames in the picture group except the target video frame according to the initial QP of the other video frames; and encode the target video frame in the picture group according to the optimized QP of the target video frame.

[0029] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the method described in the first aspect above.

[0030] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method described in the first aspect above.

[0031] The embodiments of the present application bring the following beneficial effects:

[0032] The present application provides a video encoding method, apparatus, and electronic device. The method first determines the frame level of a video frame according to a reference relationship of video frames in a picture group of a video stream; then allocates an initial quantization parameter (QP) to the video frame according to the frame level of the video frame; then, a video frame whose frame level is lower than a preset frame level threshold is used as a target video frame, the image quality of the target video frame is obtained, and the initial QP of the target video frame is adjusted to an optimized QP according to the image quality; finally, the other video frames in the picture group except the target video frame are encoded according to the initial QP of the other video frames; and the target video frame in the picture group is encoded according to the optimized QP of the target video frame. This method considers both the reference relationship and the image quality of video frames when allocating QPs of video frames before video encoding. It evaluates the image quality of target video frames whose frame levels are lower than a preset frame level threshold, and adjusts the initial QP of the target video frames according to the image quality to obtain an optimized QP. That is, the initial QP is corrected when the image quality is poor, thereby reducing the bit rate waste of video frames with poor image quality. At the same time, the image quality of other video frames whose frame levels are not lower than the preset frame level threshold is not calculated, and they are encoded according to the initial QP, saving computational effort.

[0033] Other features and advantages of the embodiments of the present application will be described in the following description, and in part will become apparent from the description or be understood by practicing the present application. The objectives and other advantages of the present application are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0034] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the specific implementation methods or the description of the prior art. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0036] Figure 1 A flowchart of a video encoding method provided in an embodiment of the present application;

[0037] Figure 2 A picture group prediction structure diagram provided in an embodiment of the present application;

[0038] Figure 3 A flowchart of another video encoding method provided in an embodiment of the present application;

[0039] Figure 4 A structural block diagram of a video encoding device provided in an embodiment of the present application;

[0040] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will clearly and completely describe the technical solutions of this application in conjunction with the embodiments. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0042] Current video coding techniques allocate quantization parameter values ​​for video frames within a group of pictures based on empirical data. The more references a particular frame receives within the group of pictures, the lower its grade, and the lower the quantization parameter value assigned. Alternatively, quantization parameter values ​​are allocated based on a rough reference relationship derived from pre-analysis for actual encoding. However, existing techniques allocate bitrates based solely on this reference relationship, resulting in wasted bitrate for low-grade images with poor image quality.

[0043] Based on this, the embodiments of the present application provide a video encoding method, apparatus, and electronic device, which can consider the reference relationship of video frames and the image quality of video frames when allocating QP of video frames before video encoding, perform image quality evaluation on target video frames whose frame levels are lower than a preset frame level threshold, and adjust the initial QP of the target video frame according to the image quality to obtain an optimized QP, that is, correct the initial QP when the image quality is poor, thereby reducing the bit rate waste of video frames with poor image quality. At the same time, for other video frames whose frame levels are not lower than the preset frame level threshold, the image quality is not calculated, and they are encoded according to the initial QP, saving computational effort.

[0044] To facilitate understanding of this embodiment, a video encoding method disclosed in an embodiment of the present application is first introduced in detail.

[0045] The present application provides a video encoding method. Figure 1 The video encoding method shown in FIG. 1 includes the following steps:

[0046] Step S102 : determining the frame level of the video frame according to the reference relationship of the video frames in the picture group of the video stream.

[0047] In video coding technology, the entire video sequence is usually encoded in units of Group of Pictures (GOPs). Each GOP includes multiple consecutive video frames, and video frames in different GOPs are non-repeated. Video frames in a GOP are divided into three types based on reference relationships: I, P, and B. I frames are key frames, which are inter-frame compressed video frames that can be individually decoded into a complete picture. P frames are predictively coded image frames that reference previous I or P frames and use motion prediction for inter-frame coding. B frames are bidirectionally predictively coded image frames that require bidirectional reference, meaning there are two reference frames before and after. Different types of frames are referenced differently. The frame with the least number of references is determined as the lowest-level frame, and the frame with the most references is determined as the highest-level frame. This is how the frame level of each video frame is determined. This frame level is used to characterize the degree to which the video frame is referenced.

[0048] like Figure 2 As shown, in the prediction structure diagram of a picture group, the 0th layer video frame with the most reference times is determined as the lowest level frame, that is, the frame level corresponding to the 0th layer video frame is level 0; the 3rd layer video frame with the least reference times (that is, the frame that will not be referenced at all) is determined as the highest level frame, that is, the frame level corresponding to the 3rd layer video frame is level 3.

[0049] Step S104 : allocating an initial quantization parameter QP to the video frame according to the frame level of the video frame.

[0050] There are many ways to assign an initial QP to a video frame. For example, each frame level corresponds to a preset QP. The preset QP corresponding to the frame level can be determined as the initial QP corresponding to the video frame based on the frame level of the video frame. Generally speaking, the more video frames are referenced and the lower the frame level, the smaller the corresponding QP should be, so that a larger bit rate can be achieved. According to this logic, the QP corresponding to each frame level can be pre-set, that is, the preset QP mentioned above, and the preset QP can be used as the starting point for QP adjustment, that is, the initial QP mentioned above.

[0051] Alternatively, the optimal QP for each frame level can be determined based on the preset QPs for each frame level and the attribute information of the video frame contained at each frame level. Based on the frame level of the video frame, the optimal QP for the frame level is determined as the initial QP for the video frame. The attribute information includes the number of header information bits, the video frame coding complexity, and macroblock information. This process actually uses an optimization algorithm to calculate the optimal QP for each frame level, and then uses the optimal QP as the starting point for QP adjustment, namely the initial QP.

[0052] Step S106 : taking a video frame whose frame level is lower than a preset frame level threshold as a target video frame, obtaining image quality of the target video frame, and adjusting the initial QP of the target video frame to an optimized QP according to the image quality.

[0053] When allocating quantization parameter values ​​at the frame level, although some frames may serve as reference frames for many frames in the picture group, if their own image quality is poor, they have little reference value for subsequent frames and it is not worth setting a smaller quantization parameter value to consume more bit rate. Therefore, image quality evaluation is required for low-level frames to determine the actual quantization parameter value allocation; for the highest level (i.e., frames that will not be referenced at all), no quality evaluation is required and quantization parameter values ​​are allocated according to the original strategy to save computational effort.

[0054] Low-level frames are video frames whose frame levels are lower than the preset frame level threshold. For example, the frame levels corresponding to the video frames in the picture group are 0 to 4, and the preset frame level threshold is 4. Then the video frames corresponding to levels 0, 1, 2, and 3 are all target video frames, and only the video frames corresponding to level 4 are other video frames that do not require quality evaluation. The setting of the preset frame level threshold can be adjusted differently according to actual conditions.

[0055] There are many ways to obtain the image quality of the target video frame, for example, it can be the mean square error value corresponding to the video frame, or it can be PSNR (Peak Signal to Noise Ratio), etc., which will not be repeated here.

[0056] Step S108 , encoding the other video frames except the target video frame in the GOP according to their initial QPs; encoding the target video frame in the GOP according to its optimized QP.

[0057] The process of encoding based on the optimized QP of the target video frame and the initial QPs of other video frames is the same as that in the prior art and will not be described in detail here.

[0058] The video encoding method provided in the embodiment of the present application can consider both the reference relationship of the video frames and the image quality of the video frames when allocating QPs of video frames before video encoding. It performs image quality evaluation on target video frames whose frame levels are lower than a preset frame level threshold, and adjusts the initial QP of the target video frames based on the image quality to obtain an optimized QP. That is, the initial QP is corrected when the image quality is poor, thereby reducing the bit rate waste of video frames with poor image quality. At the same time, the image quality of other video frames whose frame levels are not lower than the preset frame level threshold is not calculated, and they are encoded according to the initial QP, thereby saving computational effort.

[0059] The present application also provides another video encoding method, which is implemented based on the above embodiment method; the method focuses on the initial QP allocation process, the image quality determination process, and the QP adjustment process, see Figure 3The video encoding method shown in FIG. 1 includes the following steps:

[0060] Step S302: Determine the frame level of the video frame according to the reference relationship of the video frames in the picture group of the video stream. This step is the same as the above step S102 and will not be repeated here.

[0061] Step S304 , determining the optimal QP corresponding to each frame level based on the preset QP corresponding to each frame level and the attribute information of the video frame contained in each frame level; wherein the attribute information includes: the number of header information bits of the video frame, the video frame coding complexity and the macroblock information.

[0062] In this step, the optimal QP corresponding to each frame level is actually solved by using the above-mentioned multiple information and optimization algorithms. For example, under the constraints of "the sum of the bit rate changes corresponding to each frame level is less than or equal to 0, and the sum of the coding losses corresponding to each frame level is minimized", the optimal quantization parameter value corresponding to each frame level is found. The specific calculation process is as follows:

[0063] (1) Let the preset QP corresponding to each frame level be Q l , the optimal QP is Q l ', l represents the frame level, l=0,1,2…L-1; L represents the number of frame levels;

[0064] (2) According to the header information bit rate and video frame coding complexity of each frame level, and the Q l and Q l ', determine the total bit rate change corresponding to each frame level.

[0065] Specifically, for each video frame corresponding to each frame level, the following operations are performed:

[0066] According to the following formula, calculate the video frame in Q l The first bit rate under the effect of Q l 'The second code rate under action:

[0067]

[0068] Where R represents the bit rate of the video frame under the effect of QP; H represents the number of header information bits of the video frame; S represents the encoding complexity of the video frame; a and b represent adjustment parameters respectively;

[0069] Calculate the difference between the first bit rate and the second bit rate to obtain the bit rate change corresponding to the video frame;

[0070] Sum the bitrate changes of all video frames corresponding to each frame level to obtain the bitrate change corresponding to each frame level;

[0071] The bit rate changes corresponding to each frame level are summed to obtain the total bit rate changes corresponding to each frame level.

[0072] (3) According to the macroblock information of the video frame corresponding to each frame level, the Q corresponding to each frame level l and Q l ', determine the sum of coding losses corresponding to each frame level;

[0073] Specifically, for each video frame corresponding to each frame level, the following operations are performed:

[0074] According to the following formula, calculate the video frame in Q l The first coding loss is affected, and the video frame is in Q l 'The second coding loss under the action:

[0075]

[0076] Where D represents the coding loss of the video frame under the effect of QP; N represents the number of macroblocks contained in the video frame; A i Indicates the loss ratio of the i-th macroblock in the video frame; Q i Indicates the quantization parameter value corresponding to the i-th macroblock, which is based on A i The value after adjusting QP;

[0077] Calculating the difference between the first coding loss and the second coding loss to obtain the coding loss corresponding to the video frame;

[0078] Calculate the average value of the coding loss of all video frames corresponding to each frame level;

[0079] The coding loss corresponding to each frame level is obtained according to the following formula:

[0080]

[0081] Where ΔD tot，l represents the coding loss corresponding to frame level l, ΔD l represents the average value of the coding loss of all video frames corresponding to frame level l, δ = 0.5, represents the reference coefficient between different frame levels;

[0082] The coding losses corresponding to each frame level are summed to obtain the total coding losses corresponding to each frame level.

[0083] (4) Find the Q value when the sum of the bit rate changes corresponding to each frame level is less than or equal to 0 and the sum of the coding loss corresponding to each frame level is the minimum l ', and get the optimal QP corresponding to each frame level.

[0084] Specifically, the optimal QP corresponding to each frame level is solved by the Lagrange multiplier method according to the following formula:

[0085]

[0086] Where l represents the frame level, l = 0, 1, 2...L-1; ΔD tot，l represents the coding loss corresponding to frame level l, ΔR tot，l Indicates the bit rate change corresponding to frame level l; Q l ' indicates the optimal QP corresponding to each frame level when the sum of the bit rate changes corresponding to each frame level is less than or equal to 0 and the coding loss corresponding to each frame level is minimized.

[0087] Step S306 : According to the frame level of the video frame, determine the optimal QP corresponding to the frame level as the initial QP corresponding to the video frame.

[0088] Step S308: A video frame whose frame level is lower than a preset frame level threshold is used as a target video frame. A mean square error value corresponding to the target video frame is calculated based on a first pixel value and a second pixel value corresponding to the target video frame. The first pixel value is an initial pixel value before encoding the target video frame, and the second pixel value is a pixel value after encoding based on an initial QP corresponding to the target video frame. Alternatively, the first pixel value is a pixel value after encoding based on a preset QP corresponding to the target video frame, and the second pixel value is a pixel value after encoding based on the initial QP corresponding to the target video frame.

[0089] The calculation of the mean square error value here includes the following three cases:

[0090] 1) The first pixel value is the initial pixel value before encoding the target video frame, and the second pixel value is the pixel value after encoding based on the preset QP corresponding to the target video frame;

[0091] 2) The first pixel value is the initial pixel value before encoding the target video frame, and the second pixel value is the pixel value after encoding based on the optimal QP corresponding to the target video frame;

[0092] 3) The first pixel value is a pixel value encoded based on a preset QP corresponding to the target video frame; the second pixel value is a pixel value encoded based on an optimal QP corresponding to the target video frame.

[0093] Specifically, the mean square error value corresponding to the target video frame can be calculated according to the following formula:

[0094]

[0095] Among them, MSE represents the mean square error value corresponding to the target video frame; mn represents a video frame image of size m*n, I(i, j) and K(i, j) respectively represent the first pixel value and the second pixel value of the target video frame at the pixel position (i, j).

[0096] Step S310: determining the image quality of the target video frame according to the mean square error value.

[0097] In the embodiments of the present application, there are two ways to determine image quality: the first is to directly use the mean square error value as the image quality of the target video frame; the second is to further calculate the corresponding PSNR based on the mean square error value of the target video frame, and use the PSNR as the image quality of the target video frame.

[0098] Step S312: If the image quality exceeds a preset threshold, the initial QP is increased to obtain an optimized QP corresponding to the target video frame.

[0099] The image quality of the target video frame is determined based on the mean square error of the video frame. If the image quality exceeds a threshold, it means that the loss of the image after encoding the target video frame is visible to the human eye. The goal of encoding is to save bit rate as much as possible while the encoding loss is invisible to the human eye. Therefore, it is necessary to increase the initial QP corresponding to the target video frame to obtain the optimized QP corresponding to the target video frame. Using the optimized QP for encoding can reduce the bit rate of the target video frame.

[0100] In a preferred embodiment, the initial QP can be adjusted upward by the following process:

[0101] (1) According to the image quality of the target video frame and the preset threshold, the QP increase amplitude value corresponding to the target video frame is determined; specifically, the QP increase amplitude value corresponding to the target video frame can be determined according to the following formula:

[0102]

[0103] Among them, ΔQ represents the QP increase amplitude value corresponding to the target video frame; IQA l Indicates the image quality corresponding to the target video frame; IQA yu Indicates the preset threshold corresponding to the image quality; in, Represents the average quantization step size.

[0104] (2) Add the initial QP corresponding to the target video frame to the QP increase amplitude value to obtain the optimized QP corresponding to the target video frame.

[0105] In step S314, the other video frames in the GOP, except the target video frame, are encoded according to their initial QPs; and the target video frame in the GOP is encoded according to its optimized QP. This step is similar to step S108 and will not be repeated here.

[0106] The video encoding method provided in the embodiment of the present application can determine the optimal QP corresponding to each frame level when allocating quantization parameter values ​​for video frames based on the initial QP corresponding to each frame level in the picture group and some attribute information corresponding to each video frame, such as the header information bit rate, video frame coding complexity, and macroblock information. In this way, the initial QP corresponding to each video frame is determined. Then, for the target video frame at the lower frame level, its image quality is further detected. If the image quality is poor, the optimal QP of the target video frame needs to be increased. In this way, the bit rate allocation for video frames with poor image quality can be reduced, thereby effectively reducing bit rate waste while ensuring video quality.

[0107] Based on the above method embodiment, the present application embodiment also provides a video encoding device, see Figure 4 As shown, the apparatus includes: a frame level determination module 42, configured to determine the frame level of a video frame according to a reference relationship of video frames in a picture group of a video stream; an initial QP allocation module 44, configured to allocate an initial quantization parameter (QP) to the video frame according to the frame level of the video frame; a QP adjustment module 46, configured to take a video frame whose frame level is lower than a preset frame level threshold as a target video frame, obtain an image quality of the target video frame, and adjust the initial QP of the target video frame to an optimized QP according to the image quality; an encoding module 48, configured to encode other video frames in the picture group except the target video frame according to the initial QP of the other video frames; and encode the target video frame in the picture group according to the optimized QP of the target video frame.

[0108] The video encoding device provided in the embodiment of the present application can consider both the reference relationship of the video frames and the image quality of the video frames when allocating QPs of the video frames before video encoding. It performs image quality evaluation on target video frames whose frame levels are lower than a preset frame level threshold, and adjusts the initial QP of the target video frames based on the image quality to obtain an optimized QP. That is, the initial QP is corrected when the image quality is poor, thereby reducing the bit rate waste of video frames with poor image quality. At the same time, the image quality of other video frames whose frame levels are not lower than the preset frame level threshold is not calculated, and they are encoded according to the initial QP, thereby saving computational effort.

[0109] Each frame level corresponds to a preset QP. The initial QP allocation module 44 is further configured to: determine, based on the frame level of the video frame, the preset QP corresponding to the frame level as the initial QP corresponding to the video frame; or, determine, based on the preset QP corresponding to each frame level and attribute information of the video frame contained in each frame level, the optimal QP corresponding to the frame level as the initial QP corresponding to the video frame; wherein the attribute information includes: the number of header information bits of the video frame, the encoding complexity of the video frame, and macroblock information.

[0110] The QP adjustment module 46 is further configured to calculate a mean square error (MSE) value corresponding to the target video frame based on a first pixel value and a second pixel value corresponding to the target video frame; wherein the first pixel value is an initial pixel value before encoding the target video frame, and the second pixel value is a pixel value encoded based on the initial QP corresponding to the target video frame; or, alternatively, the first pixel value is a pixel value encoded based on a preset QP corresponding to the target video frame, and the second pixel value is a pixel value encoded based on the initial QP corresponding to the target video frame; and the image quality of the target video frame is determined based on the MSE value.

[0111] The QP adjustment module 46 is further configured to calculate a mean square error value corresponding to the target video frame according to the following formula:

[0112]

[0113] Among them, MSE represents the mean square error value corresponding to the target video frame; mn represents a video frame image of size m*n, I(i, j) and K(i, j) respectively represent the first pixel value and the second pixel value of the target video frame at the pixel position (i, j).

[0114] The initial QP allocation module 44 is further configured to set the preset QP corresponding to each frame level as Q l , the optimal QP is Q l ', l represents the frame level, l = 0, 1, 2...L-1; L represents the number of frame levels; according to the header information bit rate and video frame coding complexity of each frame level, and the Q corresponding to each frame level l and Q l ', determine the sum of the bit rate changes corresponding to each frame level; according to the macroblock information of the video frame corresponding to each frame level, the Q corresponding to each frame level l and Q l ', determine the sum of coding losses corresponding to each frame level; find the Q when the sum of code rate changes corresponding to each frame level is less than or equal to 0 and the sum of coding losses corresponding to each frame level is the smallest l ', and get the optimal QP corresponding to each frame level.

[0115] The initial QP allocation module 44 is further configured to perform the following operations for each video frame corresponding to each frame level: calculate the QP of the video frame according to the following formula: l The first bit rate under the effect of Q l 'The second code rate under action:

[0116]

[0117] Where R represents the bit rate of the video frame under the effect of QP; H represents the number of header information bits of the video frame; S represents the encoding complexity of the video frame; a and b represent adjustment parameters respectively; the difference between the first bit rate and the second bit rate is calculated to obtain the bit rate change corresponding to the video frame; the bit rate changes of all video frames corresponding to each frame level are summed to obtain the bit rate change corresponding to each frame level; the bit rate changes corresponding to each frame level are summed to obtain the total bit rate change corresponding to each frame level.

[0118] The initial QP allocation module 44 is further configured to perform the following operations for each video frame corresponding to each frame level: calculate the QP of the video frame according to the following formula: l The first coding loss is affected, and the video frame is in Q l 'The second coding loss under the action:

[0119]

[0120] Where D represents the coding loss of the video frame under the effect of QP; N represents the number of macroblocks contained in the video frame; A i Indicates the loss ratio of the i-th macroblock in the video frame; Q i Indicates the quantization parameter value corresponding to the i-th macroblock, which is based on A i The QP is adjusted; the difference between the first coding loss and the second coding loss is calculated to obtain the coding loss corresponding to the video frame; the average coding loss of all video frames corresponding to each frame level is calculated; and the coding loss corresponding to each frame level is obtained according to the following formula:

[0121]

[0122] Where ΔD tot，l represents the coding loss corresponding to frame level l, ΔD l represents the average value of the coding loss of all video frames corresponding to frame level l, δ = 0.5 represents the reference coefficient between different frame levels; the coding loss corresponding to each frame level is summed to obtain the total coding loss corresponding to each frame level.

[0123] The initial QP allocation module 44 is further configured to calculate the optimal QP corresponding to each frame level using the Lagrange multiplier method according to the following formula:

[0124]

[0125] Where l represents the frame level, l = 0, 1, 2...L-1; ΔD tot，l represents the coding loss corresponding to frame level l, ΔR tot，l Indicates the bit rate change corresponding to frame level l; Q l' indicates the optimal QP corresponding to each frame level when the sum of the bit rate changes corresponding to each frame level is less than or equal to 0 and the coding loss corresponding to each frame level is minimized.

[0126] The QP adjustment module 46 is further configured to increase the initial QP if the image quality exceeds a preset threshold, to obtain an optimized QP corresponding to the target video frame.

[0127] The QP adjustment module 46 is further configured to determine a QP increase value corresponding to the target video frame based on the image quality of the target video frame and a preset threshold; and to add the initial QP corresponding to the target video frame to the QP increase value to obtain an optimized QP corresponding to the target video frame.

[0128] The QP adjustment module 46 is further configured to determine the QP increase amplitude corresponding to the target video frame according to the following formula:

[0129]

[0130] Among them, ΔQ represents the QP increase amplitude value corresponding to the target video frame; IQA l Indicates the image quality corresponding to the target video frame; IQA yu Indicates the preset threshold corresponding to the image quality; in, Represents the average quantization step size.

[0131] The video encoding device provided in the embodiment of the present application has the same implementation principle and technical effects as those in the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the embodiment of the device, reference can be made to the corresponding content in the aforementioned method embodiment.

[0132] The present application also provides an electronic device, such as Figure 5 As shown, it is a structural diagram of the electronic device, wherein the electronic device includes a processor 51 and a memory 50, the memory 50 stores computer executable instructions that can be executed by the processor 51, and the processor 51 executes the computer executable instructions to implement the above method.

[0133] exist Figure 5 In the illustrated embodiment, the electronic device further includes a bus 52 and a communication interface 53 , wherein the processor 51 , the communication interface 53 and the memory 50 are connected via the bus 52 .

[0134] Among them, the memory 50 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 53 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used. The bus 52 can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 52 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0135] The processor 51 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor 51 or by software instructions. The above processor 51 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly implemented as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor 51 reads the information in the memory and completes the steps of the method of the above embodiment in combination with its hardware.

[0136] An embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by the processor, the computer-executable instructions prompt the processor to implement the above-mentioned method. The specific implementation can be found in the above-mentioned method embodiment, which will not be repeated here.

[0137] The computer program products of the methods, devices, and electronic devices provided in the embodiments of the present application include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. For specific implementation, please refer to the method embodiments and will not be repeated here.

[0138] Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0139] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0140] In the description of this application, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate the description of this application and simplify the description. They do not indicate or imply that the devices or components referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0141] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. These modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the scope of protection of the claims.

Claims

1. A video encoding method, characterized in that: The method comprises: determining a frame level of the video frame according to a reference relationship of the video frames in the picture group of the video stream; assigning an initial quantization parameter QP to the video frame according to a frame level of the video frame; Taking a video frame whose frame level is lower than a preset frame level threshold as a target video frame, obtaining an image quality of the target video frame, and adjusting an initial QP of the target video frame to an optimized QP according to the image quality; For video frames other than the target video frame in the group of pictures, encoding the other video frames according to the initial QPs of the other video frames; for the target video frame in the group of pictures, encoding the target video frame according to the optimized QP of the target video frame; Each frame level corresponds to a preset QP; the step of allocating an initial quantization parameter QP to the video frame according to the frame level of the video frame includes: determining the preset QP corresponding to the frame level as the initial QP corresponding to the video frame according to the frame level of the video frame; or determining the optimal QP corresponding to each frame level according to the preset QP corresponding to each frame level and the attribute information of the video frame contained in each frame level; determining the optimal QP corresponding to the frame level as the initial QP corresponding to the video frame according to the frame level of the video frame; wherein the attribute information includes: the number of header information bits of the video frame, the video frame coding complexity, and macroblock information.

2. The method according to claim 1, characterized in that The step of obtaining the image quality of the target video frame comprises: Calculating a mean square error value corresponding to the target video frame according to a first pixel value corresponding to the target video frame and a second pixel value of the target video frame; The first pixel value is an initial pixel value before encoding the target video frame, and the second pixel value is a pixel value encoded based on an initial QP corresponding to the target video frame; or the first pixel value is a pixel value encoded based on a preset QP corresponding to the target video frame, and the second pixel value is a pixel value encoded based on the initial QP corresponding to the target video frame; The image quality of the target video frame is determined according to the mean square error value.

3. The method according to claim 2, characterized in that The step of calculating a mean square error value corresponding to the target video frame according to a first pixel value corresponding to the target video frame and a second pixel value of the target video frame includes: The mean square error value corresponding to the target video frame is calculated according to the following formula: Where MSE represents the mean square error value corresponding to the target video frame; mn represents a video frame image of size m*n; I(i,j) and K(i,j) represent the first pixel value and the second pixel value of the target video frame at the pixel position (i,j), respectively.

4. The method according to claim 1, wherein The step of determining the optimal QP corresponding to each frame level according to the preset QP corresponding to each frame level and the attribute information of the video frame contained in each frame level includes: Let the preset QP corresponding to each frame level be Q l , the optimal QP is Q l ， , l represents the frame level, l=0,1,2…L-1; L represents the number of frame levels; According to the header information bit rate and video frame encoding complexity of each frame level, and the Q l and Q l ， , determine the sum of bit rate changes corresponding to each frame level; According to the macroblock information of the video frame corresponding to each frame level, the Q l and Q l ， , determine the sum of coding losses corresponding to each frame level; Find the Q value when the sum of the bit rate changes corresponding to each frame level is less than or equal to 0 and the sum of the coding losses corresponding to each frame level is the smallest l ， , and obtain the optimal QP corresponding to each frame level.

5. The method according to claim 4, characterized in that According to the header information bit rate and video frame encoding complexity of each frame level, and the Q l and Q l ， , the step of determining the sum of bit rate changes corresponding to each frame level includes: For each video frame at each frame level, the following operations are performed: According to the following formula, the video frame is calculated in Q l The first bit rate under the effect, and the video frame in Q l ， The second code rate under action: Where R represents the bit rate of the video frame under the effect of QP; H represents the number of header information bits of the video frame; S represents the encoding complexity of the video frame; a and b represent adjustment parameters respectively; Calculating a difference between the first bit rate and the second bit rate to obtain a bit rate change corresponding to the video frame; The bit rate changes of all video frames corresponding to each frame level are summed to obtain the bit rate change corresponding to each frame level; the bit rate changes corresponding to each frame level are summed to obtain the total bit rate change corresponding to each frame level.

6. The method according to claim 4, characterized in that According to the macroblock information of the video frame corresponding to each frame level, the Q l and Q l ， , the step of determining the sum of coding losses corresponding to each frame level includes: For each video frame at each frame level, the following operations are performed: According to the following formula, the video frame is calculated in Q l The first coding loss is affected, and the video frame is in Q l ， The second coding loss under the action of: Where D represents the coding loss of the video frame under the effect of QP; N represents the number of macroblocks contained in the video frame; A i Indicates the loss ratio of the i-th macroblock in the video frame; Q i Indicates the quantization parameter value corresponding to the i-th macroblock, which is based on A i The value after adjusting QP; Calculating a difference between the first coding loss and the second coding loss to obtain a coding loss corresponding to the video frame; Calculate the average value of the coding loss of all video frames corresponding to each frame level; The coding loss corresponding to each frame level is obtained according to the following formula: Where ΔD tot,l represents the coding loss corresponding to frame level l, ΔD l represents the average value of the coding loss of all video frames corresponding to frame level l, δ = 0.5, represents the reference coefficient between different frame levels; The coding losses corresponding to each frame level are summed to obtain the total coding losses corresponding to each frame level.

7. The method according to claim 4, characterized in that Find the Q value when the sum of the bit rate changes corresponding to each frame level is less than or equal to 0 and the sum of the coding losses corresponding to each frame level is the smallest l ， , the steps of obtaining the optimal QP corresponding to each frame level include: According to the following formula, the optimal QP corresponding to each frame level is solved by the Lagrange multiplier method: Where l represents the frame level, l = 0, 1, 2…L-1; ΔD tot,l represents the coding loss corresponding to frame level l, ΔR tot,l Indicates the bit rate change corresponding to frame level l; Q l ， It indicates the optimal QP corresponding to each frame level when the sum of the bit rate changes corresponding to each frame level is less than or equal to 0 and the coding loss corresponding to each frame level is minimized.

8. The method according to claim 1, characterized in that The step of adjusting the initial QP of the target video frame to an optimized QP according to the image quality includes: If the image quality exceeds a preset threshold, the initial QP is increased to obtain an optimized QP corresponding to the target video frame.

9. The method according to claim 8, characterized in that The step of increasing the initial QP to obtain the optimized QP corresponding to the target video frame includes: Determining a QP increase amplitude value corresponding to the target video frame according to the image quality of the target video frame and the preset threshold; The initial QP corresponding to the target video frame is added to the QP increase amplitude value to obtain an optimized QP corresponding to the target video frame.

10. The method according to claim 9, characterized in that The step of determining a QP increase amplitude value corresponding to the target video frame according to the image quality of the target video frame and the preset threshold comprises: The QP increase value corresponding to the target video frame is determined according to the following formula: Among them, ΔQ represents the QP increase amplitude value corresponding to the target video frame; IQA l Indicates the image quality corresponding to the target video frame; IQA yu Indicates the preset threshold corresponding to the image quality; in, Represents the average quantization step size.

11. A video encoding device, characterized in that: The device comprises: a frame level determination module, configured to determine the frame level of the video frame according to a reference relationship of the video frames in the picture group of the video stream; an initial QP allocation module, configured to allocate an initial quantization parameter QP to the video frame according to a frame level of the video frame; A QP adjustment module is configured to take a video frame whose frame level is lower than a preset frame level threshold as a target video frame, obtain an image quality of the target video frame, and adjust an initial QP of the target video frame to an optimized QP according to the image quality; an encoding module, configured to encode other video frames in the group of pictures except the target video frame according to the initial QP of the other video frames; and to encode the target video frame in the group of pictures according to the optimized QP of the target video frame; Each frame level corresponds to a preset QP; the initial QP allocation module is further used to determine the preset QP corresponding to the frame level as the initial QP corresponding to the video frame based on the frame level of the video frame; or determine the optimal QP corresponding to each frame level based on the preset QP corresponding to each frame level and the attribute information of the video frame contained in each frame level; based on the frame level of the video frame, determine the optimal QP corresponding to the frame level as the initial QP corresponding to the video frame; wherein the attribute information includes: the number of header information bits of the video frame, the video frame coding complexity, and macroblock information.

12. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 10.

Images