Video coding method and video coding device

Abstract

The embodiment of the invention provides a video coding method and device. The method comprises the following steps: for a to-be-coded unit in a current frame to be coded in a video sequence, determining a matched coding unit corresponding to the to-be-coded unit in coded units of a previous frame according to a motion vector of the to-be-coded unit; obtaining quantization parameter offsets of the matched coding units, and calculating an average value of the quantization parameter offsets of the matched coding units; and coding the to-be-coded unit by taking the average value as the quantization parameter offset of the to-be-coded unit. The invention provides a method for delayed use of delta QP, the delta QP of each frame in the n frames does not need to be calculated in advance, real-time coding can be realized, the code rate can be reduced, delay caused by pre-coding is reduced on the basis of improving the coding quality, and the method can be suitable for scenes such as live broadcast and the like with high requirements on real-time performance.

Description

Technical Field

[0001] The present invention relates to the technical field of video processing, and in particular, to a video encoding method and a video encoding apparatus. Background Art

[0002] Video encoding is a technology that compresses redundant components in video images and uses as little data as possible to represent video information.

[0003] Currently, video encoding usually adopts the cutree technology. The cutree technology analyzes a certain number (denoted as lookahead_num) of frames from back to front in advance, so as to adjust the QP (Quantizer Parameter) value of each frame. Among them, lookahead_num can be set as needed. For example, if lookahead_num is set to 8, it starts to deduce from the 8th frame after the current frame forward. According to the reference relationship between the 8th frame and the 7th frame, the transfer cost of the coding unit of the 7th frame to the 8th frame is deduced. According to the reference relationship between the 7th frame and the 6th frame, the transfer cost of the coding unit of the 6th frame to the 7th frame is deduced. And so on, until the cumulative transfer cost of the current frame to the subsequent 8 frames is deduced, so as to adjust the QP of the current frame coding unit. Thus, during the pre-analysis of these 8 frames, the QP value of the current frame can be adjusted according to the QP offset of the current frame deduced from the transfer cost of the current frame to the subsequent 8 frames.

[0004] It can be seen that the cutree technology needs to pre-analyze a certain number of frames, resulting in a delay in the encoding process. In scenarios with high real-time requirements such as live broadcasts, there is no delay in the encoding process. Therefore, the cutree technology cannot be used, resulting in a high bit rate for live broadcasts. Summary of the Invention

[0005] Embodiments of the present invention provide a video encoding method and a video encoding apparatus, which can reduce the delay caused by pre-encoding on the basis of improving the encoding quality, can achieve real-time encoding and reduce the bit rate, and are applicable to scenarios with high real-time requirements such as live broadcasts.

[0006] In a first aspect, embodiments of the present invention disclose a video encoding method, and the method includes:

[0007] For a coding unit to be coded in a current frame to be coded in a video sequence, determine a matching coding unit corresponding to the coding unit to be coded in the coded units of the previous frame according to the motion vector of the coding unit to be coded;

[0008] Obtain the quantization parameter offset of the matching coding unit, and calculate the average value of the quantization parameter offsets of the matching coding units;

[0009] Use the average value as the quantization parameter offset of the unit to be encoded, and encode the unit to be encoded.

[0010] In a second aspect, an embodiment of the present invention discloses a video encoding apparatus, the apparatus includes:

[0011] A matching module, configured to determine a matching encoded unit corresponding to the unit to be encoded in the previously encoded units of the previous frame according to the motion vector of the unit to be encoded in the current frame to be encoded in the video sequence;

[0012] An obtaining module, configured to obtain the quantization parameter offset of the matching encoded unit;

[0013] A calculating module, configured to calculate an average value of the quantization parameter offsets of the matching encoded units;

[0014] An encoding module, configured to use the average value as the quantization parameter offset of the unit to be encoded, and encode the unit to be encoded.

[0015] In a third aspect, an embodiment of the present invention discloses a machine-readable medium, on which instructions are stored, and when the instructions are executed by one or more processors of the apparatus, the apparatus is caused to execute the video encoding method as described in one or more of the foregoing.

[0016] The embodiments of the present invention include the following advantages:

[0017] For each video frame in the video sequence, the embodiments of the present invention can transfer the delta QP of the previously encoded units in the previous frame to the current frame through the relationship of the motion vector to calculate the delta QP of each unit to be encoded in the current frame. That is, the present invention proposes a method of delaying the use of delta QP, without the need to calculate the delta QP of each frame in n frames in advance, which can achieve real-time encoding and reduce the bit rate, reduce the delay caused by pre-encoding on the basis of improving the encoding quality, and can be applicable to scenarios with high requirements for real-time performance such as live broadcasts. Further, the embodiments of the present invention adjust the QP for the target frame in the video sequence, rather than adjusting the QP for each frame in the video sequence. For example, adjusting the QP every few frames is equivalent to simulating that the target frame and other frames belong to different reference layers, thereby realizing the idea of simulating hierarchical P frames. According to the reference relationship, frames in different layers have different importance, so as to solve the problem that adjusting the QP for each frame under real-time encoding conditions cannot save the bit rate, can save the bit rate, and can obtain higher-quality videos at the same bit rate. Description of the Drawings

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments of the present invention. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0019] Figure 1 is a flowchart of the steps of an embodiment of a video encoding method according to an embodiment of the present invention;

[0020] Figure 2 is a schematic diagram of a unit to be encoded and a matching encoding unit according to an embodiment of the present invention;

[0021] Figure 3 is a schematic structural diagram of an embodiment of a video encoding device according to an embodiment of the present invention. Detailed implementation manners

[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0023] The terms "first", "second", etc. in the specification and claims of the present invention are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are usually of the same type, and the number of objects is not limited. For example, the first object can be one or more. In addition, the term "and / or" in the specification and claims is used to describe the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after. The term "plurality" in the embodiments of the present invention refers to two or more, and other quantifiers are similar.

[0024] To enable those skilled in the art to better understand the present invention, the following concepts related to the present invention are described:

[0025] Coding Unit (CU): It is an image block with a pre-divided fixed size (such as 8×8) in a video frame and is also a processing object in the encoding process.

[0026] Quantization Parameter (QP): Quantization is the process of mapping the continuous values of a signal into multiple discrete amplitudes, achieving a many-to-one mapping of signal values. Quantization can effectively reduce the value range of the signal, thereby obtaining a better compression effect. The quantization parameter is a parameter that can derive the quantization step size during the encoding process, used to control the size and quality of the encoded bitstream. The smaller the quantization parameter, the larger the volume of the encoded bitstream and the higher the picture quality; the larger the quantization parameter, the smaller the volume of the encoded bitstream and the lower the picture quality.

[0027] CUTree: In the video encoding process, a method for adjusting the quantization parameter offset of the current encoding region based on the degree to which the current encoding region is referenced by the encoding regions of other frames.

[0028] Lookahead: When encoding the current frame in the CUTree algorithm, several subsequent frames of the current frame are quickly encoded in advance to obtain the propagation cost of the current frame to the subsequent frames.

[0029] Strength adjustment value: That is, CUTree Strength, a parameter used to control the adjustment amplitude of the quantization parameter offset of the current encoding unit. The quantization parameter offset is superimposed on the initial quantization parameter to obtain the adjusted quantization parameter.

[0030] Propagate Cost: Assume that the encoding unit b of the reference video frame references the encoding unit a of the referenced video frame. Then the propagate cost of the encoding unit a reflects the reference value provided by the encoding unit a to the encoding unit b.

[0031] Motion search: The process of finding the matching encoding unit (also called the reference block) with the smallest difference from the current encoding unit on the reference frame, used to identify the motion scene of the video frame.

[0032] A reference block refers to a pixel block from a previously encoded frame. The pixel blocks in the encoded frame are used to construct a reference image, where each pixel block is associated with a specific position. When encoding the current frame, the reference block is selected and matched with the corresponding position of the current frame. The selection and matching of the reference block are usually completed by a motion estimation algorithm. This algorithm determines the best-matching reference block by comparing the current frame with the pixel blocks in the reference image and predicts the pixel values at the corresponding positions of the current frame. Based on the difference between the predicted pixel values and the original pixel values, the error can be calculated and used for subsequent compression encoding.

[0033] Motion Vector (MV): A vector connecting the current encoding unit and the matching encoding unit, used to represent the direction of object motion in the video frame.

[0034] Pixel cost: It reflects the pixel-level difference between the current coding unit and the matching coding unit. The greater the pixel cost, the smaller the similarity between the current coding unit and the matching coding unit.

[0035] Intra prediction mode: A mode that only uses the reconstructed pixels of the current frame for prediction.

[0036] Inter prediction mode: A mode that uses the pixels of previously encoded frames for predicting the current frame.

[0037] Intra prediction cost: It reflects the total amount of information in the video frame in the intra prediction mode.

[0038] Inter prediction cost: It reflects the total amount of information in the video frame in the inter prediction mode. The difference between the intra prediction cost and the inter prediction cost represents the amount of information transferred from its reference frame to the current frame.

[0039] Refer to Figure 1 , which shows a flowchart of the steps of an embodiment of a video coding method of the present invention. The method may include the following steps:

[0040] Step 101: For the coding unit to be coded in the current frame to be coded in the video sequence, determine the matching coding unit corresponding to the coding unit to be coded in the coded units of the previous frame according to the motion vector of the coding unit to be coded;

[0041] Step 102: Obtain the quantization parameter offset of the matching coding unit, and calculate the average value of the quantization parameter offset of the matching coding unit;

[0042] Step 103: Use the average value as the quantization parameter offset of the coding unit to be coded, and code the coding unit to be coded.

[0043] The video coding method provided by the embodiments of the present invention can be applied to a terminal device. The terminal device has a video coding function, such as the terminal device is configured with an encoder. The embodiments of the present invention do not limit the specific form of the terminal device. Exemplarily, the terminal device may be a user equipment (UE), a mobile device, a user terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc.

[0044] It should be noted that the video coding method provided in the embodiments of the present invention can be applied to standards such as H.266 / VVC, H.265 / HEVC, H.264 / AVC, AVS (such as AVS3), or the next-generation video coding standard. The embodiments of the present invention do not limit this.

[0045] A video signal refers to an image sequence including multiple frames, that is, the video sequence in the embodiments of the present invention. A frame is a representation of the spatial information of the video signal. In the embodiments of the present invention, a frame refers to a frame image in the video sequence. A frame can be divided into several non-overlapping processing units, and each processing unit will perform similar compression operations. This processing unit is called a CTU (Coding Tree Unit) or an LCU (Large Coding Unit), and is uniformly referred to as an LCU in the embodiments of the present invention. The LCU can continue to be divided into finer granularity to obtain one or more basic coding units, called CUs (Coding Units).

[0046] Since the data bandwidth of the video signal is very high after digitization, it is difficult for computer devices to directly store and process it. Therefore, video compression technology needs to be used to reduce the data bandwidth of the video signal. Video compression technology is achieved through video coding. Due to different statistical characteristics, the corresponding video coding methods may also be different. It should be noted that the coding units described in the embodiments of the present invention all refer to CUs, and the coding referred to in the embodiments of the present invention refers to the coding of each CU.

[0047] The video coding method of the embodiments of the present invention can be applied to application scenarios that require video coding, such as short videos, video conferences, video live broadcasts, and network video on-demand. When a target video needs to be output in these application scenarios, the video sequence to be coded can be first coded according to the Figure 1 shown process to obtain a coded video, and then the coded video is encapsulated in a preset format to obtain a target video in the preset format for output.

[0048] For the current frame to be coded in the video sequence, each coding unit in the current frame can be coded according to the Figure 1 shown process.

[0049] In the embodiments of the present invention, according to the motion vector of the coding unit to be coded, the matching coding unit corresponding to the coding unit to be coded is determined in the coded unit of the previous frame; furthermore, the quantization parameter offset of the matching coding unit can be obtained, and the average value of the quantization parameter offsets of the matching coding unit is calculated; the average value is used as the quantization parameter offset of the coding unit to be coded, and the coding unit to be coded is coded.

[0050] Exemplarily, assume that for the coding unit CU to be coded in the current frame 1,1 , through the motion vector MV of CU 1,1 , the following four matching coding units are determined in the previous frame: CU 0,0 , CU 0,1 , CU 1,0 , CU 1,1 . Then, the delta QPs corresponding to CU 0,0 , CU 0,1 , CU 1,0 , CU 1,1 in the previous frame can be obtained respectively, and the average value is calculated as the delta QP of the coding unit CU 1,1 to be coded in the current frame.

[0051] For each video frame in the video sequence, the delta QP of the coded unit in the previous frame can be passed to the current frame through the relationship of the motion vector to calculate the delta QP of each coding unit to be coded in the current frame. That is to say, the present invention proposes a method of delaying the use of delta QP, which does not need to calculate the delta QP of each frame in n frames in advance, can achieve real-time coding, reduce the delay caused by pre-coding on the basis of improving the coding quality, and can be applied to scenarios with high real-time requirements such as live broadcast.

[0052] In a specific implementation, if the quantization parameter QP is adjusted for each frame in the video sequence, such as the average QP of each frame is reduced, it is equivalent to not performing QP adjustment, which will result in the inability to achieve the effect of bitrate saving. Therefore, the embodiments of the present invention only perform QP adjustment on the target frames in the video sequence. In an optional embodiment of the present invention, the current frame is the target frame, and the target frame may include the second frame in the video sequence and every other preset number of frames.

[0053] For the current frame to be coded in the video sequence, if it is determined that the current frame is the target frame, then the current frame is encoded according to the process shown in Figure 1 .

[0054] Exemplarily, the every other preset number may be 2 frames, that is, every other 2 frames is a target frame. For example, the video sequence to be coded includes the following video frames: I0, P1, P2, P3, P4, P5, P6, P7... Among them, the I0 frame is the first frame of the video sequence and also the first I frame, and no QP adjustment is performed. The second frame (P1 frame, that is, the first P frame) is used as the target frame for QP adjustment. Next, the P3 frame is 2 frames away from the I0 frame, so the P3 frame is the target frame and the P3 frame is adjusted for QP. And so on, the P6 frame, P9 frame, etc. are adjusted for QP.

[0055] It should be noted that the interval of two frames is only for illustrative purposes, and the number of frames of the target frame interval is not limited in the embodiments of the present invention. Since the first frame (such as the I0 frame) cannot perform QP adjustment, the first QP adjustment is delayed to the first P frame (such as the P1 frame). It can be understood that for subsequent I frames or P frames, if they are target frames, QP adjustment is also required.

[0056] In an alternative embodiment of the present invention, the method may further include:

[0057] For the current frame to be encoded in the video sequence, if the current frame is not a target frame, the encoding unit in the current frame is encoded using the preset quantization parameter.

[0058] In the embodiments of the present invention, if the current frame to be encoded is a target frame, as in the previous example, assuming the current frame is the P1 frame, since the P1 frame is a target frame, QP adjustment is performed on the P1 frame, and the P1 frame is encoded according to the Figure 1 shown process. If the current frame is the P2 frame, since the P2 frame is not a target frame, no QP adjustment is performed on the P2 frame, and the P2 frame is directly encoded using the preset quantization parameter of the P2 frame.

[0059] The preset quantization parameter may be a quantization parameter matching the current encoding rule. After selecting an encoding rule (encoder), the preset quantization parameter corresponding to the encoding rule is also selected. In the embodiments of the present invention, for non-target frames, the quantization parameter QP of the encoding unit = preset quantization parameter. For target frames, the quantization parameter QP of the encoding unit = preset quantization parameter + deltaQP.

[0060] It can be understood that the embodiments of the present invention do not limit the setting method of the preset quantization parameter. The preset quantization parameter may be a custom quantization parameter, or may be a quantization parameter calculated by a custom algorithm.

[0061] Since performing QP adjustment on each frame is equivalent to reducing the average QP of each frame, which is equivalent to having no gain in moving on the rate-distortion curve. By performing QP adjustment at intervals of a preset number of frames in the embodiments of the present invention, the quality of the target frame can be better, and at the same time, the subsequent frames of the target frame will also have their quality improved due to the existence of the reference relationship. At the same time, since the QP of the subsequent frames of the target frame does not change, the bit rate will not change much either, and the effect of saving the bit rate while improving the picture quality can be achieved.

[0062] For the current frame to be encoded, if the current frame is a target frame, then the current frame is processed according to the Figure 1Encode according to the shown process. In an embodiment of the present invention, when encoding to the target frame, the QP of the current target frame to be encoded can be adjusted according to the previously encoded previous frame, without pre-analyzing a certain number of frames, so the delay will not be increased.

[0063] In an alternative embodiment of the present invention, the method may further include:

[0064] Obtain the quantization parameter offset of each encoded unit in the previous frame of the current frame, and after determining the matching encoded unit corresponding to the unit to be encoded, obtain the quantization parameter offset of the matching encoded unit according to the obtained quantization parameter offset of each encoded unit.

[0065] Specifically, if the current frame is the target frame, first, the quantization parameter offset of each encoded unit in the previous frame of the current frame can be obtained.

[0066] Among them, the quantization parameter offset, that is, delta QP, is used to correct the preset quantization parameter. For example, if the preset quantization parameter of a certain encoded unit is QP1 and the quantization parameter offset of this encoded unit is Δ, then the preset quantization parameter of this encoded unit is corrected by using the quantization parameter offset of this encoded unit, and the final quantization parameter of this encoded unit is obtained as QP = QP1 + Δ.

[0067] In the foregoing example, assume that the current P1 frame is to be encoded. Since the P1 frame is the target frame and QP adjustment is required, first, the quantization parameter offset of each encoded unit in the previous frame (I0 frame) of the P1 frame is obtained.

[0068] In an alternative embodiment of the present invention, the obtaining of the quantization parameter offset of each encoded unit in the previous frame of the current frame may include:

[0069] Step S11: Calculate the transfer cost of each encoded unit in the previous frame to the current frame through the forward prediction algorithm;

[0070] Step S12: Calculate the quantization parameter offset of each encoded unit in the previous frame according to the transfer cost of each encoded unit in the previous frame.

[0071] The transfer cost of the encoded unit of the video frame to the subsequent frame can be obtained through the forward prediction (lookahead) algorithm. As in the foregoing example, when encoding the P1 frame, through the forward prediction (lookahead) algorithm, the transfer cost of each encoded unit in the previous frame (such as the I0 frame) of the current frame (such as the P1 frame) to the P1 frame can be calculated. Assume that the encoded unit b of the P1 frame references the encoded unit a of the I0 frame, then the transfer cost of the encoded unit a reflects the amount of information provided by the encoded unit a to the encoded unit b.

[0072] The lookahead algorithm calculates two parameters for each coding unit: the propagation cost and the propagation fraction. The propagation cost represents the amount of information contributed by the current coding unit to future frames, and the propagation fraction represents how much of the information of the current coding unit comes from the reference frame. Assuming the number of lookahead frames is n, then starting from the current frame and counting n frames backward, in the reverse order of the coding sequence, for all frames from the nth frame to the current frame, the following steps are repeated for each coding unit of each frame in turn:

[0073] (1) For the current frame, read the intra prediction cost intra cost, the inter prediction cost inter cost, and the propagation cost propagate cost of the current coding unit. The propagate cost of the last frame of forward prediction is 0.

[0074] (2) Calculate the propagate fraction of the current coding unit according to the inter cost and intra cost of the current coding unit. Assuming that the inter cost of a coding unit is only 80% of the intra cost, it means that 20% of the information of this coding unit comes from the reference frame, then the propagate fraction of this coding unit is 0.2. Exemplarily, the propagate fraction of a coding unit in a certain frame can be calculated by the following formula:

[0075] propagate fraction = (intra cost - inter cost) / intra cost (1)

[0076] where intra cost is the intra prediction cost of this coding unit, and inter cost is the inter prediction cost of this coding unit.

[0077] (3) Calculate the sum of all information related to the current coding unit: intra cost + propagate cost, that is, the sum of its own information and the information provided to subsequent frames. Multiply this sum by the propagate fraction of the current coding unit to obtain the amount of information propagated from the reference frame by the current coding unit, propagate amount.

[0078] (4) Divide the amount of information propagated, i.e., the propagate amount, of the current coding unit inherited from the reference frame among the matching coding units in the reference frame. Since the compensated area obtained by motion search of the current coding unit in the reference frame may cover multiple coding units (referred to as matching coding units), that is, multiple matching coding units in the reference frame are involved in the motion compensation of the current coding unit, the propagate amount can be allocated according to the ratio of the number of pixels participated in compensation by each matching coding unit in the reference frame to the total number of pixels of the entire coding unit. The size of the part of the propagate amount finally allocated to a certain matching coding unit in the reference frame is the propagate cost of that matching coding unit.

[0079] (5) Calculate forward from the last frame of forward prediction until the current frame, and the propagate cost of each coding unit in the current frame for the subsequent n frames can be obtained. Finally, according to the propagate cost of each coding unit in the current frame, the quantization parameter offset delta QP of each coding unit in the current frame can be calculated. Exemplarily, the delta QP of a certain coding unit can be calculated using the following formula:

[0080]

[0081] where the strength coefficient strength can be a preset constant, and the strength coefficient can be used to control the size of the quantization parameter of the coding unit in the video frame. For coding units that are not referenced, the propagate cost = 0 and the delta QP = 0.

[0082] After calculating the transfer cost of each encoded unit in the previous frame for the current frame through the forward prediction algorithm, the quantization parameter offset of each encoded unit in the previous frame can be calculated according to the transfer cost of each encoded unit in the previous frame.

[0083] In the foregoing example, when encoding the P1 frame, through the above lookahead algorithm, the delta QP of each encoded unit in the I0 frame can be calculated. For each coding unit to be encoded in the P1 frame, according to the motion vector of the coding unit to be encoded, all matching coding units corresponding to the coding unit to be encoded can be determined among the encoded units in the previous frame (I0 frame), and the average value of the delta QP of all the matching coding units can be calculated to obtain the delta QP of the coding unit to be encoded.

[0084] Exemplarily, referring to Figure 2 , a schematic diagram of a coding unit to be encoded and matching coding units in an embodiment of the present invention is shown. As Figure 2 shown, for the coding unit CU to be encoded in the P1 frame 11, through the motion vector MV of CU 11 in the I0 frame, the following matching coding units are determined: CU 0,0 , CU 0,1 , CU 1,0 , CU 1,1 . For CU in the I0 frame 0,0 , CU 0,1 , CU 1,0 , CU 1,1 , by calculating the weighted average of the corresponding delta QP respectively, the delta QP of the coding unit CU to be coded in the P1 frame can be obtained 1,1 .

[0085] It should be noted that the reference frame refers to the frame required for IPB coding. The relationship between the reference frame and the IPB frame is as follows: I frame, referring to the block coding within this image, does not require a reference frame. P frame: Refer to the previous I frame or P frame for coding, and the maximum number of reference frames is 2, all of which are forward. B frame: Refer to the previous and subsequent I frames or P frames for coding, one frame each in the front and back, or only forward or backward (any of the three options), and the number of reference frames may be 2 (both forward and backward) or 1 (forward or backward).

[0086] In the above example, when coding the P1 frame, the I0 frame is the reference frame of the P1 frame.

[0087] When coding the P2 frame, since the P2 frame is not the target frame, QP adjustment does not need to be performed. Therefore, the P2 frame can use the preset quantization parameter offset.

[0088] When coding the P3 frame, since the P3 frame is the target frame, through the above lookahead algorithm, the delta QP of each coded unit in the previous frame (P2 frame) can be calculated. For each coding unit to be coded in the P3 frame, according to the motion vector of the coding unit to be coded, all the matching coding units corresponding to the coding unit to be coded can be determined in the coded units in the previous frame (P2 frame), and the weighted average of the delta QP of all the matching coding units can be calculated to obtain the delta QP of the coding unit to be coded.

[0089] In specific implementation, a video frame can be divided into multiple CUs for encoding. Some CUs are more important and will be referenced by subsequent frames, while some CUs are not very important and will not be referenced by subsequent frames. If the quality of the reference blocks (CUs referenced by subsequent frames) can be improved, then when encoding the CUs of subsequent frames, the encoding bits will also be reduced due to the higher quality of the reference blocks. And appropriately reducing the encoding quality of the CUs not referenced by subsequent frames will not have a great impact on the overall bit rate. Therefore, for reference blocks, the encoding quality can be improved by reducing the QP value of their encoding, and through the relationship of motion vectors, the better encoding quality can be passed on through the transfer relationship. For the blocks not referenced, the QP value can be increased to appropriately reduce their encoding quality.

[0090] For each video frame in the video sequence, the delta QP of the encoded units in the previous frame can be passed to the current frame through the relationship of motion vectors to calculate the delta QP of each unit to be encoded in the current frame. That is, the present invention proposes a method of delaying the use of delta QP, which does not require pre-calculating the delta QP of each frame in n frames in advance, can achieve real-time encoding, reduce the delay caused by pre-encoding on the basis of improving the encoding quality, and can be applied to scenarios with high real-time requirements such as live broadcasts.

[0091] It should be noted that in specific implementation, in step 101, the quantization parameter offset of each encoded unit in the previous frame may not be obtained first, but the quantization parameter offset of the required encoded unit can be obtained in real time in subsequent steps. For example, for the unit to be encoded in the current frame, first, according to the motion vector of the unit to be encoded, the matching encoded unit corresponding to the unit to be encoded can be determined in the encoded units of the previous frame, and then the quantization parameter offset of each matching encoded unit in the previous frame can be obtained, and then the average value is calculated to obtain the quantization parameter offset of the unit to be encoded.

[0092] In an alternative embodiment of the present invention, the obtaining of the quantization parameter offset of the matching encoded unit may include: calculating the transfer cost of the matching encoded unit to the current frame through a forward prediction algorithm; calculating the quantization parameter offset of the matching encoded unit according to the transfer cost of the matching encoded unit.

[0093] Since the quantization parameter offset needs to be calculated for each coding unit in the current frame, each coding unit in the current frame needs to determine the matching coding unit in the previous frame and then obtain the quantization parameter offset of each matching coding unit. To avoid the operation of repeatedly obtaining the quantization parameter offset of the matching coding unit, the embodiment of the present invention can first obtain the quantization parameter offset of each coded unit in the previous frame in step 101 and directly read it in the subsequent steps to reduce the query operation each time. Of course, in specific implementation, the order of specific operations can be set as needed.

[0094] In an alternative embodiment of the present invention, calculating the average value of the quantization parameter offsets of the matching coding units in step 102 may include:

[0095] Step S21: Determine the weight of each matching coding unit based on the overlapping area ratio between each matching coding unit and the coding unit to be coded;

[0096] Step S22: Calculate the average value after multiplying the quantization parameter offset of each matching coding unit by the weight of each matching coding unit.

[0097] For the coding unit to be coded in the current frame, there may be multiple matching coding units in the reference frame, and the amount of information contributed by each matching coding unit to the coding unit to be coded is different. Therefore, the embodiment of the present invention determines the weight of each matching coding unit based on the overlapping area ratio between each matching coding unit of the coding unit to be coded and the coding unit to be coded, multiplies the quantization parameter offset of each matching coding unit by the weight of each matching coding unit, and then calculates the average value to obtain the delta QP of the coding unit to be coded, making the calculated delta QP of each coding unit to be coded more accurate.

[0098] Assume that the coding unit CU 1,1 to be coded in the current frame (such as P1 frame) has coordinates (x, y), and the MV corresponding to the coding unit CU 1,1 is (mx, my), then the coordinates of the matching coding unit corresponding to the coding unit CU 1,1 in the reference frame (such as I0 frame) in the reference frame are (x + mx, y + my).

[0099] In the embodiment of the present invention, the relationship between the coding unit to be coded and the corresponding matching coding unit in the reference frame may include the following four cases:

[0100] Case 1: The coding unit to be coded has 4 corresponding matching coding units in the reference frame. As Figure 2 shown, the coding unit CU 1,1 to be coded in the P1 frame and the CU 0,0 in the reference frame (I0 frame), CU0,1 , CU 1,0 , CU 1,1 These 4 matching coding units overlap, and the overlapping parts are denoted as S1, S2, S3, and S4. In this case, assuming that the size of each coding unit is 16 and the upper left corner of the video frame is the coordinate origin (0, 0), then in the P1 frame, the coordinate of the coding unit CU 1,1 is (16, 16). Assuming that the MV of the coding unit CU 1,1 is (-10, -6), then the position of the coding unit CU 1,1 in the I0 frame is (16 - 10, 16 - 6) = (6, 10). Therefore, the overlapping area ratio between the matching coding unit CU 0,0 in the I0 frame and the coding unit CU 1,1 in the P1 frame is 10 * 6 / (16 * 16); the overlapping area ratio between the matching coding unit CU 0,1 in the I0 frame and the coding unit CU 1,1 in the P1 frame is 6 * 6 / (16 * 16); the overlapping area ratio between the matching coding unit CU 1,0 in the I0 frame and the coding unit CU 1,1 in the P1 frame is 10 * 10 / (16 * 16); the overlapping area ratio between the matching coding unit CU 1,1 in the I0 frame and the coding unit CU 1,1 in the P1 frame is 6 * 10 / (16 * 16).

[0101] Case 2: There is 1 matching coding unit corresponding to the coding unit to be coded in the reference frame, that is, the coding unit to be coded overlaps with one matching coding unit. Exemplarily, assuming that for the coding unit CU 1,1 in the P1 frame, its MV is (0, -16), then the matching coding unit is determined as CU 1,1 in the I0 frame according to the MV of the coding unit CU 0,1 . In this case, the delta QP of this matching coding unit can be directly used as the delta QP of the coding unit to be coded.

[0102] Case 3: There are 2 matching coding units on the left and right corresponding to the coding unit to be coded in the reference frame, that is, the coding unit to be coded overlaps with two matching coding units. Exemplarily, assuming that for the coding unit CU 1,1 in the P1 frame, its MV is (-5, -16), then the matching coding units determined in the I0 frame according to the MV of the coding unit CU 1,1 include CU 0,0 and CU 0,1 . In this case, the calculation method of Case 1 can be referred to calculate the overlapping area ratio between each matching coding unit and the coding unit to be coded.

[0103] Case 4: There are two upper and lower matching coding units corresponding to the coding unit to be coded in the reference frame, that is, the coding unit to be coded overlaps with the two matching coding units. Exemplarily, for the coding unit CU in P1 frame 1,1 , the matching coding units determined according to its MV in I0 frame include CU 0,1 and CU 1,1 . In this case, the overlapping area ratio between each matching coding unit and the coding unit to be coded can be calculated by referring to the calculation method in Case 1.

[0104] In the embodiments of the present invention, the overlapping area ratio between each matching coding unit and the coding unit to be coded can be used as the weight of each matching coding unit. For a certain coding unit to be coded, the average value is obtained by multiplying the delta QP of each matching coding unit corresponding to the coding unit to be coded by the weight of each matching coding unit and then calculating the average value, and the delta QP of the coding unit to be coded is obtained.

[0105] Taking Case 1 as an example, for the coding unit CU in the current frame (P1 frame) 1,1 , assuming that the matching coding units determined according to the MV of the coding unit CU 1,1 in I0 frame include CU 0,0 , CU 0,1 , CU 1,0 and CU 1,1 . Since the delta QP of each coded unit in I0 frame has been obtained, therefore, it can be known that the delta QP corresponding to the matching coding units CU 0,0 , CU 0,1 , CU 1,0 and CU 1,1 respectively. If the delta QP of the matching coding unit CU 0,0 is denoted as Δ1, the delta QP of the matching coding unit CU 0,1 is denoted as Δ2, the delta QP of the matching coding unit CU 1,0 is denoted as Δ3, and the delta QP of the matching coding unit CU 1,1 is denoted as Δ4. The overlapping area ratio between the matching coding unit CU 0,0 and the coding unit to be coded is denoted as s1, the overlapping area ratio between the matching coding unit CU 0,1 and the coding unit to be coded is denoted as s2, the overlapping area ratio between the matching coding unit CU 1,0 and the coding unit to be coded is denoted as s3, and the overlapping area ratio between the matching coding unit CU 1,1 and the coding unit to be coded is denoted as s4. The delta QP of the coding unit CU 1,1 in P1 frame can be calculated as follows: (s1*Δ1 + s2*Δ2 + s3*Δ3 + s4*Δ4) / 4.

[0106] In summary, for each video frame in a video sequence, embodiments of the present invention can transmit the delta QP of the encoded units in the previous frame to the current frame through the relationship of the motion vectors to calculate the delta QP of each unit to be encoded in the current frame. That is, the present invention proposes a method of delaying the use of delta QP, which can realize real-time encoding and reduce the bit rate without calculating the delta QP of each frame in n frames in advance. On the basis of improving the encoding quality, the delay caused by pre-encoding can be reduced, and it can be applied to scenarios with high real-time requirements such as live broadcasts. Further, embodiments of the present invention adjust the QP for target frames in the video sequence, rather than adjusting the QP for each frame in the video sequence. For example, adjusting the QP every few frames is equivalent to simulating that the target frame and other frames belong to different reference layers, thereby realizing the idea of simulating hierarchical P frames. According to the reference relationship, frames in different layers have different importance, so that the problem that QP adjustment for each frame cannot save the bit rate under real-time encoding conditions can be solved, the bit rate can be saved, and higher-quality video can be obtained at the same bit rate.

[0107] It should be noted that for method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that embodiments of the present invention are not limited by the described action sequence, because according to embodiments of the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential for embodiments of the present invention.

[0108] Referring to Figure 3 , a structural block diagram of an embodiment of a video coding apparatus according to the present invention is shown. The apparatus includes:

[0109] A matching module 301, configured to determine a matching encoded unit corresponding to a unit to be encoded in a current frame to be encoded in a video sequence according to the motion vector of the unit to be encoded in the previous frame's encoded units.

[0110] An obtaining module 302, configured to obtain the quantization parameter offset of the matching encoded unit.

[0111] A calculating module 303, configured to calculate an average value of the quantization parameter offset of the matching encoded unit.

[0112] An encoding module 304, configured to use the average value as the quantization parameter offset of the unit to be encoded and encode the unit to be encoded.

[0113] Optionally, the calculating module includes:

[0114] A weight calculation sub-module, configured to determine the weight of each matching coding unit based on the overlapping area ratio between each matching coding unit and the coding unit to be coded;

[0115] An average value calculation sub-module, configured to calculate the average value after multiplying the quantization parameter offset of each matching coding unit by the weight of each matching coding unit.

[0116] Optionally, the obtaining module includes:

[0117] A cost calculation sub-module, configured to calculate the transmission cost of the matching coding unit for the current frame through a forward prediction algorithm;

[0118] An offset calculation sub-module, configured to calculate the quantization parameter offset of the matching coding unit according to the transmission cost of the matching coding unit.

[0119] Optionally, the current frame is a target frame, and the target frame includes the second frame in the video sequence and every preset number of frames.

[0120] Optionally, the coding module is further configured to, if the current frame is not a target frame, use a preset quantization parameter to code the coding unit to be coded in the current frame.

[0121] Optionally, the obtaining module is specifically configured to obtain the quantization parameter offset of each coded unit in the previous frame of the current frame, and after determining the matching coding unit corresponding to the coding unit to be coded, obtain the quantization parameter offset of the matching coding unit according to the quantization parameter offset of each coded unit that has been obtained.

[0122] The video encoding device according to the embodiments of the present invention can be applied to an encoder. For each video frame in a video sequence, the delta QP of the encoded units in the previous frame can be passed to the current frame through the relationship of the motion vectors to calculate the delta QP of each unit to be encoded in the current frame. That is, the video encoding device of the present invention delays the use of delta QP and does not need to calculate the delta QP of each frame in n frames in advance, which can achieve real-time encoding and reduce the bit rate, reduce the delay caused by pre-encoding on the basis of improving the encoding quality, and can be applied to scenarios with high real-time requirements such as live broadcasts. Further, the embodiments of the present invention adjust the QP for the target frames in the video sequence instead of adjusting the QP for each frame in the video sequence. For example, the QP is adjusted every few frames, which is equivalent to simulating that the target frame and other frames belong to different reference layers. Thus, the idea of simulating hierarchical P frames can be realized. According to the reference relationship, frames in different layers have different importance, thereby solving the problem that QP adjustment for each frame cannot save the bit rate under real-time encoding conditions, saving the bit rate, and obtaining higher-quality videos at the same bit rate.

[0123] For the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For the relevant parts, please refer to the partial description of the method embodiments.

[0124] The embodiments in this specification are all described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other.

[0125] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0126] The embodiments of the present invention also provide a non-transitory computer-readable storage medium. When the instructions in the storage medium are executed by a processor of a device (server or terminal), the device can execute the description of the video encoding method in the corresponding embodiments mentioned above. Therefore, it will not be elaborated here. In addition, the description of the beneficial effects of using the same method will not be elaborated either. For the technical details not disclosed in the computer program product or computer program embodiments involved in the present invention, please refer to the description of the method embodiments of the present invention. Figure 1

[0127] ​Other embodiments of the present invention will be readily apparent to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include known common knowledge or conventional technical means in the technical field not disclosed by the present invention. The specification and examples are only to be considered as exemplary, and the true scope and spirit of the present invention are pointed out by the following claims.

[0128] It should be understood that the present invention is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

[0129] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

[0130] The above has introduced in detail a video encoding method and a video encoding device provided by the present invention. Specific examples are used herein to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A video encoding method, characterized in that, The method includes: For a coding unit to be coded in a current frame to be coded in a video sequence, determining a matching coding unit corresponding to the coding unit to be coded in the coded units of the previous frame according to the motion vector of the coding unit to be coded; Obtaining the quantization parameter offset of the matching coding unit, and calculating an average value of the quantization parameter offsets of the matching coding units; Using the average value as the quantization parameter offset of the coding unit to be coded, and coding the coding unit to be coded.

2. The method according to claim 1, wherein The calculating the average value of the quantization parameter offsets of the matching coding units includes: Determining the weight of each matching coding unit based on the overlapping area ratio between each matching coding unit and the coding unit to be coded; Calculating the average value after multiplying the quantization parameter offset of each matching coding unit by the weight of each matching coding unit.

3. The method according to claim 1, wherein The obtaining the quantization parameter offset of the matching coding unit includes: Calculating the transmission cost of the matching coding unit for the current frame through a forward prediction algorithm; Calculating the quantization parameter offset of the matching coding unit according to the transmission cost of the matching coding unit.

4. The method according to claim 1, wherein The current frame is a target frame, and the target frame includes the second frame in the video sequence and every preset number of frames.

5. The method according to claim 1, wherein The method further includes: If the current frame is not a target frame, using a preset quantization parameter to code the coding unit to be coded in the current frame.

6. The method according to claim 1, wherein The method further includes: Obtaining the quantization parameter offset of each coded unit in the previous frame of the current frame, and after determining the matching coding unit corresponding to the coding unit to be coded, obtaining the quantization parameter offset of the matching coding unit according to the obtained quantization parameter offsets of each coded unit.

7. A video encoding device, characterized in that, The apparatus includes: A matching module, configured to determine, for a coding unit to be coded in a current frame to be coded in a video sequence, a matching coding unit corresponding to the coding unit to be coded in the coded units of the previous frame according to the motion vector of the coding unit to be coded; An obtaining module, configured to obtain the quantization parameter offset of the matching coding unit; A calculating module, configured to calculate an average value of the quantization parameter offsets of the matching coding units; A coding module, configured to use the average value as the quantization parameter offset of the coding unit to be coded, and code the coding unit to be coded.

8. The device according to claim 7, characterized in that, The calculating module includes: A weight calculating sub-module, configured to determine the weight of each matching coding unit based on the overlapping area ratio between each matching coding unit and the coding unit to be coded; An average value calculating sub-module, configured to calculate the average value after multiplying the quantization parameter offset of each matching coding unit by the weight of each matching coding unit.

9. The device according to claim 7, characterized in that, The obtaining module includes: A cost calculating sub-module, configured to calculate the transmission cost of the matching coding unit for the current frame through a forward prediction algorithm; An offset calculating sub-module, configured to calculate the quantization parameter offset of the matching coding unit according to the transmission cost of the matching coding unit.

10. The device according to claim 7, characterized in that, The current frame is a target frame, and the target frame includes the second frame in the video sequence and every preset number of frames.

Images