A hardware acceleration method for motion compensation calculation in H.265 format merge mode

Abstract

This invention provides a hardware acceleration method for motion compensation calculation in H.265 format merge mode. The method includes determining whether the current mmvx and nmv are the same, where the motion vector mv obtained after merge encoding is denoted as mmv, mmvx is the x-th mmv, and the motion vector mv obtained after motion estimation me is denoted as nmv. If the current mmvx and nmv values ​​are the same, MC calculation is not performed; instead, the pred pixel result of nmv is copied as the pred pixel of the current mmvx, thus omitting the MC calculation for the current mmvx. If the current mmvx and nmv values ​​are different, the reference frame pixels of mmvx need to be read, and MC calculation is performed to obtain the pred pixel. This method utilizes the correlation between MVs to reduce the MC calculation of MVs in merge mode without affecting encoding quality. This reduces both computational load and the number of DDR accesses, thereby achieving hardware acceleration.

Description

Technical Field

[0001] This invention relates to the field of hardware acceleration technology, and in particular to a hardware acceleration method for motion compensation computation in H.265 format merge mode. Background Technology

[0002] With the widespread application of high-definition and ultra-high-definition video (resolutions up to 4Kx2K and 8Kx4K), existing video coding standards can no longer meet the current needs of video applications, and the next-generation video codec standard H.265 is gradually being adopted. The H.265 format still uses a hybrid coding framework, including modules such as transform, quantization, entropy coding, intra-frame prediction, inter-frame prediction, and loop filtering. However, the H.265 format introduces new coding technologies at each stage, including quadtree-based block partitioning, support for more intra-frame prediction modes, more advanced inter-frame prediction technologies (merge, AMVP), and deblocking filtering and adaptive sample compensation in the post-processing stage. Inter-frame prediction is a key coding technology in H.265. This module uses the already encoded images of adjacent frames as reference images for the current frame. By acquiring the motion information of moving blocks, it predicts the block to be encoded, removing a large amount of temporal redundancy. Compared to other stages, inter-frame prediction achieves higher compression efficiency. In the H.265 standard, inter-frame prediction uses NME mode and merge mode. NME mode predicts MV through motion estimation process, while merge mode can predict MV using adjacent coded PU blocks.

[0003] In short, during H.265 encoding, the mv (predicted value) needs to be processed by mc (predicted value) to obtain the pred (predicted value) pixel. There are two types of mv for inter-frame prediction: one is the nmv obtained through me (predicted value), and the other is the mmv obtained through merge technology. Performing mc operations on the mv yields the corresponding pred value. The number of mmv (predicted values), maxNumMergeCand, is configurable, ranging from 1 to 5.

[0004] The basic process of H.265 inter-frame prediction NME mode is as follows:

[0005] 1. Obtain nmv through motion estimation;

[0006] 2. Perform mc on nmv to obtain pred pixels.

[0007] The basic process of H.265 inter-frame prediction merge mode is as follows:

[0008] 1. Through the merge technology, multiple candidate mmv are obtained from adjacent PUs. The number of mmv, maxNumMergeCand, can be configured, ranging from 1 to 5.

[0009] 2. Perform mc on each mmv to obtain the pred pixel.

[0010] The process of performing MC calculations on 1 nmV and 5 mmV in hardware design, such as... Figure 1 As shown, the detailed process is as follows:

[0011] 1. Read the nmv, read the ref data from the ddr, perform motion compensation, and store the motion compensation result of the nmv into the RAM;

[0012] 2. Read mmv0, read the ref data from ddr for motion compensation, and store the motion compensation result of mmv0 into RAM;

[0013] 3. Read mmv1, read the ref data from ddr for motion compensation, and store the motion compensation result of mmv1 into RAM;

[0014] 4. Read mmv2, read ref data from ddr for motion compensation, and store the motion compensation result of mmv2 into RAM;

[0015] 5. Read mmv3, read ref data from ddr for motion compensation, and store the motion compensation results of mmv3 into RAM;

[0016] 6. Read mmv4, read ref data from ddr for motion compensation, and store the motion compensation results of mmv4 into RAM;

[0017] The pseudocode for the hardware's MC calculation process for 1 nmV and 5 mmV is as follows: Figure 2 As shown.

[0018] However, the shortcomings of existing technologies include:

[0019] Performing a merge operation on a vehicle (MV) requires reading its corresponding reference data from the DDR (Data Source Depository). Reading the reference data impacts the DDR bandwidth. The more times the DDR reference data is accessed, the greater the impact on the DDR bandwidth. The merge technique can have a maximum of 5 MV candidates, while the mere merge (ME) can only have 1 MV. Therefore, a single processing unit (PU) needs to perform merge operations on a maximum of 6 MVs, which impacts the DDR bandwidth and requires sufficient computational resources for the merge operations.

[0020] The following are terms used in the prior art, including:

[0021] H.265: A video encoding format;

[0022] mv: abbreviation for motion vector; the offset in a reference frame relative to the current frame;

[0023] Me: Motion estimation; the motion of an object between two frames is translational motion, and the displacement is not very large, so motion vectors are allocated in blocks. Motion estimation uses a large number of reference frame predictions to improve accuracy. The current block to be encoded can find the optimal matching block among all reference frames in the buffer for motion compensation, thus effectively removing temporal redundancy. A motion vector MV is found for each block, and motion compensation predictive coding is performed. Each segmented region has its corresponding motion vector, and the motion vector and block selection method are encoded and transmitted.

[0024] MC: an abbreviation for Motion Compensation; Motion Compensation predicts and compensates for the current local image by referencing the local image in the reference frame. It is an effective method to reduce redundant information in the frame sequence. Motion Compensation is a method to describe the differences between adjacent frames (adjacent here means adjacent in terms of coding relationship, but not necessarily adjacent in terms of playback order). Specifically, it describes how each best-matching block in the previous frame (adjacent here means preceding in terms of coding relationship, but not necessarily preceding in terms of playback order) moves to the position of the block to be encoded in the current frame.

[0025] Merge can be seen as an encoding mode in which the current PU's mv is directly predicted from the spatially or temporally neighboring PUs.

[0026] pred pixel: The pixel obtained by performing mc on the pixel of the reference frame;

[0027] nme: Nme can be seen as a coding mode in which the current PU's mv is predicted by motion estimation;

[0028] AMVP: an abbreviation for Advanced Motion Vector Predictor;

[0029] PU: Abbreviation for prediction unit. Summary of the Invention

[0030] To address the problems in the existing technology, the purpose of this application is to reduce the computational complexity (MC) of MV in merge mode by leveraging the correlation between MV and MV without affecting encoding quality. This reduces both the computational load and the number of DDR accesses, thereby achieving hardware acceleration.

[0031] Specifically, this invention provides a hardware acceleration method for motion compensation calculation in H.265 format merge mode. The method includes determining whether the current mmvx and nmv are the same, wherein the motion vector mv obtained after merge encoding mode is denoted as mmv, the mmvx is the xth mmv, and the motion vector mv obtained after motion estimation me is denoted as nmv; if the current mmvx and nmv values ​​are the same, then MC calculation is not performed, but the pred pixel result obtained by MC calculation of nmv is copied as the pred pixel of the current mmvx, thereby omitting the MC calculation of the current mmvx; if the current mmvx and nmv values ​​are different, then the reference frame pixels of mmvx need to be read, and MC calculation is performed to obtain the pred pixel.

[0032] The method further includes storing the motion compensation results in a random access memory (RAM).

[0033] In the method described, the pipelined motion estimation (me) of the hardware structure precedes the merge mode. Here, pipeline refers to the number of pipeline stages, with earlier stages producing results first. In other words, the motion vector mv of me is obtained before the merge mv. At the same time, the mv of the merge mode and the mv of the motion estimation me are correlated in space and time. Therefore, the mv of the merge mode may be equal to the mv of me. Thus, if the mv is equal, their pred pixel values ​​are also the same.

[0034] The method utilizes the aforementioned characteristics; after obtaining the candidate mv list in the Merge mode of H265 format, it is necessary to perform mc operation for each mmvx.

[0035] The MV list is .

[0036] The motion compensation mc operation is to calculate the position of the reference block in the reference frame using the position of the current block and the motion vector nmv, and use the pixel of the reference block as the predicted value of the pixel of the current block.

[0037] The method further includes the following steps:

[0038] S1. Read the reference frame pixels of nmv, perform motion compensation MC operation on nmv, and save the pred pixels of nmv to random access memory ram;

[0039] S2. Determine if the current mmvx and nmv are the same:

[0040] S2.1, denot the first mmv as mmv0 and compare it with nmv.

[0041] When the two values ​​are equal, the mc operation is no longer performed on mmv0. Instead, the pred pixel of nmv is directly copied as the pred pixel of mmv0, and step S3 is performed.

[0042] When the two values ​​are not equal, it is necessary to read the reference frame pixel of mmv0, perform mc operation on mmv0 to obtain the pred pixel, and proceed to step S3.

[0043] S2.2, denoted as mmv1, the second mmv is compared with nmv.

[0044] When the two values ​​are equal, the mc operation is no longer performed on mmv1. Instead, the pred pixel of nmv is directly copied as the pred pixel of mmv1, and step S3 is performed.

[0045] When the two values ​​are not equal, it is necessary to read the reference frame pixel of mmv1, perform mc operation on mmv1 to obtain the pred pixel, and proceed to step S3.

[0046] S2.3, denot the third mmv as mmv2 and compare it with nmv.

[0047] When the two values ​​are equal, the mc operation is no longer performed on mmv2. Instead, the pred pixel of nmv is directly copied as the pred pixel of mmv2, and step S3 is performed.

[0048] When the two values ​​are not equal, it is necessary to read the reference frame pixel of mmv2, perform mc operation on mmv2 to obtain the pred pixel, and proceed to step S3.

[0049] S2.4, denot the fourth mmv as mmv3 and compare it with nmv.

[0050] When the two values ​​are equal, the mc operation is no longer performed on mmv3. Instead, the pred pixel of nmv is directly copied as the pred pixel of mmv3, and step S3 is performed.

[0051] When the two values ​​are not equal, it is necessary to read the reference frame pixel of mmv3, perform mc operation on mmv3 to obtain the pred pixel, and proceed to step S3.

[0052] S2.5, denot the 5th mmv as mmv4 and compare it with nmv.

[0053] When the two values ​​are equal, the mc operation is no longer performed on mmv4, and the pred pixel of nmv is directly used as the pred pixel of mmv4, and step S3 is performed.

[0054] When the two values ​​are not equal, it is necessary to read the reference frame pixel of mmv4, perform mc operation on mmv4 to obtain the pred pixel, and proceed to step S3.

[0055] S3. Save the pred pixel to RAM.

[0056] Therefore, the advantage of this application is that when mmv and nmv are equal, the mc operation corresponding to mmv can be omitted. This proposed hardware structure can accelerate hardware operations and reduce software computation without affecting encoding quality. Attached Figure Description

[0057] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0058] Figure 1 This is a schematic diagram of the process of performing MC calculations on 1 nmv and 5 mmv in the hardware design of existing technology.

[0059] Figure 2 yes Figure 1 A pseudocode diagram illustrating the computational process.

[0060] Figure 3 This application is aimed at A flowchart illustrating the mc operation process.

[0061] Figure 4 This is a pseudocode diagram illustrating the hardware acceleration computation process for 1 nmv and 5 mmv in this application.

[0062] Figure 5 This is a flowchart of a method that relates to the present invention. Detailed Implementation

[0063] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.

[0064] The terminology used in this article also includes:

[0065] nmv: Abbreviation for normal mv, which in this article represents the mv obtained after passing through me;

[0066] mmv: an abbreviation for merge mv, which in this article refers to the mv obtained through the merge technique;

[0067] Ref data: Reference frame data, which is selected from the coded frames as reference data for the current frame to be encoded. It is mainly used to obtain prediction blocks from reference frames during motion compensation. In H.265, the number of selectable reference frames is 1 to 15.

[0068] ddr: Double Data Rate Synchronous Dynamic Random Access Memory, which stores the reference frame data of an entire frame and has a large capacity;

[0069] RAM: Random Access Memory, where pred pixels are stored. It has a small capacity, typically enough to store the pred pixels of a few PUs.

[0070] maxNumMergeCand: Used to configure the number of candidate mv in merge mode;

[0071] mmvx: The xth mmv;

[0072] read_mmvx_ref: Reads the pixel values ​​of the reference frame corresponding to mmvx;

[0073] do_mmvx_mc: Performs mc on the pixel values ​​of the reference frame corresponding to mmvx;

[0074] store_mmvx_pred: Stores the pred pixel value of mmvx to RAM;

[0075] copy_nmv_pred: Copy the pred pixel value of nmv to RAM.

[0076] Because the pipelined ME (Meeting) process in the hardware architecture precedes the MER (Merge) process, the MV (Modular Value) of the ME is obtained before the MV of the MER. Furthermore, the MV of the MER and the MV of the ME are strongly correlated in both space and time. Therefore, there is a high probability that the MER MV is equal to the ME MV, and equal MV means that their pred (Pred) pixel values ​​are the same.

[0077] Utilizing the above characteristics, H.265 obtains a candidate mv list in Merge mode. After that, MC operations need to be performed for each mmvx. To perform the mc operation, this paper proposes the following methods, such as... Figure 3 As shown, first determine whether the current mmvx and nmv are the same:

[0078] (1) If the current values ​​of mmvx and nmv are the same, then mc calculation is not performed. Instead, the pred pixel result obtained by the mc operation of nmv is copied as the pred pixel of the current mmvx. This omits the current mmvx's mc operation.

[0079] (2) If the current values ​​of mmvx and nmv are inconsistent, it is necessary to read the reference frame pixels of mmvx. ), perform mc operation ( (Get the pred pixel.)

[0080] The mc operation is performed by using the position of the current block and the motion vector nmv to calculate the position of the reference block in the reference frame, and using the pixel of the reference block as the predicted value of the pixel of the current block.

[0081] like Figure 5 As shown, this invention relates to a hardware acceleration method for motion compensation computation in H.265 format merge mode, the method comprising the following steps:

[0082] S1. Read the reference frame pixels of nmv, perform motion compensation MC operation on nmv, and save the pred pixels of nmv to random access memory ram;

[0083] S2. Determine if the current mmvx and nmv are the same:

[0084] If the current mmvx and nmv values ​​are the same, then mc calculation is not performed; instead, the pred pixel result of nmv is copied as the pred pixel of the current mmvx. This omits the current mmvx's mc operation.

[0085] If the current mmvx and nmv values ​​are inconsistent, it is necessary to read the reference frame pixels of mmvx. ), perform mc operation ( (Get the pred pixel.)

[0086] S3. Save the pred pixel to RAM.

[0087] Then step S2 further includes:

[0088] S2.1, denot the first mmv as mmv0 and compare it with nmv.

[0089] When the two values ​​are equal, the mc operation is no longer performed on mmv0. Instead, the pred pixel of nmv is directly copied as the pred pixel of mmv0, and step S3 is performed.

[0090] When the two values ​​are not equal, it is necessary to read the reference frame pixel of mmv0, perform mc operation on mmv0 to obtain the pred pixel, and proceed to step S3.

[0091] S2.2, denoted as mmv1, the second mmv is compared with nmv.

[0092] When the two values ​​are equal, the mc operation is no longer performed on mmv1. Instead, the pred pixel of nmv is directly copied as the pred pixel of mmv1, and step S3 is performed.

[0093] When the two values ​​are not equal, it is necessary to read the reference frame pixel of mmv1, perform mc operation on mmv1 to obtain the pred pixel, and proceed to step S3.

[0094] S2.3, denot the third mmv as mmv2 and compare it with nmv.

[0095] When the two values ​​are equal, the mc operation is no longer performed on mmv2. Instead, the pred pixel of nmv is directly copied as the pred pixel of mmv2, and step S3 is performed.

[0096] When the two values ​​are not equal, it is necessary to read the reference frame pixel of mmv2, perform mc operation on mmv2 to obtain the pred pixel, and proceed to step S3.

[0097] S2.4, denot the fourth mmv as mmv3 and compare it with nmv.

[0098] When the two values ​​are equal, the mc operation is no longer performed on mmv3. Instead, the pred pixel of nmv is directly copied as the pred pixel of mmv3, and step S3 is performed.

[0099] When the two values ​​are not equal, it is necessary to read the reference frame pixel of mmv3, perform mc operation on mmv3 to obtain the pred pixel, and proceed to step S3.

[0100] S2.5, denot the 5th mmv as mmv4 and compare it with nmv.

[0101] When the two values ​​are equal, the mc operation is no longer performed on mmv4, and the pred pixel of nmv is directly used as the pred pixel of mmv4, and step S3 is performed.

[0102] When the two values ​​are not equal, it is necessary to read the reference frame pixel of mmv4, perform mc operation on mmv4 to obtain the pred pixel, and proceed to step S3.

[0103] The detailed process is as follows:

[0104] S1. Read the reference frame pixels of nmv, perform mc operation on nmv, and save the pred pixels of nmv to RAM.

[0105] S2.1 Compare mmv0 and nmv. When the two values ​​are equal, the MC operation is no longer performed on mmv0. Instead, the pred pixel of nmv is directly copied as the pred pixel of mmv0 and saved to RAM. When the two values ​​are not equal, the reference frame pixel of mmv0 needs to be read, the MC operation is performed on mmv0 to obtain the pred pixel, and then saved to RAM.

[0106] S2.2 Compare mmv1 and nmv. When the two values ​​are equal, the MC operation is no longer performed on mmv1. Instead, the pred pixel of nmv is directly copied as the pred pixel of mmv1 and saved to RAM. When the two values ​​are not equal, the reference frame pixel of mmv1 needs to be read, the MC operation is performed on mmv1 to obtain the pred pixel, and then saved to RAM.

[0107] S2.3. Compare mmv2 and nmv. When the two values ​​are equal, the MC operation is no longer performed on mmv2. Instead, the pred pixel of nmv is directly copied as the pred pixel of mmv2 and saved to RAM. When the two values ​​are not equal, the reference frame pixel of mmv2 needs to be read, the MC operation is performed on mmv2 to obtain the pred pixel, and then saved to RAM.

[0108] S2.4. Compare mmv3 and nmv. If their values ​​are equal, do not perform MC operation on mmv3. Instead, directly copy the pred pixel of nmv as the pred pixel of mmv3 and save it to RAM. If their values ​​are not equal, read the reference frame pixel of mmv3, perform MC operation on mmv3 to obtain the pred pixel, and save it to RAM.

[0109] S2.5. Compare mmv4 and nmv. When their values ​​are equal, do not perform MC operation on mmv4 again, and directly use the pred pixel of nmv as the pred pixel of mmv4 and save it to RAM. When their values ​​are not equal, it is necessary to read the reference frame pixel of mmv4, perform MC operation on mmv4 to obtain the pred pixel, and save it to RAM.

[0110] By summarizing the above process, the MC operation of merge mv (which is equal to me mv) and the reading of the reference frame can be directly omitted, thereby saving hardware resources and achieving hardware acceleration.

[0111] The pseudocode for the hardware-accelerated MC computation process for 1 nmV and 5 mmV proposed in this application is as follows: Figure 4 As shown.

[0112] This application allows for the omission of the MC operation corresponding to mmv when mmv and nmv are equal. This proposed hardware architecture accelerates hardware processing and reduces software computation without compromising encoding quality.

[0113] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A hardware acceleration method for motion compensation calculation in H.265 format merge mode, characterized in that, The method includes determining whether the current mmvx and nmv are the same, wherein the motion vector mv obtained after the merge encoding mode is denoted as mmv, the mmvx is the xth mmv, and the motion vector mv obtained after motion estimation me is denoted as nmv; if the current mmvx and nmv values ​​are the same, then motion compensation MC operation is not performed, but the pred pixel result obtained by MC operation of nmv is copied as the pred pixel of the current mmvx, thus omitting the MC operation of the current mmvx; if the current mmvx and nmv values ​​are different, then the reference frame pixels of mmvx need to be read, and MC operation is performed to obtain the pred pixel; The method further includes: storing the motion compensation result into a random access memory (RAM); In the method described above, the pipelined motion estimation me in the hardware structure precedes the merge mode, meaning that the motion vector mv of me is obtained before the merge mv. At the same time, the mv of the merge mode and the mv of the motion estimation me are correlated in space and time, so the mv of the merge mode may be equal to the mv of me. Therefore, it has the characteristic that if the mv is equal, the pred pixel value is also the same. The method utilizes the aforementioned characteristics to obtain a candidate mv list in the Merge mode of H265 format; The MV list is ; The method further includes the following steps: S1. Read the reference frame pixels of nmv, perform motion compensation MC operation on nmv, and save the pred pixels of nmv to random access memory ram; S2. Determine if the current mmvx and nmv are the same: S2.1, denot the first mmv as mmv0 and compare it with nmv. When the two values ​​are equal, the mc operation is no longer performed on mmv0. Instead, the pred pixel of nmv is directly copied as the pred pixel of mmv0, and step S3 is performed. When the two values ​​are not equal, it is necessary to read the reference frame pixel of mmv0, perform mc operation on mmv0 to obtain the pred pixel, and proceed to step S3. S2.2, denoted as mmv1, the second mmv is compared with nmv. When the two values ​​are equal, the mc operation is no longer performed on mmv1. Instead, the pred pixel of nmv is directly copied as the pred pixel of mmv1, and step S3 is performed. When the two values ​​are not equal, it is necessary to read the reference frame pixel of mmv1, perform mc operation on mmv1 to obtain the pred pixel, and proceed to step S3. S2.3, denot the third mmv as mmv2 and compare it with nmv. When the two values ​​are equal, the mc operation is no longer performed on mmv2. Instead, the pred pixel of nmv is directly copied as the pred pixel of mmv2, and step S3 is performed. When the two values ​​are not equal, it is necessary to read the reference frame pixel of mmv2, perform mc operation on mmv2 to obtain the pred pixel, and proceed to step S3. S2.4, denot the fourth mmv as mmv3 and compare it with nmv. When the two values ​​are equal, the mc operation is no longer performed on mmv3. Instead, the pred pixel of nmv is directly copied as the pred pixel of mmv3, and step S3 is performed. When the two values ​​are not equal, it is necessary to read the reference frame pixel of mmv3, perform mc operation on mmv3 to obtain the pred pixel, and proceed to step S3. S2.5, denot the 5th mmv as mmv4 and compare it with nmv. When the two values ​​are equal, the mc operation is no longer performed on mmv4, and the pred pixel of nmv is directly used as the pred pixel of mmv4, and step S3 is performed. When the two values ​​are not equal, it is necessary to read the reference frame pixel of mmv4, perform mc operation on mmv4 to obtain the pred pixel, and proceed to step S3. S3. Save the pred pixel to RAM.

2. The hardware acceleration method for motion compensation calculation in H.265 format merge mode according to claim 1, characterized in that, The motion compensation mc operation is to calculate the position of the reference block in the reference frame using the position of the current block and the motion vector mv, and use the pixel of the reference block as the predicted value of the pixel of the current block.

Images