A multi-directional motion refinement optimization method based on merge mode of VVC

Abstract

The present application discloses a VVC multi-directional motion refinement optimization method based on a Merge mode from the VVC inter prediction with a Merge process with a motion vector difference, which only has a small part of coding units (CUs) to search in a large step, and fails to consider the multi-directionality of the motion in the actual process. First, the first two MV information in the extended Merge candidate list is selected as the initial MV, and is expanded from four step lengths and eight directions; it is judged whether the current step length index is greater than 1, if yes, skip the chroma block motion compensation and the bi-directional optical flow prediction mode; otherwise, skip the chroma block motion compensation, and after the optimal Merge mode is obtained, entropy coding is carried out. Compared with the VVC coding standard, the average bit rate of Y, Cb and Cr is saved by 0.57%, 0.62% and 0.25% respectively by the VVC multi-directional motion refinement optimization method based on the Merge mode.

Description

Technical Field

[0001] This invention relates to the problem of inter-frame prediction optimization in the field of video coding, and in particular to a multi-directional motion refinement optimization method for the Versatile Video Coding (VVC) standard based on the Merge mode. Background Technology

[0002] With the rapid advancement of communication technology and the widespread adoption of smart terminals, digital video is ubiquitous in our lives. People have increasingly higher demands for the quality and resolution of digital video, making emerging video coding technologies crucial in today's internet age. However, due to limited bandwidth and storage capacity of various terminal devices, previous video compression coding technologies like H.264 / AVC and H.265 / HEVC can no longer meet market demands. In July 2020, Versatile Video Coding (VVC) was released as a new generation video coding standard. Compared to its predecessor, High Efficiency Video Coding (HEVC), VVC introduces many new technologies to improve compression efficiency, resulting in an approximately 50% improvement in coding performance.

[0003] The overall coding framework of the VVC standard is largely consistent with HEVC, still using a block-based hybrid coding structure. However, improvements or new technologies have been added to each stage of the framework. Taking the inter-frame prediction module as an example, some important improvements have been made: VVC adds motion vector prediction candidates based on historical information, increasing the length of the original Merge mode candidate list in HEVC from 5 to 6, and naming it the Extended Merge Mode. Simultaneously, to improve prediction accuracy, the inter-frame prediction signal is combined with the intra-frame prediction signal, adding a new inter-frame joint prediction mode. This mode uses weighting coefficients to weight the information from the two prediction blocks to obtain the final prediction value. VVC also adds a Merge technique with motion vector difference, using three parameters—start point, step size, and motion direction—to characterize the motion vector, reducing the slight deviation between this motion information and the actual coded block motion. In addition, the introduction of other new technologies has also improved the prediction accuracy to some extent, such as symmetric motion vector difference coding, sub-block-based temporal motion vector prediction, bidirectional weighted prediction, bidirectional optical flow, triangulation mode, affine motion compensation prediction, adaptive motion vector accuracy, and decoding-end motion vector correction.

[0004] While the Merge mode can further compress data, it also introduces certain errors, leaving much room for improvement. Currently, there is considerable research in this area. To improve the prediction accuracy of the Merge mode, Hong et al. proposed a novel Merge mode decision algorithm based on absolute bidirectional prediction difference and template matching criteria, utilizing a motion vector candidate algorithm derived from the decoder for Merge mode decision-making. Li et al. proposed a motion-assisted merging mode, obtaining more accurate motion vectors by searching for small motions in a specific search region. Shinobu Kudo et al. considered that encoding the reference block in intra-prediction mode reduces motion compensation efficiency in Merge mode, and therefore proposed an MV prediction method that considers prediction continuity, improving coding efficiency by setting the MV prediction candidate list to the blocks encoded in intra-prediction mode. Fan et al. discovered the shortcomings of existing intra-prediction in HEVC, explained the coding gain of boundary filtering, and established a boundary filtering method using the generalized Gaussian distribution function, extending the proposed boundary filter to the inter-frame prediction Merge process. Park et al. proposed an efficient method for constructing a candidate list of merged patterns, which reduces the use of zero motion vectors as candidate lists by generating an alternative merged candidate list. Summary of the Invention

[0005] In VVC inter-frame prediction, during the Merge mode prediction process, only a small portion of the coding units (CUs) in the Merge process with motion vector differences choose to search with a larger step size. However, a larger step size requires more bits for representation, which leads to a decrease in prediction performance.

[0006] To address the issue that VVC inter-frame prediction does not adequately consider the multi-directional nature of motion in actual processes, this invention aims to propose a method to effectively improve the coding performance of Merge mode in VVC without significantly increasing VVC coding time. This method adjusts the prediction process by modifying the search step size, adding search directions, and increasing the ways in which motion is represented. This method is then applied to the next-generation video coding standard VVC. The method mainly includes the following steps:

[0007] (1) Execute the extended Merge candidate procedure to initialize the extended Merge pattern candidate list.

[0008] (2) Initialize the rate distortion candidate pattern list and determine the length of the candidate list.

[0009] (3) Execute the CIIP candidate procedure and the standard MMVD candidate procedure to update the candidate pattern list in (2).

[0010] (4) Perform MMVD_PRO candidate selection, select the first two MV information from the extended Merge candidate list in (1) as the initial MV, and expand from four step sizes and eight directions.

[0011] (5) Determine if the current step size index is greater than 1. If it is, skip chroma block motion compensation and bidirectional optical flow prediction. Otherwise, skip chroma block motion compensation.

[0012] (6) Obtain the current CU predicted pixel value through motion compensation, and update the candidate mode list in (3) by calculating the rate distortion cost corresponding to different predicted values.

[0013] (7) The candidate mode list in (6) is reduced according to the rate distortion cost to refine the selection. The predicted value is calculated again by motion compensation and residual coding is performed to finally obtain the current best Merge mode and its cost.

[0014] (8) Entropy coding is performed on the optimal mode finally selected. If the selected mode is MMVD_PRO, the MMVD_PRO flag is encoded; otherwise, the flags of other modes are encoded.

[0015] The advantages and beneficial technical effects of this invention compared with the prior art are as follows:

[0016] (1) The VVC multi-directional motion refinement optimization method based on Merge mode proposed in this invention effectively improves the coding performance of video sequences without significantly increasing time complexity compared with the traditional VVC standard coding.

[0017] (2) The VVC multi-directional motion refinement optimization method based on Merge mode proposed in this invention analyzes the characteristic that the step size selection in Merge mode with motion vector difference is mainly concentrated on smaller step sizes, and uses this characteristic to compress the step size search range.

[0018] (3) The VVC multi-directional motion refinement optimization method based on Merge mode proposed in this invention fully considers the multi-directionality of motion in actual process, increases the expression of motion features in the tilt direction, and improves the accuracy of inter-frame prediction.

[0019] (4) The VVC multi-directional motion refinement optimization method based on Merge mode proposed in this invention utilizes the differences in the encoding methods corresponding to the search step size and search direction, adjusts the entropy encoding method, reduces the number of bits required for encoding, and improves the accuracy of inter-frame prediction. Attached Figure Description

[0020] Figure 1 This is a flowchart of the overall process for the VVC multi-directional motion refinement and optimization method based on the Merge pattern.

[0021] Figure 2 This is a schematic diagram of the search process of the method of the present invention.

[0022] Figure 3 Rate distortion curves and local magnifications of the sequence BQSquare under LDP configuration based on the method of this invention and the VVC standard.

[0023] Figure 4 Rate distortion curves and local magnifications of the PartyScene sequence under LDP configuration based on the method of this invention and the VVC standard. Detailed Implementation

[0024] The present invention will be further described in detail below with reference to the embodiments. It should be noted that the following embodiments are only used to further illustrate the present invention and should not be construed as limiting the scope of protection of the present invention. Those skilled in the art can make some non-essential improvements and adjustments to the present invention based on the above-described invention, and these improvements and adjustments should still fall within the scope of protection of the present invention.

[0025] (1) The method proposed in this invention is carried out on the VVC standard test code VTM12.0 platform. The configuration file is selected as encoder_lowdelay_P_vtm.cfg, the GOP is 16, the number of encoded frames is 100, and the quantization parameter QP is set to 22, 27, 32, 37 during encoding;

[0026] (2) Five official standard test sequences of types B, C, D, E, and F were used to verify the coding performance of the method proposed in this invention. Specifically, the standard test sequences used were Cactus, BasketballDrive, BQTerrace, BQMall, PartyScene, RaceHorsesC, BasketballDrill, BasketballPass, BlowingBubbles, BQSquare, RaceHorses, FourPeople, KristenAndSara, Johnny, BasketballDrillText, SlideEditing, and SlideShow;

[0027] (3) In the program of the algorithm proposed in this invention, the parameters in (1) are set in the configuration file encoder_lowdelay_P_vtm.cfg, and then all the video standard sequences to be tested as shown in (2) are input to obtain the results of the method of this invention, and the encoding time, bit rate and peak signal-to-noise ratio PSNR are recorded and statistically analyzed.

[0028] (4) In the VVC standard algorithm program, set the parameters in (1) in the configuration file encoder_lowdelay_P_vtm.cfg, and then input all the video standard sequences to be tested as shown in (2) to obtain the results of the VVC standard algorithm, and record and count the encoding time, bit rate and peak signal-to-noise ratio PSNR.

[0029] (5) The results of the above two categories are processed separately. The BD-rate is used to calculate the coding performance of the method of the present invention relative to VVC, and the relative time complexity is calculated using the relative coding time T. If the BD-rate is negative, it indicates that the coding performance of our method is improved.

[0030] (6) Table 1 shows a comparison of the coding performance between the present invention and the VVC standard algorithm under the lowdelay_P (LDP) coding structure configuration. Compared with the VVC standard algorithm, the method proposed in this invention achieves average bit savings of 0.57%, 0.62%, and 0.25% in the Y, Cb, and Cr components, respectively, with the highest bit savings reaching 1.33%, 3.18%, and 1.35%, respectively. This indicates that our method achieves better results.

[0031] (7) From Figure 3 and Figure 4 As can be seen, the rate-distortion curve of our proposed method is above the VVC standard. Therefore, under the same objective quality conditions, i.e. the same PSNR, our proposed method requires fewer coding bits, which is more conducive to saving bit information.

[0032] Table 1 compares the coding performance of this invention with VVC under LDP configuration.

[0033]

Claims

1. A VVC multi-directional motion refinement optimization method based on Merge mode, characterized in that: (1) Perform MMVD_PRO candidate selection, select the first two MV information in the extended Merge candidate list as the initial MV, jointly optimize step size selection and direction selection and extend: in the four directions of up, down, left and right, the original eight step sizes are reduced to the first four smaller step sizes, and the size of these four step sizes remains unchanged; in the tilt direction, the four step sizes are half of the first four smaller step sizes in the up, down, left and right directions respectively, and the extension of four step sizes and eight directions is achieved through the above settings; at the same time, it is determined whether the current step size index is greater than 1. If so, skip the chroma block motion compensation and bidirectional optical flow prediction method. Otherwise, skip only the chroma block motion compensation; (2) Based on motion compensation and rate distortion cost, the optimal Merge mode is obtained. The optimal mode is entropy encoded: the different points of the search step size and search direction corresponding to the encoding method are fully utilized to adjust the entropy encoding method. The search step size index code length is reduced by half, and the direction index code length is increased by only one bit. If the MMVD_PRO mode is selected, the MMVD_PRO flag bit is encoded; otherwise, the flag bit of other modes is encoded.

2. The VVC multi-directional motion refinement and optimization method based on Merge mode as described in claim 1, characterized in that... By leveraging the characteristic that step size selection in the Merge pattern mainly focuses on smaller step sizes, the step size search range is compressed, and the multi-directionality of motion in actual processes is fully considered, increasing the expression of motion features in the tilt direction.

3. The VVC multi-directional motion refinement and optimization method based on Merge mode as described in claim 1, characterized in that... The mode selection process is adaptively adjusted based on the rate-distortion cost criterion, taking into account both the step size and direction selection methods of the MMVD mode and the MMVD_PRO mode.

Images