Video coding method and related apparatus

Abstract

A video coding system is provided that generates merge mode with motion vector difference (MMVD) candidates and reduces resource usage. The system receives data to be encoded or decoded as a current block of a current picture of a video. The system identifies a plurality of MMVD candidates with different offset positions based on merge candidates of the current block. The system generates reference samples of the identified MMVD candidates. The system reconstructs the current block or encodes the current block into a bitstream using the generated reference samples. The system processes MMVD candidates in different groups: a first group of vertical MMVD candidates and a second group of horizontal MMVD candidates. The system generates reference samples of the identified MMVD candidates by applying a vertical filter to source reference samples of the horizontal MMVD candidates and then applying a horizontal filter to an output of the vertical filter.

Description

Technical Field

[0001] This invention relates to video decoding. Specifically, this disclosure relates to a method for generating merge mode with motion vector difference (MMVD) candidates. Background Technology

[0002] Unless otherwise stated herein, the methods described in this section are not prior art to the claims listed below, nor are those included in this section but are recognized as prior art.

[0003] High-Efficiency Video Coding (HEVC) is an international video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). HEVC is based on a hybrid block-based motion-compensated DCT transform coding architecture. The basic unit of compression, called a coding unit (CU), is a 2Nx2N square block. Each CU can be recursively divided into four smaller CUs until a predetermined minimum size is reached. Each CU contains one or more prediction units (PUs).

[0004] To achieve optimal encoding / decoding efficiency in HEVC's hybrid codec architecture, each PU has two prediction modes: intra-frame prediction and inter-frame prediction. For intra-frame prediction, spatially adjacent reconstructed pixels can be used to generate orientation predictions. HEVC allows for a maximum of 35 orientations. For inter-frame prediction, a temporally reconstructed reference frame can be used to generate motion-compensated predictions. There are three different modes: skip, merge, and Advanced Motion Vector Prediction (AMVP) mode.

[0005] When the PU performs encoding and decoding in inter-frame AMVP mode, motion compensation prediction is performed using transmitted motion vector difference (MVD), which can be used together with the motion vector predictor (MVP) to derive the motion vector (MV). To determine the MVP in inter-frame AMVP mode, an advanced motion vector prediction (AMVP) scheme is used to select the motion vector predictor from a set of AMVP candidates, which includes two spatial MVPs and one temporal MVP. Therefore, in AMVP mode, the MVP index and the corresponding MVD of the MVP need to be encoded and transmitted. Furthermore, inter-frame prediction directions are used to specify the prediction direction between bidirectional and unidirectional prediction, i.e., list 0 (L0) and list 1 (L1), and the reference frame index for each list should also be encoded and transmitted.

[0006] When encoding / decoding a PU in skip or merge modes, no motion information is transmitted except for the merge index of the selected candidate. This is because skip and merge modes utilize a motion inference method (MV = MVP + MVD, where MVD is zero) to obtain motion information from spatially adjacent blocks (spatial candidates) or temporally adjacent blocks (temporal candidates) located in the co-located picture, where the co-located picture is the first reference picture in list 0 or list 1, which is transmitted in the segment header. In the case of a skip PU, the residual signal is omitted. To determine the merge index for skip and merge modes, a merge scheme is used to select motion vector predictors from a merge candidate set containing four spatial MVPs and one temporal MVP. Summary of the Invention

[0007] The following overview is illustrative only and is not intended to be limiting in any way. That is, the following overview is provided to introduce the concepts, highlights, benefits, and advantages of the novel and non-obvious techniques described herein. Selected, but not all, implementations are further described in the detailed description below. Therefore, the following overview is not intended to identify the essential features of the claimed subject matter, nor is it intended to determine the scope of the claimed subject matter.

[0008] Some embodiments of this disclosure provide a video encoding / decoding system that generates merge mode (MMVD) candidates with motion vector difference and reduces resource usage. The system receives data that will be encoded or decoded into a current block of a current frame of video. The system identifies multiple MMVD candidates at different offset positions based on the merge candidates of the current block. The system generates reference samples of the identified MMVD candidates. The system reconstructs the current block or encodes the current block into a bitstream using the generated reference samples.

[0009] In some embodiments, the identified MMVD candidates may include a first set of MMVD candidates and a second set of MMVD candidates. The system can generate reference samples by deriving a first set of reference samples from the first set of MMVD candidates and a second set of reference samples from the second set of MMVD candidates. The system can derive the first set of reference samples by accessing reference samples restricted to a first access range, and can derive the second set of reference samples by accessing reference samples restricted to different second access ranges. In some embodiments, the first set of MMVD candidates includes two or more vertical MMVD candidates having a vertical offset position relative to the merged candidate, and the second set of MMVD candidates includes two or more horizontal MMVD candidates having a horizontal offset position relative to the merged candidate. The first set of MMVD candidates may include all vertical MMVD candidates and the second set of MMVD candidates may include all horizontal MMVD candidates for encoding and decoding the current block using an MMVD mode.

[0010] In some embodiments, the system generates reference samples for the identified MMVD candidates by applying a vertical filter to source reference samples of horizontal MMVD candidates (MMVD candidates with horizontal offset positions), and then applying the horizontal filter to the output of the vertical filter. The source reference samples may be stored in a shift register of the vertical filter. In some embodiments, each source reference sample has fewer bits (e.g., 10 bits) than the filtered result of each reference sample (e.g., 16 bits). When MMVD candidates come from an unscaled list (not time-scaled), at least some of the output of the horizontal filter is reused for different horizontal filter candidates. Attached Figure Description

[0011] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this disclosure. The drawings illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure. It should be noted that the drawings are not necessarily drawn to scale, as in actual implementations certain elements may be shown out of proportion to clearly illustrate the concepts of the present disclosure.

[0012] Figure 1 The MMVD candidates and their corresponding offsets are conceptually illustrated.

[0013] Figure 2 This conceptually illustrates the access range for generating reference samples for merging and MMVD modes.

[0014] Figure 3 MMVD candidates are shown, and the MMVD candidates are divided into two groups to reduce the access range of the reference samples of the MMVD mode.

[0015] Figure 4A -B indicates an alternative partition for MMVD candidates, used to reduce the access range for generating reference samples.

[0016] Figure 5 The filter for the horizontal MMVD candidate is shown.

[0017] Figure 6A -B indicates filtering the horizontal MMVD candidates with a reduced filter size, where vertical filtering is applied before horizontal filtering.

[0018] Figure 7 An example video encoder that can be used in MMVD mode is shown.

[0019] Figure 8 This shows a portion of the video encoder that implements MMVD mode separation and interpolation reordering.

[0020] Figure 9 This conceptually illustrates the process of encoding pixel blocks using MMVD.

[0021] Figure 10 This shows an example video decoder using MMVD mode.

[0022] Figure 11 This shows a portion of the video decoder that implements MMVD mode separation and interpolation reordering.

[0023] Figure 12 This conceptually illustrates the process of decoding pixel blocks using MMVD.

[0024] Figure 13 An electronic system implementing some embodiments of the present disclosure is conceptually illustrated. Detailed Implementation

[0025] In the following detailed description, numerous specific details are illustrated by way of example in order to provide a thorough understanding of the teachings herein. Any variations, derivatives, and / or extensions based on the teachings described herein are within the scope of this disclosure. In some cases, well-known methods, processes, elements, and / or circuits relating to one or more exemplary embodiments disclosed herein may be described at a relatively high level without detail to avoid unnecessarily obscuring aspects of the teachings of this disclosure.

[0026] I. Merge Mode with Motion Vector Difference (MMVD)

[0027] Merge Mode with Motion Vector Difference (MMVD) is a new codec tool for the Versatile Video Coding (VVC) standard. Unlike conventional merge modes where implicitly derived motion information is directly used to generate prediction samples from the current CU, in MMVD, the derived motion information is further refined using motion vector difference (MVD). MMVD also expands the candidate list of merge modes by adding additional MMVD candidates based on a predetermined offset (also known as the MMVD offset).

[0028] After sending the skip and merge flags, the MMVD flag is sent to specify whether the MMVD mode is used for the CU. If the MMVD mode is used, the selected merge candidate is refined using the MVD information. The MVD information also includes a merge candidate flag, a distance index specifying the motion amplitude, and an index indicating the motion direction. The merge candidate flag is sent to specify which of the first two merge candidates will be used as the starting MV.

[0029] The distance index is used to specify motion amplitude information by indicating a predetermined offset from the starting MV. The offset can be added to either the horizontal or vertical component of the starting MV. Table l-1 below specifies an example mapping from the distance index to the predetermined offset:

[0030] Table 1-1. Distance Index

[0031]

[0032] The direction index indicates the direction of the MVD relative to the starting point. The direction index can represent one of the four directions shown in Table 1-2.

[0033] Table 1-2 specifies the sign of the MV offset by the direction index.

[0034] Distance Index 00 01 10 11 x-axis + – N / A N / A y-axis N / A N / A + –

[0035] It is important to note that the meaning of the MVD symbol can vary depending on the information of the starting MV. When the starting MV is a unidirectional prediction MV or a bidirectional prediction MV where both lists point to the same side of the current image (i.e., the image order count or POC of the two reference images is either greater than or less than the POC of the current image), the symbols in Table 1-2 specify the symbol of the MV offset added to the starting MV. When the starting MV is a bidirectional prediction MV where both MVs point to different sides of the current image (i.e., one reference has a POC greater than the POC of the current image, while the other reference has a POC less than the POC of the current image), each symbol in Table 1-2 specifies the symbol of the MV offset added to the list 0 MV component of the starting MV, while the symbols for list 1 MV have the opposite value. In some embodiments, the predetermined offset (MmvdOffset) of the MMVD candidate is derived from the distance value (MmvdDistance) and the direction sign (MmvdSign) or expressed as the distance value (MmvdDistance) and the direction sign (MmvdSign).

[0036] Figure 1 The diagram conceptually illustrates MMVD candidates and their corresponding offsets. It shows a merge candidate 110 as the starting MV and several MMVD candidates in the vertical and horizontal directions. Each MMVD candidate is derived by applying an offset to the starting MV 110. For example, MMVD candidate 122 is derived by adding an offset of 2 to the horizontal component of merge candidate 110, and MMVD candidate 124 is derived by adding an offset of -1 to the vertical component of merge candidate 110. MMVD candidates with offsets in the horizontal direction, such as MMVD candidate 122, are referred to as horizontal MMVD candidates. MMVD candidates with offsets in the vertical direction, such as MMVD candidate 124, are referred to as vertical MMVD candidates.

[0037] In some embodiments, the MMVD offsets used to derive MMVD candidates are scaled based on the time positions (e.g., POCs) of the L1 and L0 references. Specifically, for each MMVD candidate, the offset of the L1 MVD is scaled if abs(currPocDiffL0) ≥ abs(currPocDiffL1), otherwise the offset of the L0 MVD is scaled. If currPocDiffL0 and currPocDiffL1 have different signs, inverse matching is applied when scaling the MMVD offsets.

[0038] When generating reference samples for each MMVD candidate, a software-style approach can be used to generate reference samples for all MMVD candidates in a rate-distortion optimization (RDO) phase. Figure 2 This diagram conceptually illustrates the access ranges used to generate merge and MMVD pattern reference samples. The current block 200 is shown. MMVD candidates 210, 220, 230, and 240 are also shown. MMVD candidate 210 is a horizontal MMVD candidate with a horizontal offset of -8. MMVD candidate 220 is a vertical MMVD candidate with a vertical offset of -8. MMVD candidate 230 is a horizontal MMVD candidate with a horizontal offset of +8. MMVD candidate 240 is a vertical MMVD candidate with a vertical offset of +8.

[0039] Larger MMVD offsets in different directions may require larger access ranges to generate reference samples, thus increasing hardware costs. Figure 2 In the example, when the current block 200 is 4x4, the access range of the reference sample 250 in merge mode is only 11x11. However, if MMVD mode is used, and when the maximum offset of MMVD is 8, the access range of the reference sample 260 will increase to 27x27.

[0040] Some embodiments provide a video encoding / decoding system in which the access range of reference samples used to generate MMVD patterns is reduced. In some embodiments, MMVD candidates are divided into several parts. For example, in some embodiments, the MMVD candidates of the current module are divided into two different groups of MMVD candidates, such that the MMVD patterns are processed in two low-complexity (LC) rate-distortion-optimization (RDO) stages: a first LC-RDO stage for processing horizontal MMVD candidates and a second LC-RDO stage for processing vertical MMVD candidates.

[0041] Figure 3The diagram illustrates that MMVD candidates are divided into two groups to reduce the access range of the MMVD mode reference sample. In the diagram, the MMVD candidate module of the current block 200 is processed in two groups: a vertical group 301 and a horizontal group 302. The vertical group 301 includes all vertical MMVD candidates, including MMVD candidate 220 (vertical offset -8) and MMVD candidate 240 (vertical offset +8). The horizontal group includes all horizontal MMVD candidates, including MMVD candidate 210 (horizontal offset -8) and MMVD candidate 230 (horizontal offset +8). The horizontal group 302 has an access range 310 of size 27x11. The vertical group 301 has an access range 320 of size 11x27.

[0042] Other arrangements are also possible for dividing MMVD candidates into different groups to narrow the access range. Figures 4A-B illustrate other partitioning of MMVD candidates to reduce the access range for generating reference samples.

[0043] Figure 4A The diagram illustrates two L-shaped groups 401 and 402 that divide the MMVD candidates into the current block 200. Each L-shaped group comprises half of the horizontal MMVD candidates and half of the vertical MMVD candidates. L-shaped group 401 includes horizontal MMVD candidates with a negative offset down to offset -8, including MMVD candidate 210. L-shaped group 401 also includes vertical MMVD candidates with a positive offset of +8, including MMVD candidate 240. The access range of L-shaped group 401 is 19x19.

[0044] L-shaped group 402 includes horizontal MMVD candidates with positive offsets up to +8, including MMVD candidate 230. L-shaped group 402 also includes vertical MMVD candidates with negative offsets down to -8, including MMVD candidate 220 (offset -8). The access range of L-shaped group 402 is 19x19 in size (therefore the total size is 19x19x2).

[0045] Figure 4B The diagram illustrates the division of MMVD candidates into four candidate groups 411-414. Group 411 includes horizontal MMVD candidates with negative offsets down to -8, including MMVD candidate 210. Group 412 includes vertical MMVD candidates with negative offsets down to -8, including MMVD candidate 220. Group 413 includes horizontal MMVD candidates with positive offsets up to +8, including MMVD candidate 230. Group 414 includes vertical MMVD candidates with positive offsets up to +8, including MMVD candidate 240. Each of the four groups 411-414 has a 19x11 access range (therefore, the total size is 19x11x4).

[0046] In some embodiments, the reference samples of the MMVD candidates are filtered due to the merging of the candidate fractional positions and the MMVD candidate fractional offsets. To derive reference samples for different MMVD candidates, particularly those with fractional positions, vertical and horizontal filtering are used to generate the filtered reference samples.

[0047] In some embodiments, reference samples for all horizontal MMVD candidates are first filtered by a horizontal filter and then by a vertical filter. For example, for horizontal MMVD candidates in horizontal candidate group 302 (horizontal offset from -8 to +8), the size of the reference sample is 28x4. The video codec may first apply the horizontal filter to the reference sample and then store the filtered result in a shift register (28 is the width of the access range of the horizontal MMVD candidates, and 4 is the number of rows in the 4x4 block). The video codec then applies the vertical filter to the data stored in the shift register.

[0048] Figure 5 This diagram illustrates filtering for horizontal MMVD candidates. The figure shows filtering for a 4x4 pixel block of 500, with the merged candidate score positions at (+1 / 4, -1 / 4). The diagram also shows reference samples of the filtered results generated by horizontal and vertical filtering for all horizontal MMVD offsets (-8, -4, -2, -1, -1 / 2, -1 / 4, 1 / 4, 1 / 2, 1, 2, 4, 8). As shown, 33 reference samples of the filtered results are generated per row, with 4 rows for the 4x4 block.

[0049] To generate these 33 x 4 = 132 filtered reference samples, the video codec first applies a horizontal filter of 132 samples, and then applies a vertical filter of 132 samples. Shift registers are used to store the intermediate results of the horizontal filtering. The shift registers initially store the results of the horizontal filtering. Each sample of the horizontal filtering result has 16 bits, and the size of the shift register is 33 (samples / row) x 7 (rows) x 16 (bits) = 3696 bits. (The vertical filtering result for each row of horizontal MMVD candidates consumes the results of the first 3 rows of horizontal filtering, therefore a total of 7 rows of shift registers are used).

[0050] Figure 5An example is an MMVD candidate sample that is not scaled based on time position (unscaled list), and some filtered reference samples can be shared or reused by different MMVD candidates with different offsets. However, if the MMVD candidates are scaled based on time position (scaled list), different MMVD candidates with different offsets cannot share or reuse filtered reference samples. Therefore, for 12 MMVD candidate samples in the scaled list, horizontal and vertical filtering is used to generate 4 (width) x 12 (MMVD candidates) x 4 (rows) = 192 reference samples for pixel block 500. To generate these 192 filtered reference samples, the video codec can first apply horizontal filtering to 192 samples and then apply vertical filtering to 192 samples. If a shift register is used to store the horizontal filtering result used for vertical filtering, and each filtered reference sample has 16 bits, then the size of the shift register is 4 (width) x 12 (MMVD candidates) x 7 (rows) x 16 (bits) = 5376 bits.

[0051] Because MMVD modes have many fractional offsets, numerous horizontal and vertical filters are used, particularly for horizontal MMVD candidates with offsets of ±1, ±1 / 2, and ±1 / 4. It is observed that all horizontal MMVD candidates have the same vertical interpolation phase. Some embodiments of this disclosure utilize this characteristic to reduce the size of the vertical filters.

[0052] In some embodiments, the video codec stores a reference sample (e.g., 10 bits instead of the 16 bits used for the horizontal filtering result) directly in a shift register, and applies vertical filtering to the data stored in the shift register. The video codec then applies horizontal filtering to the result or output of vertical filtering. This results in a filter size smaller than the filter size if horizontal filtering were applied before vertical filtering.

[0053] Figure 6A -B shows a horizontal MMVD candidate filter with a reduced filter size, where the vertical filter is applied before the horizontal filter. This example is for a 4x4 block of 600, whose merged candidates have fractional positions (+1 / 4, -1 / 4). The filter is used to generate a filter reference sample with horizontal MMVD candidates offset by -8, -4, -2, -1, -1 / 2, -1 / 2, -1 / 4, +1 / 4, +1 / 2, 1, 2, 4, 8.

[0054] Figure 6A This figure shows the vertical filtering applied to reference samples within the access range of the horizontal MMVD candidates. The figure illustrates 27x4 = 108 vertical filtering results generated from 27x4 reference samples, since the horizontal MMVD candidates, which are in a 4x4 block, require 27 reference samples per row.

[0055] Figure 6B The horizontal filtering, resulting from the vertical filtering, is shown. Horizontal filtering is used to generate filtered reference samples for each horizontal MMVD candidate. As shown, horizontal filtering generates 33 filtered samples for each of the four rows. As mentioned above, for MMVD candidates not scaled based on time position (the unscaled list), the filtered samples can be shared by different MMVD candidates. Therefore, 33 samples per row is sufficient for all horizontal MMVD candidates with offsets from -8 to +8. Thus, 33 x 4 = 132 filtered reference samples are generated.

[0056] To generate these 33 x 4 = 132 filtered reference samples, the video codec applies a vertical filter of 108 samples to the source reference samples (pixel values ​​at integer positions) of the horizontal MMVD candidates, and then applies the horizontal filter of 132 samples to the result of the vertical filter. A shift register is used to store the source reference samples. The shift register initially stores the source reference samples at integer positions. Each filtered source reference sample is 10 bits, and the size of the shift register is 27 (samples / row) x 7 (rows) x 10 (bits) = 1890 bits. (Generating the vertical filter result for each row of horizontal MMVD candidates consumes the reference samples from the first 3 rows, so a total of 7 rows of shift registers are used.)

[0057] If the MMVD candidates are scaled based on time position (scaled list), different horizontal MMVD candidates with different offsets cannot reuse or share the filtered reference samples. Therefore, for 12 MMVD candidates in the scaled list, using horizontal and vertical filtering, 4 (width) x 12 (MMVD candidates) x 4 (rows) = 192 reference samples are generated for 4 rows. A shift register can be used to store the source reference samples. The shift register initially stores the source reference samples at integer positions. Each filtered source reference sample is 10 bits, and the size of the shift register is 27 (samples / row) x 7 (rows) x 10 (bits) = 1890 bits.

[0058] Therefore, significant resource savings can be achieved in computing the filtered reference samples for horizontal MMVD candidates by applying vertical filtering before horizontal filtering. Table I-3 below summarizes the computational resource usage for filtering horizontal MMVD candidates. The table also shows the percentage of resource savings when vertical filtering is applied first (compared to applying horizontal filtering first). As shown in the table, applying vertical filtering first saves on the vertical filter size and shift register size without incurring additional computational resource costs.

[0059] Table I-3: Calculations Used

[0060]

[0061]

[0062] II. Example Video Encoder

[0063] Figure 7 An example video encoder 700 is shown that can encode pixel blocks using MMVD mode. As shown, the video encoder 700 receives an input video signal from a video source 705 and encodes the signal into a bitstream 795. The video encoder 700 has several elements or modules for encoding the signal from the video source 705, including at least some of the following: a transform module 710, a quantization module 711, an inverse quantization module 714, an inverse transform module 715, an intra-frame estimation module 720, an intra-frame prediction module 725, a motion compensation module 730, a motion estimation module 735, a loop filter 745, a reconstructed picture buffer 750, an MV buffer 765, an MV prediction module 775, and an entropy encoder 790. The motion compensation module 730 and the motion estimation module 735 are part of the inter-frame prediction module 740.

[0064] In some embodiments, modules 710-790 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device or electronic device. In some embodiments, modules 710-790 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 710-790 are shown as separate modules, some modules may be combined into a single module.

[0065] Video source 705 provides the raw video signal, which presents the pixel data of each video frame without compression. Subtractor 708 calculates the difference between the raw video pixel data of video source 705 and the predicted pixel data 713 from motion compensation module 730 or intra-frame prediction module 725. Transform module 710 converts the difference (or residual pixel data or residual signal) into transform coefficients (e.g., by performing discrete cosine transform or DCT). Quantization module 711 quantizes the transform coefficients into quantized data (or quantization coefficients) 712, which is encoded into a bitstream 795 by entropy encoder 790.

[0066] The inverse quantization module 714 dequantizes the quantized data (or quantization coefficients) 712 to obtain transform coefficients, and the inverse transform module 715 performs an inverse transform on the transform coefficients to generate a reconstruction residual 719. The reconstruction residual 719 is added to the predicted pixel data 713 to generate reconstructed pixel data 717. In some embodiments, the reconstructed pixel data 717 is temporarily stored in a line buffer (not shown) for intra-frame prediction and spatial MV prediction. The reconstructed pixels are filtered by a loop filter 745 and stored in a reconstructed image buffer 750. In some embodiments, the reconstructed image buffer 750 is external memory to the video encoder 700. In some embodiments, the reconstructed image buffer 750 is internal memory to the video encoder 700.

[0067] Intra-frame estimation module 720 performs intra-frame prediction based on reconstructed pixel data 717 to generate intra-frame prediction data. The intra-frame prediction data is provided to entropy encoder 790 to be encoded into bitstream 795. The intra-frame prediction data is also used by intra-frame prediction module 725 to generate predicted pixel data 713.

[0068] The motion estimation module 735 performs inter-frame prediction by generating MVs (Motion Values) to reference pixel data of previously decoded frames stored in the reconstructed image buffer 750. These MVs are provided to the motion compensation module 730 to generate predicted pixel data.

[0069] Instead of encoding the complete actual MV in the bitstream, the video encoder 700 uses MV prediction to generate a predicted MV, and the difference between the MV used for motion compensation and the predicted MV is encoded as residual motion data and stored in the bitstream 795.

[0070] Based on the reference MV generated for encoding the previous video frame, i.e., the motion-compensated MV used to perform motion compensation, the MV prediction module 775 generates the predicted MV. The MV prediction module 775 obtains the reference MV from the previous video frame from the MV buffer 765. The video encoder 700 stores the MV generated for the current video frame in the MV buffer 765 as the reference MV for generating the predicted MV.

[0071] The MV prediction module 775 uses a reference MV to create the predicted MV. The predicted MV can be calculated through spatial MV prediction or temporal MV prediction. The difference (residual motion data) between the predicted MV and the motion compensation MV (MC MV) of the current frame is encoded into the bitstream 795 by the entropy encoder 790.

[0072] The entropy encoder 790 encodes various parameters and data into a bitstream 795 using entropy encoding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding. The entropy encoder 790 encodes various header elements, flags, along with quantized transform coefficients 712 and residual motion data as syntax elements into the bitstream 795. The bitstream 795 is then stored in a storage device or transmitted to the decoder via a communication medium such as a network.

[0073] The loop filter 745 performs filtering or smoothing operations on the reconstructed pixel data 717 to reduce encoding / decoding artifacts, particularly at pixel block boundaries. In some embodiments, the filtering operation performed includes sample adaptive offset (SAO). In some embodiments, the filtering operation includes adaptive loop filter (ALF).

[0074] Figure 8 This figure illustrates a portion of a video encoder 700 that implements MMVD mode separation and interpolation reordering. Specifically, the figure shows elements of the video encoder 700, which is configured to process MMVD candidates from two or more groups, and to perform vertical filtering before horizontal filtering of some of the MMVD candidates.

[0075] As shown in the figure, the MV prediction module 775 accesses the MV buffer 765 to identify candidates for various MV prediction modes, including candidates for merged modes and MMVD modes. The interpolation filter module 800 receives pixel data from the reconstructed image buffer 750 to use as source reference samples. The interpolation filter module 800 performs fractional position interpolation based on the source reference samples of various candidates identified by the MV prediction module 775. Specifically, the MMVD candidates are processed into two (or more) groups, including a vertical MMVD candidate group 802 and a horizontal MMVD candidate group 804. In some embodiments, the two groups 802 and 804 are processed in two different LC-RDO stages.

[0076] The interpolation filter module 800 includes a horizontal priority filter 810 and a vertical priority filter 820. The horizontal priority filter 810 performs horizontal filtering on the source reference samples, stores the filtered result in a shift register, and performs vertical filtering on the data stored in the shift register. The vertical priority filter 820 stores the source reference samples in a shift register, performs vertical filtering on the data in the shift register, and performs horizontal filtering on the result of the vertical filtering. In some embodiments, the horizontal MMVD candidate group 804 is filtered by the vertical priority filter 820, while other prediction candidate groups, including the vertical MMVD candidate group 802, are processed by the horizontal priority filter 810. The output of the interpolation filter module 800 is then provided to the inter-frame prediction module 740 for motion compensation and motion estimation operations. Based on the filtered reference samples, the inter-frame prediction module 740 can select one of the identified MMVD candidates as the prediction mode to encode the current block, such that the generated reference samples of the selected MMVD candidate are the basis for the residual signal encoded into the bitstream.

[0077] Figure 9 A conceptual illustration shows a process 900 for encoding pixel blocks using MMVD. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing encoder 700 perform process 900 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing encoder 700 performs process 900.

[0078] The encoder receives (in block 910) data that will be encoded as the current block of pixels in the current image. The encoder identifies (in block 920) multiple MMVD candidates at different offset positions based on the merged candidates of the current block. The identified MMVD candidates include those with fractional offset positions (e.g., ±1 / 2, ±1 / 4) and those with integer offset positions (e.g., ±1, ±2, ±4, ±8).

[0079] The encoder generates (in block 930) reference samples of the identified MMVD candidates. The encoder encodes the current block (in block 940) into a bitstream using the generated reference samples. In some embodiments, the encoder selects one of the identified MMVD candidates as a prediction mode to encode the current block such that the generated reference samples of the selected MMVD candidate form the basis of the residual signal encoded into the bitstream.

[0080] In some embodiments, the identified MMVD candidates may include a first set of MMVD candidates and a second set of MMVD candidates, as referred to above. Figure 3As described in Figure 4, the encoder can generate reference samples by deriving a first set of reference samples for a first set of MMVD candidates and a second set of reference samples for a second set of MMVD candidates. The encoder can derive the first set of reference samples by accessing reference samples restricted to a first access range, and can derive the second set of reference samples by accessing reference samples restricted to different second access ranges. In some embodiments, the first set of MMVD candidates includes two or more vertical MMVD candidates having a vertical offset position relative to the merged candidates, and the second set of MMVD candidates includes two or more MMVD candidates having a horizontal offset position relative to the merged candidates. The first set of MMVD candidates may include all vertical MMVD candidates and the second set of MMVD candidates may include all horizontal MMVD candidates for encoding and decoding the current block using MMVD mode. Processing MMVD candidates in separate groups allows the encoder to use a reduced access range in MMVD mode.

[0081] In some embodiments, as described with reference to Sections 6A-B, the encoder generates reference samples of the identified MMVD candidates by applying a vertical filter to source reference samples of horizontal MMVD candidates (MMVD candidates with horizontal offset positions), and then applying the horizontal filter to the output of the vertical filter. The source reference samples may be stored in a shift register of the vertical filter. In some embodiments, each source reference sample has fewer bits (e.g., 10 bits) than the filtered result of each reference sample (e.g., 16 bits). When MMVD candidates come from an unscaled list (not time-scaled), at least some of the output of the horizontal filter is reused for different horizontal filter candidates. Applying the vertical filter to the horizontal MMVD candidates before the horizontal filter can achieve significant savings in computational resources compared to applying the horizontal filter before the vertical filter.

[0082] III. Example Video Decoder

[0083] In some embodiments, the encoder may send (or generate) one or more syntax elements in a bitstream, such that the decoder can parse the one or more syntax elements from the bitstream.

[0084] Figure 10An example video decoder 1000 that can use the MMVD mode is shown. As shown, the video decoder 1000 is an image decoding or video decoding circuit that receives a bitstream 1095 and decodes the contents of the bitstream into pixel data of video frames for display. The video decoder 1000 has several elements or modules for decoding the bitstream 1095, including some of the following: an inverse quantization module 1011, an inverse transform module 1010, an intra-frame prediction module 1025, a motion compensation module 1030, a loop filter 1045, a decoded image buffer 1050, an MV buffer 1065, an MV prediction module 1075, and a parser 1090. The motion compensation module 1030 is part of the inter-frame prediction module 1040.

[0085] In some embodiments, modules 1010-1090 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 1010-1090 are hardware circuit modules implemented by one or more ICs of an electronic device. Although modules 1010-1090 are shown as separate modules, some modules may be combined into a single module.

[0086] Parser 1090 (or entropy decoder) receives bitstream 1095 and performs initial parsing according to the syntax defined by the video coding or image coding standard. The parsed syntax elements include various header elements, flags, and quantization data (or quantization coefficients) 1012. Parser 1090 parses the various syntax elements using entropy coding techniques such as context-adaptive binary arithmetic coding (ABAC) or Huffman coding.

[0087] The inverse quantization module 1011 dequantizes the quantized data (or quantization coefficients) 1012 to obtain transform coefficients, and the inverse transform module 1010 performs an inverse transform on the transform coefficients 1016 to generate a reconstructed residual signal 1019. The reconstructed residual signal 1019 is added to the predicted pixel data 1013 from the intra-frame prediction module 1025 or the motion compensation module 1030 to generate decoded pixel data 1017. The decoded pixel data is filtered by the loop filter 1045 and stored in the decoded image buffer 1050. In some embodiments, the decoded image buffer 1050 is external memory to the video decoder 1000. In some embodiments, the decoded image buffer 1050 is internal memory to the video decoder 1000.

[0088] Intra-prediction module 1025 receives intra-prediction data from bitstream 1095, and accordingly generates predicted pixel data 1013 from decoded pixel data 1017 stored in decoded image buffer 1050. In some embodiments, decoded pixel data 1017 is also stored in a line buffer (not shown) for intra-prediction and spatial MV prediction.

[0089] In some embodiments, the contents of the decoded image buffer 1050 are used for display. The display device 1055 either obtains the contents of the decoded image buffer 1050 for direct display, or obtains the contents of the decoded image buffer into a display buffer. In some embodiments, the display device receives pixel values ​​from the decoded image buffer 1050 via pixel transfer.

[0090] The motion compensation module 1030 generates predicted pixel data 1013 from the decoded pixel data 1017 stored in the decoded image buffer 1050 based on the motion compensation MV (MC MV). These motion compensation MVs are decoded by adding the residual motion data received from the bitstream 1095 to the predicted MV received from the MV prediction module 1075.

[0091] The MV prediction module 1075 generates a predicted MV based on a reference MV generated for decoding a previous video frame (e.g., a motion-compensated MV used to perform motion compensation). The MV prediction module 1075 retrieves the reference MV of the previous video frame from the MV buffer 1065. The video decoder 1000 stores the motion-compensated MV generated for decoding the current video frame in the MV buffer 1065 as a reference MV for generating the predicted MV.

[0092] The loop filter 1045 performs filtering or smoothing operations on the decoded pixel data 1017 to reduce encoding / decoding artifacts, particularly at pixel block boundaries. In some embodiments, the filtering operation includes sample adaptive offset (SAO). In some embodiments, the filtering operation includes adaptive loop filter (ALF).

[0093] Figure 11 This diagram illustrates portions of a video decoder 1000 that implements MMVD mode separation and interpolation reordering. Specifically, the diagram shows elements of the video decoder 1000 configured to process MMVD candidates from two or more groups and to perform vertical filtering before horizontal filtering of some of the MMVD candidates.

[0094] As shown in the figure, the MV prediction module 1075 accesses the MV buffer 1065 to identify candidates for various MV prediction modes, including merge mode and MMVD mode candidates. The interpolation filter module 1100 receives pixel data from the decoded image buffer 1050 to use as source reference samples. The interpolation filter module 1100 performs fractional position interpolation on the various candidates identified by the MV prediction module 1075 based on the source reference samples. For MMVD, an MMVD candidate is generated, which can be an MMVD candidate with a horizontal offset or an MMVD candidate with a vertical offset.

[0095] The interpolation filter module 1100 includes a horizontal priority filter 1110 and a vertical priority filter 1120. The horizontal priority filter 1110 performs horizontal filtering on the source reference samples, stores the filtered result in a shift register, and performs vertical filtering on the data stored in the shift register. The vertical priority filter 1120 stores the source reference samples in a shift register, performs vertical filtering on the data in the shift register, and performs horizontal filtering on the result of the vertical filtering. The entropy decoder 1090 can provide the selection of MMVD candidates (based on signaling in the bitstream 1095) for which the interpolation filter 1100 acquires the source reference samples and performs filtering. The output of the interpolation filter module 1100 is then provided to the inter-frame prediction module 1040 as a filtered reference sample for motion compensation operations.

[0096] Figure 12 A conceptual illustration shows a process 1200 for decoding pixel blocks using MMVD. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing decoder 1000 perform process 1200 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing decoder 1000 performs process 1200.

[0097] The decoder receives data (in block 1210) that will be decoded into the current block of the current image. The decoder identifies MMVD candidates with offset positions based on the merge candidates of the current block (in block 1220). The identified MMVD candidates can be MMVD candidates with fractional offset positions (e.g., ±1 / 2, ±1 / 4) or integer offset positions (e.g., ±1, ±2, ±4, ±8). In some embodiments, the identified MMVD candidates are selected by the encoder and sent in the bitstream for encoding and decoding the current block.

[0098] The decoder generates (in block 1230) reference samples of the identified MMVD candidates. In some embodiments, as described with reference to Figures 6A-B, the decoder generates reference samples of the identified MMVD candidates by applying a vertical filter to the source reference samples of the horizontal MMVD candidates (MMVD candidates with horizontal offset positions), and then applying the horizontal filter to the output of the vertical filter. The source reference samples may be stored in the shift register of the vertical filter. In some embodiments, each source reference sample has fewer bits (e.g., 10 bits) than the filtered result of each reference sample (e.g., 16 bits). The decoder reconstructs (in block 1240) the current block using the generated reference samples.

[0099] IV. Example Electronic System

[0100] Many of the aforementioned features and applications are implemented as software processes, which are specified as a set of instructions recorded on a computer-readable storage medium (also known as a computer-readable medium). When these instructions are executed by one or more computing or processing units (e.g., one or more processors, processor cores, or other processing units), they cause the processing unit to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, compact discread-only memory (CD-ROM), flash memory drives, random-access memory (RAM) chips, hard disk drives, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. Computer-readable media do not include carrier waves and electronic signals transmitted via wireless or wired connections.

[0101] In this specification, the term "software" is intended to include firmware residing in read-only memory or an application stored in magnetic memory that can be read into memory for processor processing. Furthermore, in some embodiments, multiple software inventions may be implemented as sub-parts of a larger program while retaining distinct software inventions. In some embodiments, multiple software inventions may also be implemented as separate programs. Finally, any combination of separate programs that collectively implement the software inventions described herein is within the scope of this disclosure. In some embodiments, a software program, when installed to run on one or more electronic systems, defines one or more specific machine implementations that process and perform the operations of the software program.

[0102] Figure 13An electronic system 1300 implementing some embodiments of the present disclosure is conceptually illustrated. The electronic system 1300 may be a computer (e.g., a desktop computer, personal computer, tablet computer, etc.), a telephone, a PDA, or any other type of electronic device. Such an electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 1300 includes a bus 1305, a processing unit 1310, a graphics-processing unit (GPU) 1315, system memory 1320, a network 1325, a read-only memory 1330, a permanent storage device 1335, an input device 1340, and an output device 1345.

[0103] Bus 1305 collectively represents all system, peripheral, and chipset buses of the numerous internal devices that are communicatively connected to electronic system 1300. For example, bus 1305 communicatively connects processing unit 1310 to GPU 1315, read-only memory 1330, system memory 1320, and permanent storage device 1335.

[0104] Processing unit 1310 retrieves instructions to be executed and data to be processed from these various memory units in order to perform the processing of this disclosure. In different embodiments, the processing unit may be a single processor or a multi-core processor. Some instructions are passed to GPU 1315 and executed thereon. GPU 1315 may offload various computations or supplement the image processing provided by processing unit 1310.

[0105] Read-only memory (ROM) 1330 stores static data and instructions used by processing unit 1310 and other modules of the electronic system. On the other hand, permanent storage device 1335 is a read-write storage device. This device is a non-volatile storage unit that stores instructions and data even when the electronic system 1300 is powered off. Some embodiments of this disclosure use mass storage devices (e.g., magnetic disks or optical disks and their corresponding disk drives) as permanent storage device 1335.

[0106] Other embodiments use removable storage devices (e.g., floppy disks, flash memory devices, etc., and their corresponding disk drives) as permanent storage devices. Like permanent storage device 1335, system memory 1320 is a read-write memory device. However, unlike permanent storage device 1335, system memory 1320 is volatile read-write memory, such as random access memory. System memory 1320 stores some instructions and data used by the processor during runtime. In some embodiments, the processing according to this disclosure is stored in system memory 1320, permanent storage device 1335, and / or read-only memory 1330. For example, according to some embodiments of this disclosure, various memory units include instructions for processing multimedia clips. From these various memory units, processing unit 1310 obtains instructions to be executed and data to be processed in order to perform the processing of some embodiments.

[0107] Bus 1305 is also connected to input device 1340 and output device 1345. Input device 1340 enables a user to communicate information and select commands to the electronic system. Input device 1340 includes an alphanumeric keypad and a pointing device (also referred to as a "cursor control device"), a camera (e.g., a webcam), a microphone, or similar devices for receiving voice commands. Output device 1345 displays images or output data generated by the electronic system. Output device 1345 includes printers and display devices, such as cathode ray tubes (CRTs) or liquid crystal displays (LCDs), as well as speakers or similar audio output devices. Some embodiments include devices used as both input and output devices, such as touchscreens.

[0108] Finally, as Figure 13 As shown, bus 1305 also couples electronic system 1300 to network 1325 via a network adapter (not shown). In this way, the computer can be part of a computer network (e.g., a local area network (“LAN”), a wide area network (“WAN”), or an internal network, or a network of multiple networks, such as the Internet. Any or all elements of electronic system 1300 can be used in conjunction with this disclosure.

[0109] Some embodiments include electronic components such as microprocessors, storage devices, and memory that store computer program instructions in a machine-readable or computer-readable medium (or referred to as a computer-readable storage medium, machine-readable medium, or machine-readable storage medium). Examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD cards, mini SD cards, micro SD cards, etc.), magnetic and / or solid-state hard disk drives, and read-only and recordable... Optical discs, high-density optical discs, any other optical or magnetic media, and floppy disks. Computer-readable media may store computer programs that can be executed by at least one processing unit and include a set of instructions for performing various operations. Examples of computer programs or computer code include machine code generated by a compiler, and documents that include high-level code executed by a computer, electronic components, or a microprocessor using an interpreter.

[0110] While the above discussion primarily concerns microprocessors or multi-core processors that execute software, many of the aforementioned features and applications are implemented by one or more integrated circuits, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuit itself. Furthermore, some embodiments execute software stored in a programmable logic device (PLD), ROM, or RAM device.

[0111] As used in this specification and any claim of this application, the terms "computer," "server," "processor," and "memory" refer to electronic or other technical devices. These terms do not include people or groups of people. For the purposes of this specification, the term "display" or "show" means display on an electronic device. As used in this specification and any claim of this application, the terms "computer-readable medium," "machine-readable medium," and "computer-readable medium" are strictly limited to tangible physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals, or any other transient signals.

[0112] Although this disclosure has been described with reference to numerous specific details, those skilled in the art will recognize that this disclosure may be implemented in other particular forms without departing from the spirit of this disclosure. Furthermore, many figures (including...) Figure 9 and Figure 12 This invention provides a conceptual illustration of the processing. The specific operations of these processes may not be performed in the exact order shown and described. The specific operations may not be performed in a continuous series of operations; different specific operations may be performed in different embodiments. Furthermore, the processing may be implemented using several sub-processes or as part of a larger macro-process. Therefore, those skilled in the art will understand that this disclosure is not limited to the foregoing illustrative details but is defined by the appended claims.

[0113] Supplementary Explanation

[0114] The topics described herein sometimes refer to different elements that are contained in or connected to other different elements. It is understood that the described structures are merely examples, and many other structures can actually be implemented to achieve the same function. Conceptually, any arrangement of elements that achieve the same function is actually “associated” in order to achieve the desired function. Therefore, any two elements combined to achieve a specific function, whether structural or intermediate components, are considered “interrelated” to achieve the desired function. Similarly, any two associated elements are considered to be “operably connected” or “operably coupled” to each other to achieve a specific function. Any two elements that can be associated with each other are also considered to be “operably coupled” to each other to achieve a specific function. Any two elements that can be associated with each other are also considered to be “operably coupled” to each other to achieve a specific function. Specific examples of operable connections include, but are not limited to, physically pairable and / or physically interacting elements, and / or wirelessly interactive and / or wirelessly interacting elements, and / or logically interacting and / or logically interactive elements.

[0115] Furthermore, regarding the use of virtually any plural and / or singular terms, those skilled in the art can transform from plural to singular and / or from singular to plural depending on the context and / or application. For clarity, the present invention explicitly describes different singular / plural permutations.

[0116] Furthermore, those skilled in the art will understand that, generally, the terminology used in this invention, particularly in the claims, and the subject matter of the claims, is typically used as "open" terms. For example, "comprising" should be interpreted as "including but not limited to," "having" should be interpreted as "at least," and "comprising" should be interpreted as "including but not limited to," etc. Those skilled in the art will further understand that if a specific number of claims are intended to be described, this will be explicitly stated within the claims, and will not be shown if such content is not present. For example, to aid understanding, the following claims may contain the phrases "at least one" and "one or more" to describe the content of the claims. However, the use of these phrases should not be construed as implying the use of the indefinite articles "one" or "a" to describe the content of the claims, thereby limiting the scope of any particular patent. Even when the same claim includes the introductory phrases "one or more" or "at least one," the indefinite articles, such as "one" or "a," should be interpreted as indicating at least one or more, and the same applies to the use of explicit descriptions to describe the claims. Furthermore, even when a specific number of introductory phrases are explicitly cited, those skilled in the art will recognize that such phrases should be interpreted as indicating the number of citations. For example, "two citations" without further modification means at least two citations, or two or more citations. Additionally, in the use of expressions such as "at least one of A, B, and C," this is generally done so that those skilled in the art can understand the expression. For example, "the system comprises at least one of A, B, and C" will include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc. Those skilled in the art will further understand that any separated words and / or phrases represented by two or more alternative terms, whether in the specification, claims, or drawings, should be understood to include one, one, or both of these terms. For example, "A or B" should be understood as "A," or "B," or "A and B."

[0117] As can be seen from the foregoing, various embodiments of the present invention have been described for illustrative purposes, and various modifications can be made without departing from the scope and spirit of the invention. Therefore, the various embodiments disclosed herein are not intended to be limiting, and the true scope and application are indicated by the claims.

Claims

1. A video encoding method, comprising: Receive raw pixel data of pixel blocks, which will be encoded into the current block of the current image of the video into the bitstream; Based on the merging candidates of the current block, identify multiple merging pattern candidates with motion vector differences at multiple different offset positions; Multiple reference samples are generated from the identified multiple merging pattern candidates with motion vector differences; as well as The current block is encoded into the bitstream using the generated plurality of reference samples. The multiple reference samples for generating the multiple merged pattern candidates with motion vector differences include applying a vertical filter to multiple source reference samples of multiple horizontal merged pattern candidates with motion vector differences and applying a horizontal filter to multiple outputs of the vertical filter.

2. The video encoding method as described in claim 1, characterized in that, The plurality of merging pattern candidates with motion vector differences include a first group of merging pattern candidates with motion vector differences and a second group of merging pattern candidates with motion vector differences. The generation of the plurality of reference samples includes deriving a first group of reference samples and a second group of reference samples. The first group of reference samples is used for the first group of merging pattern candidates with motion vector differences, and the second group of reference samples is used for the second group of merging pattern candidates with motion vector differences.

3. The video encoding method as described in claim 2, characterized in that, Deriving the first set of reference samples includes accessing a plurality of reference samples restricted to a first access range, and deriving the second set of reference samples includes accessing a plurality of reference samples restricted to a second access range, wherein the first access range is different from the second access range.

4. The video encoding method as described in claim 2, characterized in that, The first group of merging pattern candidates with motion vector differences includes two or more merging pattern candidates with vertical motion vector differences, the two or more merging pattern candidates with vertical motion vector differences having multiple vertical offset positions from the merging candidate; and the second group of merging pattern candidates with motion vector differences includes two or more merging pattern candidates with horizontal motion vector differences, the two or more merging pattern candidates with horizontal motion vector differences having multiple horizontal offset positions from the merging candidate.

5. The video encoding method as described in claim 4, characterized in that, The first group of merge mode candidates with motion vector differences includes all vertical merge mode candidates with motion vector differences for encoding and decoding the current block, and the second group of merge mode candidates with motion vector differences includes all horizontal merge mode candidates with motion vector differences for encoding and decoding the current block.

6. The video encoding method as described in claim 1, characterized in that, The multiple source reference samples are stored in the shift register of the vertical filter.

7. The video encoding method as described in claim 1, characterized in that, Each of the plurality of source reference samples has fewer bits than the filtered result of each of the plurality of reference samples.

8. The video encoding method as described in claim 1, characterized in that, At least a portion of the outputs of the horizontal filter are reused for different horizontal filter candidates.

9. The video encoding method as described in claim 1, characterized in that, The identified plurality of merging pattern candidates with motion vector differences include a plurality of merging pattern candidates with motion vector differences having multiple fractional offsets and a plurality of merging pattern candidates with motion vector differences having multiple integer offsets.

10. A video decoding method, comprising: Receive data to be decoded from pixel blocks in the bitstream, which will be decoded into the current block of the current image of the video; Based on the current block, merge candidate identification includes merge pattern candidates with motion vector differences, and the merge pattern candidates with motion vector differences have offset positions. Multiple reference samples are generated for the identified merging pattern candidates with motion vector differences; as well as The current block is reconstructed using the generated reference samples. The generation of the plurality of reference samples for the merged pattern candidate with motion vector difference includes applying a vertical filter to a plurality of source reference samples of the merged pattern candidate with motion vector difference at a horizontal level, and applying a horizontal filter to a plurality of outputs of the vertical filter.

11. The video decoding method as described in claim 10, characterized in that, The multiple source reference samples are stored in the shift register of the vertical filter.

12. The video decoding method as described in claim 10, characterized in that, Each of the plurality of source reference samples has fewer bits than the filtered result of each of the plurality of reference samples.

13. The video decoding method as described in claim 10, characterized in that, The identified merged pattern candidates with motion vector differences have fractional or integer offsets.

14. An electronic device comprising: The encoder circuit is arranged to perform multiple operations, including: Receive raw pixel data of pixel blocks, which will be encoded into the current block of the current image of the video into the bitstream; Based on the merging candidates of the current block, identify multiple merging pattern candidates with motion vector differences at multiple different offset positions; Multiple reference samples are generated from the identified multiple candidate merged patterns with motion vector differences; and The current block is encoded into the bitstream using the generated plurality of reference samples. The generation of the plurality of reference samples for the merged pattern candidate with motion vector difference includes applying a vertical filter to a plurality of source reference samples of the merged pattern candidate with motion vector difference at a horizontal level, and applying a horizontal filter to a plurality of outputs of the vertical filter.

Images