Method for Encoding Video Data

Abstract

A method for encoding video data by an electronic device is provided. Block units are determined from image frames according to the video data. Multiple merge mode with motion vector difference (MMVD) prediction parameters of the block units are selected to predict the block units. A set flag and multiple MMVD indications of the block units are determined. The set flag indicates one of multiple candidate groups corresponding to the block unit. The multiple MMVD indications are used to indicate multiple MMVD prediction parameters of the block unit according to the indicated candidate group. The set flag and the multiple MMVD indications are encoded into a bitstream. Each candidate group includes one or more first MMVD candidate lists each having multiple first MMVD candidate parameters, one or more of the multiple MMVD prediction parameters are selected from the multiple first MMVD candidate parameters in one or more of the first MMVD candidate lists of the indicated candidate group, and MMVD prediction parameters other than the one or more of the multiple MMVD prediction parameters are selected from multiple second candidate parameters in one or more second MMVD candidate lists.

Description

[0001] This application is a divisional application. The application number of the original application is 201980055881.8, the original application date is August 26, 2019, and the entire content of the original application is incorporated herein by reference.

[0002] Cross-reference to related applications

[0003] This application claims the benefit and priority of U.S. Provisional Application No. 62 / 723,880, filed on August 28, 2018, with the invention title Candidate Sets for Ultimate Motion Vector Expression and attorney docket number US74791 (hereinafter referred to as the US74791 application). The disclosure of the US74791 application is hereby incorporated by reference in its entirety into this application. Technical field

[0004] This application generally relates to video coding, and particularly to techniques for predicting block units in an image frame based on multiple candidate parameters in multiple candidate sets. Background art

[0005] Intra prediction is an encoding tool for video coding. In common video coding methods, an encoder and a decoder use previously reconstructed pixels in a reference frame different from the image frame of an encoded block to generate reference pixels and predictors for predicting or reconstructing the encoded block.

[0006] The previously reconstructed pixels can be determined from multiple candidate parameters. When there are a large number of candidate parameters, the high number of certain candidate parameters may result in poor efficiency of image data compression. Therefore, an encoding (and / or decoding) device may need a prediction method that is used to predict an encoded block in an image frame based on candidate parameters distributed in multiple candidate sets. Summary of the invention

[0007] This application is directed to an apparatus and method for reconstructing block units in an image frame based on multiple candidate parameters in multiple candidate sets.

[0008] In a first aspect of the present application, a method for encoding video data by an electronic device is provided. The method includes determining a block unit from an image frame according to the video data; selecting a plurality of merge mode with motion vector difference (MMVD) prediction parameters of the block unit to predict the block unit; determining a set flag and a plurality of MMVD indications of the block unit, the set flag indicating one of a plurality of candidate groups corresponding to the block unit, the plurality of MMVD indications being used to indicate the plurality of MMVD prediction parameters of the block unit according to the indicated candidate group; and encoding the set flag and the plurality of MMVD indications into a bitstream, wherein: each of the plurality of candidate groups includes one or more first MMVD candidate lists, each of the one or more first MMVD candidate lists in the one or more first MMVD candidate lists includes a plurality of first MMVD candidate parameters, one or more of the plurality of MMVD prediction parameters are selected from the plurality of first MMVD candidate parameters in the one or more first MMVD candidate lists of the indicated candidate group, and MMVD prediction parameters other than the one or more of the plurality of MMVD prediction parameters are selected from a plurality of second candidate parameters in one or more second MMVD candidate lists, each of the one or more second MMVD candidate lists in the one or more second MMVD candidate lists is different from the one or more first MMVD candidate lists.

[0009] In a second aspect of the present application, a method for encoding video data by an electronic device is provided. The method includes determining a block unit from an image frame according to the video data; selecting a plurality of merge mode with motion vector difference (MMVD) prediction parameters of the block unit to predict the block unit; determining a set flag indicating one of a plurality of candidate groups, wherein each of the plurality of candidate groups includes one or more first merge mode with motion vector difference (MMVD) candidate lists; determining a set index indicating one or more first MMVD prediction parameters of the MMVD prediction parameters of the block unit from the one or more first MMVD candidate lists included in the indicated candidate group; determining one or more MMVD indexes indicating one or more second MMVD prediction parameters of the MMVD prediction parameters of the block unit from one or more second MMVD candidate lists not included in the plurality of candidate groups; and encoding the set flag, the set index, and the one or more MMVD indexes into a bitstream. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] When read in conjunction Figure 1 with the accompanying

[0011] Figure 1 is a block diagram of an exemplary system configured to encode and decode video data according to an exemplary embodiment of the present application.

[0012] Figure 2 According to an exemplary embodiment of the present application, Figure 1 is a block diagram of an exemplary decoder module of a destination device in a

[0013] Figure 3 is a flowchart of an exemplary reconstruction method for reconstructing a block unit based on multiple candidate parameters in multiple candidate groups according to an exemplary embodiment of the present application.

[0014] Figures 4A - 4C is a schematic diagram of a block unit and multiple search ranges according to an exemplary embodiment of the present application.

[0015] Figure 5 is a flowchart of an exemplary reconstruction method for reconstructing a block unit based on multiple MMVD candidate parameters in multiple candidate groups according to an exemplary embodiment of the present application.

[0016] Figure 6 is a flowchart of an exemplary reconstruction method for reconstructing a block unit based on multiple MMVD candidate parameters in multiple candidate groups according to an exemplary embodiment of the present application.

[0017] Figure 7 According to an exemplary embodiment of the present application, Figure 1 is a block diagram of an exemplary encoder module of a source device in a

[0018] Figure 8 is a flowchart of an exemplary prediction method for predicting a block unit based on multiple MMVD candidate parameters in multiple candidate groups according to an exemplary embodiment of the present application. Detailed Embodiments

[0019] The following description contains specific information related to the exemplary embodiments in the present application. The accompanying drawings and the detailed description thereof in the present application are only exemplary embodiments. However, the present application is not limited to these exemplary embodiments. Those skilled in the art will think of other variations and embodiments of the present application. Unless otherwise specified, the same or corresponding elements in the drawings may be indicated by the same or corresponding reference numerals. In addition, the drawings and illustrations in the present application are generally not drawn to scale and are not intended to correspond to actual relative sizes.

[0020] For purposes of consistency and ease of understanding, like features are designated by reference numerals in the exemplary figures (although not so designated in some instances). However, features in different embodiments may otherwise vary and should not be narrowly construed to be limited to the features shown in the figures.

[0021] The phrase "in one embodiment" or "in some embodiments" as used in the specification may each refer to the same or different one or more of the embodiments. The term "coupled" is defined as being directly or indirectly connected through intervening elements and need not be limited to physical connections. When the term "comprising" is used, it means "including, but not necessarily limited to"; it clearly indicates an open inclusion or members of the recited combination, group, series, and equivalents.

[0022] Furthermore, for purposes of explanation and not limitation, specific details such as functional entities, technologies, protocols, standards, etc. are set forth to provide an understanding of the recited technology. In other instances, well-known methods, technologies, systems, architectures, and equivalents are not described in detail so as not to obscure the description with unnecessary details.

[0023] Those skilled in the art will immediately recognize that any of the encoding functions or algorithms described in this application can be implemented by hardware, software, or a combination of software and hardware. The described functions may correspond to modules that can be software, hardware, firmware, or any combination thereof. Software implementations may include computer-executable instructions stored on a computer-readable medium such as a memory or other type of storage device. For example, one or more microprocessors or general-purpose computers with communication processing capabilities can be programmed with corresponding executable instructions to perform the described network functions or algorithms. The microprocessor or general-purpose computer can be formed by an application specific integrated circuitry (ASIC), a programmable logic array, and / or using one or more digital signal processors (DSPs). Although several exemplary embodiments described in this specification tend to be software installed and executed on computer hardware, alternative exemplary embodiments in firmware or hardware or a combination of hardware and software are also within the scope of this application.

[0024] Computer-readable media include, but are not limited to, random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, compact disc read-only memory (CD ROM), magnetic cassettes, magnetic tapes, magnetic disk storage or any other equivalent medium capable of storing computer-readable instructions.

[0025] Figure 1 is a block diagram of an exemplary system configured to encode and decode video data according to an exemplary embodiment of the present application. In the embodiment, the system includes a source device 11, a destination device 12, and a communication medium 13. In at least one embodiment, the source device 11 may include any device configured to encode video data and send the encoded video data to the communication medium 13. In at least one embodiment, the destination device 12 may include any device configured to receive the encoded video data via the communication medium 13 and decode the encoded video data.

[0026] In at least one embodiment, the source device 11 may communicate with the destination device 12 wired and / or wirelessly via the communication medium 13. The source device 11 may include a source module 111, an encoder module 112, and a first interface 113. The destination device 12 may include a display module 121, a decoder module 122, and a second interface 123. In at least one embodiment, the source device 11 may be a video encoder, and the destination device 12 may be a video decoder.

[0027] In at least one embodiment, the source device 11 and / or the destination device 12 may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, or other electronic devices. Figure 1 Only one example of the source device 11 and the destination device 12 is shown, and in other embodiments, the source device 11 and the destination device 12 may include more or fewer elements than those shown, or have different configurations of various elements.

[0028] In at least one embodiment, the source module 111 of the source device 11 may include a video capture device for capturing new videos, a video archive for storing previously captured videos, and / or a video feed interface for receiving videos from a video content provider. In at least one embodiment, the source module 111 of the source device 11 may generate computer graphics-based data as the source video, or a combination of live video, archived video, and computer-generated video. In at least one embodiment, the video capture device may be a Charge-Coupled Device (CCD) image sensor, a Complementary Metal–Oxide–Semiconductor (CMOS) image sensor, or a camera.

[0029] In at least one embodiment, the encoder module 112 and the decoder module 122 may each be implemented as any of a variety of suitable encoder / decoder circuits, such as one or more microprocessors, a Central Processing Unit (CPU), a Graphic Processing Unit (GPU), a System On Chip (SoC), Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the technology is implemented partially in software, the device may store instructions for the software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in hardware to perform the technology of the present application. In at least one embodiment, each of the encoder module 112 and the decoder module 122 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (CODEC) in the corresponding device.

[0030] In at least one embodiment, the first interface 113 and the second interface 123 may employ a custom protocol or conform to an existing standard or de facto standard. Existing standards or de facto standards include, but are not limited to, Ethernet, IEEE 802.11 or IEEE 802.15 series, Wireless USB, or telecommunications standards. Telecommunications standards include, but are not limited to, GSM, CDMA2000, TD-SCDMA, WiMAX, 3GPP-LTE, or TD-LTE. In at least one embodiment, the first interface 113 and the second interface 123 may each include any device configured to transmit and / or store a compatible video bitstream to and from the communication medium 13. In at least one embodiment, the first interface 113 and the second interface 123 may include a computer system interface that may enable a compatible video bitstream to be stored on or received from a storage device. For example, the first interface 113 and the second interface 123 may include a chipset that supports Peripheral Component Interconnect (PCI) and Peripheral Component Interconnect Express (PCIe) bus protocols, proprietary bus protocols, Universal Serial Bus (USB) protocols, I2C, or other logical and physical structures for interconnecting peer devices.

[0031] In at least one embodiment, although other display technologies may be used in other embodiments, the display module 121 may include a display using Liquid Crystal Display (LCD) technology, Plasma Display technology, Organic Light Emitting Diode (OLED) display technology, or Light Emitting Polymer Display (LPD) technology. In at least one embodiment, the display module 121 may include a high-definition display or an ultra-high-definition display.

[0032] Figure 2 is a block diagram of a decoder module 222 according to an exemplary embodiment of the present application, the decoder module 222 representing Figure 1Exemplary implementation of the decoder module 122 of the destination device 12 in the system. In at least one implementation, the decoder module 222 includes an entropy decoder (e.g., entropy decoding unit 2221), a prediction processor (e.g., prediction processing unit 2222), an inverse quantization / inverse transform processor (e.g., inverse quantization / inverse transform unit 2223), an adder (e.g., first adder 2224), a filter (e.g., filtering unit 2225), and a decoded picture buffer (e.g., decoded picture buffer 2226). In at least one implementation, the prediction processing unit 2222 of the decoder module 222 further includes an intra prediction processor (e.g., intra prediction unit 22221) and an inter prediction processor (e.g., inter prediction unit 22222). In at least one implementation, the decoder module 222 receives a bitstream, decodes the bitstream, and outputs the decoded video.

[0033] In at least one implementation, the entropy decoding unit 2221 may receive Figure 1 from the second interface 123 in a bitstream including a plurality of syntax elements, and perform a parsing operation on the bitstream to extract syntax elements from the bitstream. The entropy decoding unit 2221 may perform entropy decoding on the bitstream to generate quantized transform coefficients, quantization parameters, transform data, motion vectors, intra modes, segmentation information, and other syntax information as part of performing the parsing operation. In at least one implementation, the entropy decoding unit 2221 may perform Context Adaptive Variable Length Coding (CAVLC), Context Adaptive Binary Arithmetic Coding (CABAC), Syntax-based Context-Adaptive Binary Arithmetic Coding (SBAC), Probability Interval Partitioning Entropy (PIPE) coding, or another entropy coding technique to generate quantized transform coefficients. In at least one implementation, the entropy decoding unit 2221 may provide the quantized transform coefficients, quantization parameters, and transform data to the inverse quantization / inverse transform unit 2223, and send the motion vectors, intra modes, segmentation information, and other syntax information to the prediction processing unit 2222.

[0034] In at least one embodiment, the prediction processing unit 2222 may receive syntax elements, such as motion vectors, intra modes, segmentation information, and other syntax information, from the entropy decoding unit 2221. In at least one embodiment, the prediction processing unit 2222 may receive a syntax element including segmentation information and then segment a plurality of image frames according to the segmentation information. In at least one embodiment, each image frame may be segmented into at least one image block according to the segmentation information. At least one image block may include a luminance block for reconstructing a plurality of luminance samples and at least one chrominance block for reconstructing a plurality of chrominance samples. The luminance block and the at least one chrominance block may be further segmented to generate macroblocks, Coding Tree Units (CTUs), coding blocks (CBs), their sub-segmentations, and / or other equivalent coding units.

[0035] In at least one embodiment, during the decoding process, the prediction processing unit 2222 receives prediction data, which includes an intra mode or a motion vector for a current image block of a specific one of the plurality of image frames. The current image block may be one of the luminance block and the at least one chrominance block in the specific image frame.

[0036] In at least one embodiment, the intra prediction unit 22221 may perform intra prediction coding of a current block unit on one or more adjacent blocks in the same frame as the current block unit based on syntax elements related to the intra mode to generate a prediction block. In at least one embodiment, the intra mode may specify the positions of reference samples selected from adjacent blocks within the current frame.

[0037] In at least one embodiment, when reconstructing the chrominance component of the current block by the prediction processing unit 2222, the intra prediction unit 22221 may reconstruct the plurality of chrominance components of the current block unit based on the plurality of luminance components of the current block unit.

[0038] In at least one embodiment, the inter prediction unit 22222 may perform inter prediction coding of a current block unit on one or more blocks in one or more reference image blocks based on syntax elements related to the motion vector to generate a prediction block. In at least one embodiment, the motion vector may indicate the displacement of the current block unit within the current image block relative to a reference block unit within the reference image block. The reference block unit is a block determined to closely match the current block unit. In at least one embodiment, the inter prediction unit 22222 receives a reference image block stored in the decoded picture buffer 2226 and reconstructs the current block unit based on the received reference image block.

[0039] In at least one embodiment, the inverse quantization / inverse transform unit 2223 may use inverse quantization and inverse transform to reconstruct the residual block in the pixel domain. In at least one embodiment, the inverse quantization / inverse transform unit 2223 may apply inverse quantization to the residual quantization transform coefficients to generate residual transform coefficients, and then apply inverse transform to the residual transform coefficients to generate the residual block in the pixel domain. In at least one embodiment, the inverse transform may be used in reverse of the transform process, such as Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Adaptive Multiple Transform (AMT), Mode-Dependent Non-Separable Secondary Transform (MDNSST), Hypercube-Givens Transform (HyGT), F signal-dependent transform, Karhunen-Loéve Transform (KLT), wavelet transform, integer transform, subband transform, or conceptually similar transforms. In at least one embodiment, the inverse transform may convert the residual information from a transform domain such as the frequency domain back to the pixel domain. In at least one embodiment, the degree of inverse quantization may be modified by adjusting the quantization parameter.

[0040] In at least one embodiment, the first adder 2224 adds the reconstructed residual block to the prediction block provided by the prediction processing unit 2222 to generate a reconstructed block.

[0041] In at least one embodiment, the filtering unit 2225 may include a deblocking filter, a Sample Adaptive Offset (SAO) filter, a bilateral filter, and / or an Adaptive Loop Filter (ALF) to remove blocking artifacts from the reconstructed block. In addition to the deblocking filter, SAO filter, bilateral filter, and ALF, other filters (either in-loop or post-loop) may also be used. For the sake of brevity, such filters are not shown, but if needed, the output of the first adder 2224 may be filtered. In at least one embodiment, after performing the filtering process on the reconstructed block of a specific image frame, the filtering unit 2225 outputs the decoded video to the display module 121 or other video receiving units.

[0042] In at least one embodiment, the decoded picture buffer 2226 may be a reference picture memory that stores, for example, reference blocks used by the prediction processing unit 2222 to decode a bitstream in an inter-coded mode. The decoded picture buffer 2226 may be formed of any of a variety of storage devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magneto-resistive RAM (MRAM), resistive RAM (RRAM), or other types of storage devices. In at least one embodiment, the decoded picture buffer 2226 may be on-chip with other elements of the decoder module 222 or off-chip relative to those elements.

[0043] Figure 3 FIG. shows a flowchart of an exemplary reconstruction method for reconstructing a block unit based on a plurality of candidate parameters in a plurality of candidate groups according to an exemplary embodiment of the present application. Since there are multiple ways to perform this method, an example method is provided only by way of example. For example, the methods described below may be performed using the configurations shown in Figure 1 and Figure 2 and various elements of these figures are referred to in the explanation of the example method. Each block shown in Figure 3 may represent one or more processes, methods, or subroutines performed in the example method. Additionally, the order of the blocks is illustrative only and may be changed. Additional blocks may be added or fewer blocks may be utilized without departing from the present application.

[0044] At block 31, the decoder module 222 may determine a block unit from an image frame according to video data and determine a plurality of candidate parameters for the block unit.

[0045] In at least one embodiment, the video data may be a bitstream. In at least one embodiment, the destination device 12 may receive a bitstream from an encoder such as the source device 11 through the second interface 123 of the destination device 12. The second interface 123 may provide the bitstream to the decoder module 222. The decoder module 222 may determine an image frame according to the bitstream and segment the image frame according to a plurality of segmentation indications in the bitstream to determine a block unit. For example, the decoder module 222 may segment the image frame to generate a plurality of coding tree units and further segment one of the coding tree units based on the segmentation indication in a video coding standard to determine a block unit.

[0046] In at least one embodiment, the source device 11 may determine the multiple candidate parameters for the prediction mode of the block unit, and select multiple prediction parameters from the multiple candidate parameters to predict the block unit. In at least one embodiment, the prediction mode may be a mode of merge mode with motion vector difference (MMVD) selected from multiple candidate modes. In the embodiment, the source device 11 may provide multiple prediction indications to the target device 12, and the multiple prediction indications indicate the multiple prediction parameters of the block unit in the bitstream. In at least one embodiment, the entropy decoding unit 2221 may decode the bitstream to determine the multiple prediction indications of the block unit, and then the decoder module 222 may further reconstruct the block unit based on the multiple prediction indications. In at least one embodiment, the multiple prediction indications may include multiple flags and multiple indexes.

[0047] In at least one embodiment, when the prediction mode is the MMVD mode, the prediction mode may include multiple motion parameters. In at least one embodiment, the multiple motion parameters may include a base motion, a motion difference, and a prediction direction. In at least one embodiment, each motion parameter has multiple candidate parameters. In at least one embodiment, the multiple prediction indications may include at least one of a base candidate index, a motion difference indication, and a prediction direction index for the MMVD mode. In at least one embodiment, the prediction parameter for each motion parameter may be selected based on the corresponding one of the multiple prediction indications. For example, the prediction parameter of the base motion may be selected from the multiple candidate parameters of the base motion based on the base candidate index.

[0048] In at least one embodiment, the inter-frame prediction unit 22222 may generate a plurality of base candidates for the block unit based on neighboring motion information. In at least one embodiment, a base candidate is one of the plurality of candidate parameters for selecting a base motion. In at least one embodiment, the base candidate index may be the MMVD flag mmvd_cand_flag, which indicates the selected base candidate included in the prediction parameters. For example, the base candidates may be generated in skip mode, direct mode, and merge mode based on video coding standards such as high efficiency video coding (HEVC) or versatile video coding (VVC). Each base candidate may include at least one frame candidate and at least one vector candidate. In one embodiment, when a specific one of the plurality of base candidates is a bi-prediction candidate, the specific base candidate may include a first frame candidate in the first reference list L0, a first vector candidate corresponding to the first frame candidate, a second frame candidate in the second reference list L1, and a second vector candidate corresponding to the second frame candidate. In another embodiment, when the specific base candidate is a uni-prediction candidate for the first reference list L0, the specific base candidate may include the first frame candidate in the first reference list L0 and the first vector candidate corresponding to the first frame candidate. In other embodiments, when the specific base candidate is a uni-prediction candidate for the second reference list L1, the specific base candidate may include the second frame candidate in the second reference list L1 and the second vector candidate corresponding to the second frame candidate.

[0049] In at least one embodiment, the motion difference indication may further include a motion distance index and a motion direction index. In at least one embodiment, the inter-frame prediction unit 22222 may select one of the plurality of base candidates based on the base candidate index and determine a motion difference based on the motion difference indication to adjust the vector candidate in the selected base candidate. In one embodiment, the motion difference may be represented by a difference magnitude and a difference direction determined based on the motion distance index and the motion direction index. In at least one embodiment, the motion distance index may be the first MMVD index mmvd_distance_idx indicating the difference magnitude included in the plurality of prediction parameters, and the motion direction index may be the second MMVD index mmvd_direction_idx indicating the difference direction included in the plurality of prediction parameters. In at least one embodiment, the plurality of difference distance candidates may be the plurality of candidate parameters for selecting the difference magnitude based on the motion distance index, and the plurality of difference direction candidates may be the plurality of candidate parameters for selecting the difference direction based on the motion direction index.

[0050] In at least one embodiment, the difference direction may indicate a direction angle for adjusting a vector candidate determined based on the base candidate index. In one embodiment, each direction angle may be predefined by the motion direction index. For example, the direction angles may include 0°, 22.5°, 37.5°, 150° or any other angle. In at least one embodiment, the motion direction index may indicate one of four coordinate directions (+, 0), (0, +), (-, 0) and (0, -). In this embodiment, the direction angles of the four coordinate directions may be 0°, 90°, 180° and 270°. In one embodiment, the motion difference indication may be the motion difference index indicating a motion difference including an X coordinate difference and a Y coordinate difference. Thus, the inter-frame prediction unit 22222 may determine a motion difference based on the X coordinate difference and the Y coordinate difference indicated by the motion difference index to adjust the selected vector candidate.

[0051] In at least one embodiment, the inter-frame prediction unit 22222 may select at least one of a first reference list L0 and a second reference list L1 from a plurality of prediction direction candidates based on the prediction direction index. When the block unit is predicted based on one of the plurality of first frame candidates in the first reference list L0 and one of the plurality of second frame candidates in the second reference list L1, the block unit is a bi-directional prediction block based on the first reference list prediction L0 and the second reference list L1. When the block unit is predicted based on one of the plurality of first frame candidates in the first reference list L0, the block unit is a uni-directional prediction block predicted based on the first reference list L0. When the block unit is predicted based on one of the plurality of second frame candidates in the second reference list L1, the block unit is a uni-directional prediction block predicted based on the second reference list L1. In the embodiment, the inter-frame prediction unit 22222 may check the relationship between the base candidate index and the prediction direction index. When the reference list determined by the prediction direction index is different from the reference list determined by the base candidate index, the inter-frame prediction unit 22222 may search for a frame candidate in the reference list determined by the prediction direction index by mirroring the frame candidate in the reference list determined by the base candidate index. For example, when the prediction direction index indicates that the block unit is bi-directional prediction and the base candidate index indicates a uni-directional prediction candidate, there is no corresponding frame of the block unit in the second reference list L1. Therefore, the inter-frame prediction unit 22222 may mirror the first frame candidate in the first reference list L0 determined by the base candidate index into the second reference list L1 to select one of the second frame candidates. Additionally, the inter-frame prediction unit 22222 may mirror the first vector candidate corresponding to the first reference list L0 determined by the base candidate index to determine the second vector candidate for the second reference list L1. In at least one embodiment, there may be no prediction direction index among the plurality of prediction indications. In the embodiment, the reference list determined by the base candidate index may be the reference list of the block unit.

[0052] In at least one embodiment, the plurality of prediction indicators may include a set index. In this embodiment, when the motion difference indicator is the motion difference index, the set index may represent at least one of the base candidate index, the motion difference index, and the prediction direction index. In at least one embodiment, when the motion difference indicator includes the motion distance index and the motion direction index, the set index may represent at least one of the base candidate index, the motion distance index, the motion direction index, and the prediction direction index. For example, when the set index represents the motion distance index and the motion direction index, the decoder module 222 may directly determine the difference magnitude and the difference direction according to the set index. Additionally, when the set index only represents the motion distance index, the decoder module 222 may determine the difference magnitude based on the set index.

[0053] Referring Figure 3 , at block 32, the decoder module 222 may determine a plurality of candidate groups, each candidate group including a plurality of group parameters selected from the plurality of candidate parameters.

[0054] In at least one embodiment, the inter-frame prediction unit 22222 may determine a plurality of previous blocks from the bitstream. In this embodiment, a plurality of previous blocks reconstructed from a plurality of previous patterns selected from the plurality of candidate patterns are decoded before the block unit. In at least one embodiment, when the plurality of previous patterns are MMVD patterns, the plurality of previous patterns may include a plurality of previous parameters selected from the plurality of candidate parameters. Thus, the inter-frame prediction unit 22222 may determine the plurality of previous patterns of the plurality of previous blocks, and classify a plurality of candidate parameters that are the same as or similar to the plurality of previous parameters of the prediction pattern into a first candidate group. For example, when the inter-frame prediction unit 22222 uses the set index to represent the difference magnitude and the difference direction, the inter-frame prediction unit 22222 may compare the plurality of difference distance candidates and the plurality of difference direction candidates among the plurality of candidate parameters with the plurality of difference magnitudes and the plurality of difference directions of the plurality of previous patterns of the plurality of previous blocks. Additionally, when the inter-frame prediction unit 22222 uses the set index to represent the difference magnitude, the inter-frame prediction unit 22222 may compare the plurality of difference distance candidates among the plurality of candidate parameters with the plurality of difference magnitudes of the plurality of previous patterns of the plurality of previous blocks.

[0055] In at least one embodiment, the decoder module 222 may divide the image frame according to the segmentation indication in the bitstream to determine the block unit, and determine the search range based on a plurality of predetermined regions. Figures 4A - 4CSchematic diagram of an exemplary embodiment of block units 41-43 and search ranges 410-430. In one embodiment, Figure 4A The search range 410 of the block unit 41 in includes a first search area 411 located to the left of the block unit 41, a second search area 412 located above the block unit 41, and a third search area 413 located in the upper left of the block unit 41. The first search area 411 has a first area width W1 and a first area height H1, and the second search area 412 has a second area width W2 and a second area height H2. The area width of the third search area 413 may be equal to the first area width W1, and the area height of the third search area 413 may be equal to the second area height H2. In one embodiment, each of the first area width W1, the second area width W2, the first area height H1, and the second area height H2 may be different from each other. In another embodiment, at least two of the first area width W1, the second area width W2, the first area height H1, and the second area height H2 may be equal to each other. For example, in Figure 4B the first area width W1 of the first search area 421 of the block unit 42 in may be equal to the second area height H2 of the second search area 422 of the block unit 42. As a result, the third search area 423 of the block unit 42 may be a square search area. In one embodiment, each of the first area width W1, the second area width W2, the first area height H1, and the second area height H2 may be divisible by four. In one embodiment, each of the first area width W1, the second area width W2, the first area height H1, and the second area height H2 may be predefined as a preset length. In another embodiment, the encoder module 112 of the source device 11 may determine the lengths of the first area width W1, the second area width W2, the first area height H1, and the second area height H2, and provide at least one prediction indication in one of the sequence parameter set (SPS) and the picture parameter set (PPS) in the bitstream. Therefore, the decoder module 222 may determine the lengths of the first area width W1, the second area width W2, the first area height H1, and the second area height H2 according to one of the SPS and PPS in the bitstream. In other embodiments, the first area width W1, the second area width W2, the first area height H1, and the second area height H2 may be proportional to the block width W0 and the block height H0 of the block unit 41. Therefore, the inter-frame prediction unit 22222 may determine the lengths of the first area width W1, the second area width W2, the first area height H1, and the second area height H2 based on a plurality of predetermined proportional parameters.

[0056] In at least one embodiment, Figure 4CThe search range 430 of the block unit 43 in [description] can include a plurality of search blocks adjacent to the block unit 43. For example, the search range 430 can include a first search block 431, a second search block 432, a third search block 433, a fourth search block 434, and a fifth search block 435. In one embodiment, the search blocks 431-435 can be the same as a plurality of spatial blocks that are adjacent to the block unit 41 and are used by the inter-frame prediction unit 22222 for a plurality of inter-frame merge candidates. In one embodiment, the search block width and search block height of the search blocks can be equal to four.

[0057] In at least one embodiment, a previously decoded slice can be defined as the search range. In this embodiment, each of the plurality of previous blocks in the previously decoded slice is decoded before the block unit. In this embodiment, the previous blocks reconstructed by a plurality of prediction modes selected from the plurality of prediction candidates are decoded before the block unit.

[0058] In at least one embodiment, the inter-frame prediction unit 22222 can determine the plurality of previous modes of the plurality of previous blocks in the search range. In at least one embodiment, the inter-frame prediction unit 22222 can divide the search range to generate a plurality of sub-blocks. In one embodiment, each sub-block can have a block size equal to 4×4. In this embodiment, each sub-block can be included in one of the plurality of previous blocks. When a specific one of the plurality of sub-blocks is included in a specific one of the plurality of previous blocks, the inter-frame prediction unit 22222 can determine that the previous mode of the specific sub-block is the same as the previous mode of the specific previous block. Thus, the inter-frame prediction unit 22222 can determine the plurality of previous modes of the plurality of sub-blocks and calculate the number of the plurality of sub-blocks predicted based on the plurality of candidate parameters. Thus, the inter-frame prediction unit 22222 can determine the usage rate of the plurality of candidate parameters. In at least one embodiment, the inter-frame prediction unit 22222 can arrange the plurality of candidate parameters based on the plurality of usage rates. In this embodiment, the inter-frame prediction unit 22222 can arrange the plurality of candidate parameters from the largest to the smallest based on the plurality of usage rates.

[0059] In at least one embodiment, the inter-frame prediction unit 22222 may arrange the plurality of candidate parameters for each of the plurality of motion parameters according to the plurality of previous modes of the plurality of previous blocks to generate a sorted candidate list. Then, the inter-frame prediction unit 22222 may add the first Nm candidate parameters to the first candidate group as one of the motion parameters in the sorted candidate list. In at least one embodiment, Nm may be an integer greater than zero. In at least one embodiment, the inter-frame prediction unit 22222 may arrange the plurality of candidate parameters for one of the plurality of motion parameters according to the usage rate of the plurality of previous modes to generate a sorted candidate list. Additionally, the number of bits for the first of the plurality of group parameters in the first candidate group in the bitstream may be less than the number of bits for the second of the plurality of group parameters in the first candidate group in the bitstream.

[0060] In at least one embodiment, the maximum number of the plurality of group parameters in the first candidate group may be Nm. In one embodiment, when the decoder module 222 starts decoding the bitstream, the number of the plurality of group parameters in the first candidate group may be equal to zero. When the inter-frame prediction unit 22222 determines that the first block in the video data is predicted based on the MMVD mode having the plurality of previous parameters, the inter-frame prediction unit 22222 may add the candidate parameters identical to the plurality of previous parameters of the MMVD mode to the first candidate group. Thus, the number of the plurality of group parameters in the first candidate group may increase to one. In at least one embodiment, when the number of the plurality of previous modes is equal to or greater than Nm, the number of the plurality of group parameters in the first candidate group may be equal to the maximum number Nm. Therefore, when a new previous mode is determined to be added to the Nm group parameters in the first candidate group, the inter-frame prediction unit 22222 may remove the first added candidate parameter from the plurality of candidate parameters in the first candidate group.

[0061] In at least one embodiment, the inter-frame prediction unit 22222 may classify the remaining candidate parameters into other candidate groups. In one embodiment, the number of the plurality of group parameters in the first candidate group may be greater than or less than the number of the plurality of group parameters in the other candidate groups. In another embodiment, the number of the plurality of group parameters in the first candidate group may be equal to the number of the plurality of group parameters in the other candidate groups.

[0062] In at least one embodiment, the distribution of the plurality of candidate parameters in the plurality of candidate groups may be predefined (e.g., in the encoder module 112 and the decoder module 222). Thus, the inter-frame prediction unit 22222 may directly determine the classification result of the plurality of candidate parameters in the plurality of candidate groups from the decoder module 222. In at least one embodiment, the inter-frame prediction unit 22222 may directly determine the plurality of candidate groups including the plurality of group parameters. In this embodiment, the plurality of group parameters in each candidate group may be predefined in the decoder module 222.

[0063] Referring Figure 3 , at block 33, the decoder module 222 may determine whether a particular one of the plurality of candidate parameters for predicting the block unit is included in the first candidate group. If the particular candidate parameter of the block unit is included in the first candidate group, the program may proceed to block 34. If the particular candidate parameter of the block unit is not included in the first candidate group, the program may proceed to block 35.

[0064] In at least one embodiment, the prediction indication may further include a set flag that indicates whether the particular candidate parameter of the block unit is included in the first candidate group. Thus, when the set flag is equal to 1, the inter-frame prediction unit 22222 may directly determine that the particular candidate parameter of the block unit is included in the first candidate group. Additionally, when the set flag is equal to zero, the inter-frame prediction unit 22222 may directly determine that the particular candidate parameter of the block unit is not included in the first candidate group. In at least one embodiment, the set flag may be fpel_mmvd_enable_flag to indicate whether the particular candidate parameter of the block unit is included in the first candidate group. In at least one embodiment, when the set flag is included in the slice header, the set flag may be slice_fpel_mmvd_enabled_flag.

[0065] At block 34, the decoder module 222 may determine the plurality of prediction parameters based on the particular candidate parameter corresponding to the set index.

[0066] In at least one embodiment, each group parameter in the first candidate group may correspond to an index value of the set index. Thus, since the particular candidate parameter is included in the first candidate group, the inter-frame prediction unit 22222 may determine the particular candidate parameter based on the set index.

[0067] In at least one embodiment, the inter-frame prediction unit 22222 may determine the plurality of prediction parameters based on a set index. For example, when the set index represents a base candidate index and a motion difference index, the inter-frame prediction unit 22222 may directly determine the base candidate and the motion difference of a block unit from the set index. Additionally, the inter-frame prediction unit 22222 may also determine whether a block unit is a bi-directional prediction block or a uni-directional prediction block based on a prediction direction index. Thus, the decoder module 222 may select one of the plurality of vector candidates and one of the plurality of frame candidates based on the determined base candidate, and adjust the selected vector candidate to generate a motion vector corresponding to the selected frame candidate. Then, the decoder module 222 may determine whether to mirror the motion vector to generate a mirrored vector based on the selected frame candidate and the prediction direction index. In at least one embodiment, when the set index represents a motion distance index, the inter-frame prediction unit 22222 may determine the difference magnitude of a block unit from the set index. Thus, the inter-frame prediction unit 22222 may further determine the base candidate and the difference direction based on the base candidate index and the motion direction index, to predict the block unit based on the determined base candidate, the determined difference magnitude, and the determined difference direction.

[0068] In block 35, the decoder module 222 may determine the plurality of prediction parameters based on the specific candidate parameters in the remaining candidate group.

[0069] In at least one embodiment, each group parameter in the remaining candidate group may correspond to an index value of a group index. Thus, when the specific candidate parameter is included in the remaining candidate group, the inter-frame prediction unit 22222 may determine the specific candidate parameter through the group index. In at least one embodiment, all candidate parameters excluded in the first candidate group may be added to the remaining candidate group. In at least one embodiment, the plurality of candidate parameters represented by the set index for the first candidate group may be the same as the candidate parameters represented by the group index for each of the remaining candidate groups. For example, when the set index indicates the base candidate and the motion difference in the first candidate group, the inter-frame prediction unit 22222 may determine that the group index may indicate the base candidate and the motion difference in the remaining candidate group. Thus, the decoder module 222 may select one of the plurality of vector candidates and one of the plurality of frame candidates based on the determined base candidate (e.g., determined by the group index), determine the motion difference determined by the group index, and adjust the selected vector candidate based on the determined motion difference to generate a motion vector corresponding to the selected frame candidate. In at least one embodiment, when the group index only indicates the difference magnitude in the first candidate group, the group index may indicate the difference magnitude in the remaining candidate group.

[0070] In at least one embodiment, the number of candidate groups may be greater than two. For example, the number of candidate groups may be equal to four. The inter-frame prediction unit 22222 may classify the plurality of candidate parameters into four candidate groups based on the plurality of prediction patterns of the plurality of previous blocks. In this embodiment, the set flag may be changed to an index indicating which candidate group includes the specific candidate parameter.

[0071] In at least one embodiment, when the number of candidate groups is equal to two, the set index and the group index may be represented by the same index. In at least one embodiment, when the specific candidate parameter is included in the first candidate group, one of the plurality of prediction indications may be regarded as a set index to determine at least one of the plurality of prediction parameters. In at least one embodiment, when the specific candidate parameter is included in another candidate group, one of the plurality of prediction indications may be regarded as a group index to determine at least one of the prediction parameters.

[0072] In at least one embodiment, the inter-frame prediction unit 22222 may determine the specific candidate parameter in the remaining candidate groups without a group index. The inter-frame prediction unit 22222 may determine a base candidate through a base candidate index, and may determine a motion difference through a motion difference indication. Additionally, the inter-frame prediction unit 22222 may also determine whether a block unit is a bi-directional prediction block or a uni-directional prediction block based on a prediction direction index. Therefore, the decoder module 222 may select one of the plurality of vector candidates and one of the plurality of frame candidates based on the determined base candidate, and adjust the selected vector candidate to generate a motion vector corresponding to the selected frame candidate. Additionally, the decoder module 222 may determine whether to mirror the motion vector to generate a mirror vector based on the selected frame candidate and the prediction direction index, and mirror the selected frame candidate to generate a mirrored frame candidate.

[0073] Referring to Figure 3 , at block 36, the decoder module 222 may reconstruct the block unit based on the determined prediction parameters.

[0074] In at least one embodiment, the inter-frame prediction unit 22222 may generate a prediction component for one of the plurality of block components in a block unit according to the plurality of prediction parameters. In at least one embodiment, the first adder 2224 may generate a plurality of reconstructed components based on the plurality of prediction components and a plurality of residual components for reconstructing the block unit. In the embodiment, the first adder 2224 may receive the plurality of residual components of the block unit via the entropy decoding unit 2221 and the inverse quantization / inverse transformation unit 2223. In the embodiment, the plurality of residual components may be determined from a bitstream. In at least one embodiment, the decoder module 222 may reconstruct all other block units in the image frame to reconstruct the image frame and the video.

[0075] Figure 5 FIG. is a flowchart of an exemplary reconstruction method for reconstructing a block unit based on a plurality of candidate parameters of merge mode with motion vector difference (MMVD) in a plurality of candidate groups according to an exemplary embodiment of the present application. Since there are multiple ways to perform this method, only an example method is provided by way of example. For example, the following-described method may be performed using the configurations shown in Figure 1 and Figure 2 and various elements of these figures are referred to in the explanation of the example method. Each block shown in Figure 5 may represent one or more processes, methods, or subroutines performed in the example method. Additionally, the order of the blocks is illustrative only and may be changed. Additional blocks may be added or fewer blocks may be utilized without departing from the present application.

[0076] In block 51, the decoder module 222 may determine a block unit from an image frame according to video data.

[0077] In at least one embodiment, the video data may be a bitstream. In at least one embodiment, the destination device 12 may receive the bitstream from an encoder such as the source device 11 through the second interface 123 of the destination device 12. The second interface 123 may provide the bitstream to the decoder module 222. The decoder module 222 may determine an image frame according to the bitstream and segment the image frame according to a plurality of segmentation indications in the bitstream to determine a block unit. For example, the decoder module 222 may segment the image frame to generate a plurality of coding tree units and further segment one of the coding tree units according to the segmentation indication in the video coding standard to determine a block unit.

[0078] In at least one embodiment, the entropy decoding unit 2221 may decode the bitstream to determine a plurality of prediction indications for a block unit, and then the decoder module 222 may further reconstruct the block unit based on the prediction indications. In at least one embodiment, the prediction indications may include a plurality of flags and a plurality of indexes.

[0079] At block 52, the prediction processing unit 2222 may select one of a plurality of candidate groups corresponding to a set flag.

[0080] In at least one embodiment, when a block unit is predicted in the MMVD mode, there may be a plurality of motion parameters for predicting the block unit. In at least one embodiment, the plurality of motion parameters may include a base motion, a motion difference, and a prediction direction. In at least one embodiment, each of the plurality of motion parameters may include a plurality of candidate parameters. In at least one embodiment, the decoder module 222 may select a prediction parameter from the plurality of candidate parameters for each of the plurality of motion parameters. For example, one of the plurality of prediction parameters may be selected from the plurality of candidate parameters of the base motion.

[0081] In at least one embodiment, the inter-frame prediction unit 22222 may generate a plurality of base candidates for a block unit based on adjacent motion information. In at least one embodiment, the plurality of base candidates may be the plurality of candidate parameters for selecting a base motion. For example, the base candidates may be generated in a skip mode, a direct mode, and a merge mode based on video coding standards such as high efficiency video coding (HEVC) or versatile video coding (VVC). Each base candidate may include at least one frame candidate and at least one vector candidate. In one embodiment, when a particular one of the plurality of base candidates is a bi-directional prediction candidate, the particular base candidate may include a first frame candidate in a first reference list L0, a first vector candidate corresponding to the first frame candidate, a second frame candidate in a second reference list L1, and a second vector candidate corresponding to the second frame candidate. In another embodiment, when the particular base candidate is a uni-directional prediction candidate for the first reference list, the particular base candidate may include a first frame candidate in the first reference list L0 and a first vector candidate corresponding to the first frame candidate.

[0082] In at least one embodiment, the motion difference may further include a difference magnitude and a difference direction. In at least one embodiment, the inter-frame prediction unit 22222 may select one of the plurality of basic candidates and determine a motion difference to adjust a vector candidate in the selected basic candidate. In at least one embodiment, the plurality of difference distance candidates are the plurality of candidate parameters for selecting the difference magnitude, and the plurality of difference direction candidates are the plurality of candidate parameters for selecting the difference direction. In at least one embodiment, the difference direction may indicate a direction angle for adjusting the vector candidate of the selected basic candidate. In at least one embodiment, the plurality of difference direction candidates may include four coordinate directions (+, 0), (0, +), (-, 0), and (0, -). In the embodiment, the direction angles of the four coordinate directions may be 0°, 90°, 180°, and 270°.

[0083] In at least one embodiment, the inter-frame prediction unit 22222 may select at least one of a first reference list L0 and a second reference list L1 to predict a block unit. When the block unit is predicted based on one of the plurality of first frame candidates in the first reference list L0 and one of the plurality of second frame candidates in the second reference list L1, the block unit is a bi-directional prediction block predicted based on the first reference list L0 and the second reference list L1. When the block unit is predicted based on one of the plurality of first frame candidates in the first reference list L0, the block unit is a uni-directional prediction block predicted based on the first reference list L0. When the reference list for predicting the block unit is different from the reference list determined by the basic motion, the inter-frame prediction unit 22222 may search for a frame candidate in the reference list of the block unit by mirroring the frame candidate in the reference list determined by the basic motion.

[0084] In at least one embodiment, the plurality of candidate parameters of the plurality of motion parameters may include a plurality of first MMVD candidate parameters for selecting one or more first motion parameters and a plurality of second MMVD candidate parameters for selecting one or more second motion parameters. In at least one embodiment, the plurality of first MMVD candidate parameters among the one or more first motion parameters may be distributed among the plurality of candidate groups. In at least one embodiment, each of the plurality of candidate groups includes one or more first MMVD candidate lists, and each first MMVD candidate list includes first MMVD candidate parameters for predicting a respective one of the first motion parameters. In at least one embodiment, the number of the one or more first motion parameters may be equal to the number of the one or more first MMVD candidate lists in each of the plurality of candidate groups. In at least one embodiment, the plurality of second MMVD candidate parameters may be included in one or more second MMVD candidate lists different from the plurality of first MMVD candidate lists. In at least one embodiment, the second MMVD candidate parameters may be excluded from the candidate groups.

[0085] In at least one embodiment, the distribution of the plurality of candidate parameters among the plurality of candidate groups may be predefined (e.g., in the encoder module 112 and the decoder module 222). Thus, the inter-frame prediction unit 22222 may directly determine the classification result of the plurality of first MMVD candidate parameters in the plurality of candidate groups from the decoder module 222. In at least one embodiment, the inter-frame prediction unit 22222 may directly determine the plurality of candidate groups including the plurality of first sets of MMVD candidate parameters. In the embodiment, the plurality of first MMVD candidate parameters in each of the plurality of candidate groups may be predefined in the decoder module 222.

[0086] In at least one embodiment, the plurality of prediction indications may further include a set flag indicating whether at least one of the plurality of prediction parameters of the block unit is selected from a specific one of the plurality of candidate groups. Thus, when the set flag is equal to 1, the inter prediction unit 22222 may directly determine that at least one of the plurality of prediction parameters of the block unit is selected from the specific candidate group. Additionally, when the set flag is equal to zero, the inter prediction unit 22222 may directly determine that at least one of the plurality of prediction parameters of the block unit is not selected from the specific candidate group. In at least one embodiment, the set flag may be fpel_mmvd_enable_flag to indicate whether at least one of the plurality of prediction parameters of the block unit is selected from the specific candidate group. In at least one embodiment, when the set flag is included in the slice header, the set flag may be slice_fpel_mmvd_enabled_flag.

[0087] Referring Figure 5 , in block 53, the prediction processing unit 2222 may determine a plurality of MMVD indications of the block unit according to the video data.

[0088] In at least one embodiment, the plurality of prediction indications may include at least one of a base candidate index of the MMVD mode, a motion difference indication, and a prediction direction index. In at least one embodiment, the decoder module 222 may select one of the plurality of prediction parameters from the plurality of candidate parameters for each of the plurality of motion parameters based on a corresponding one of the plurality of prediction indications. For example, one of the plurality of prediction parameters may be selected from the plurality of candidate parameters of the base motion based on the base candidate index.

[0089] In at least one embodiment, the base candidate index may be the MMVD flag mmvd_cand_flag, which indicates the selected base candidate included in the plurality of prediction parameters. In one embodiment, when the base motion of the block unit is a bi - prediction candidate selected from the plurality of base candidates based on the base candidate index, the base motion of the block unit may indicate a first frame candidate in the first reference list L0, a first vector candidate corresponding to the first frame candidate, a second frame candidate in the second reference list L1, and a second vector candidate corresponding to the second frame candidate.

[0090] In at least one embodiment, the motion difference indication may further include a motion distance index and a motion direction index. In at least one embodiment, the inter-frame prediction unit 22222 may determine a motion difference based on the motion difference indication to adjust a vector candidate in the selected base candidate. In at least one embodiment, the motion distance index may be a first MMVD index mmvd_distance_idx indicating a difference magnitude from among the plurality of difference distance candidates, and the motion direction index may be a second MMVD index mmvd_direction_idx indicating a difference direction from among the plurality of difference direction candidates. In at least one embodiment, the motion difference indication may be a motion difference index indicating a motion difference including an X coordinate difference and a Y coordinate difference. Accordingly, the inter-frame prediction unit 22222 may determine a motion difference based on the X coordinate difference and the Y coordinate difference indicated by the motion difference index to adjust the selected vector candidate.

[0091] In at least one embodiment, the inter-frame prediction unit 22222 may select at least one of a first reference list L0 and a second reference list L1 from among a plurality of prediction direction candidates based on a prediction direction index. When the reference list determined by the prediction direction index is different from the reference list determined by the base candidate index, the inter-frame prediction unit 22222 may search for a frame candidate in the reference list determined by the prediction direction index by mirroring a frame candidate in the reference list determined by the base candidate index. In at least one embodiment, there may be no prediction direction index in the prediction indication. In this embodiment, the reference list determined by the base candidate index is preset as the selected reference list for predicting the block unit.

[0092] In at least one embodiment, the plurality of prediction indicators may include a set index. In this embodiment, when the motion difference indicator is the motion difference index, the set index may represent at least one of the base candidate index, the motion difference index, and the prediction direction index. In at least one embodiment, when the motion difference indicator includes the motion distance index and the motion direction index, the set index may represent at least one of the base candidate index, the motion distance index, the motion direction index, and the prediction direction index. For example, when the set index represents the motion distance index and the motion direction index, the decoder module 222 may directly determine the difference magnitude and the difference direction according to the set index. Additionally, when the set index only represents the motion distance index, the decoder module 222 may determine the difference magnitude based on the set index. In this embodiment, the set index is the motion distance index. In at least one embodiment, when the set index represents the motion distance index and the motion direction index, the plurality of difference distance candidates and the plurality of difference direction candidates may be the plurality of group parameters distributed in the plurality of candidate groups. In this embodiment, the plurality of base candidates and the plurality of prediction direction candidates are not included in the plurality of candidate groups. In at least one embodiment, when the set index represents the motion distance index, the plurality of difference distance candidates may be the plurality of group parameters distributed in the plurality of candidate groups. In this embodiment, the plurality of base candidates, the plurality of difference direction candidates, and the plurality of prediction direction candidates are not included in the plurality of candidate groups.

[0093] In at least one embodiment, when the set flag is equal to 1, the inter-frame prediction unit 22222 may directly determine at least one of the plurality of prediction parameters from the specific candidate group based on the set index. Additionally, when the set flag is equal to zero, the inter-frame prediction unit 22222 may directly determine at least one of the plurality of prediction parameters in other candidate groups based on the group index. In at least one embodiment, when the number of the plurality of candidate groups is equal to 2, the set index and the group index may be represented by the same index. In at least one embodiment, when the set flag indicating the specific candidate group is equal to one, one of the plurality of prediction indicators may be regarded as the set index for determining at least one of the plurality of prediction parameters. In this embodiment, when the set flag indicating another candidate group is equal to zero, one of the plurality of prediction indicators may be regarded as the group index for determining at least one of the plurality of prediction parameters.

[0094] Refer to Figure 5, at block 54, the decoder module 222 may select multiple MMVD prediction parameters of the block unit based on the multiple MMVD indications and the selected candidate group.

[0095] In at least one embodiment, when the block unit is predicted in the MMVD mode, the multiple prediction parameters are MMVD prediction parameters. In at least one embodiment, when the set flag is equal to one, the inter prediction unit 22222 may determine the MMVD prediction parameters based on the set index and other prediction indications. For example, when the set index represents the base candidate index and the motion difference index, the inter prediction unit 22222 may directly determine the base candidate and the motion difference of the block unit based on the set index, and determine whether the block unit is a bi - directional prediction block or a uni - directional prediction block based on the prediction direction index. In at least one embodiment, when each of the set index and the group index represents the motion distance index, the inter prediction unit 22222 may determine the difference magnitude of the block unit based on the set index, and may determine other prediction parameters based on the base candidate index, the difference direction index, and the prediction direction index.

[0096] In at least one embodiment, when the set flag is equal to zero, the inter prediction unit 22222 may determine the multiple MMVD prediction parameters based on the group index and other prediction indications. For example, when each of the set index and the group index represents the motion distance index, the inter prediction unit 22222 may determine the difference magnitude of the block unit from the group index, and determine other prediction parameters based on the base candidate index, the difference direction index, and the prediction direction index.

[0097] At block 55, the decoder module 222 may reconstruct the block unit based on the multiple MMVD prediction parameters.

[0098] In at least one embodiment, the inter prediction unit 22222 may generate a prediction component for one of the multiple block components in the block unit according to the multiple prediction parameters. In at least one embodiment, the first adder 2224 may generate multiple reconstructed components based on the multiple prediction components and multiple residual components for reconstructing the block unit. In this embodiment, the first adder 2224 may receive the multiple residual components of the block unit via the entropy decoding unit 2221 and the inverse quantization / inverse transform unit 2223. In this embodiment, the multiple residual components may be determined from the bitstream. In at least one embodiment, the decoder module 222 may reconstruct all other block units in the image frame to reconstruct the image frame and the video.

[0099] Figure 6It is a flowchart of an exemplary reconstruction method for reconstructing a block unit based on multiple merge mode with motion vector difference (MMVD) candidate parameters in multiple candidate groups according to an exemplary embodiment of the present application. Since there are multiple ways to execute this method, only an example method is provided by way of example. For example, the method described below can be executed using the configurations shown in Figure 1 and Figure 2 , and various elements of these figures are referred to when explaining the example method. Each box shown in Figure 6 may represent one or more processes, methods, or subroutines executed in the example method. In addition, the order of the boxes is illustrative only and can be changed. Additional boxes can be added or fewer boxes can be utilized without departing from the present application.

[0100] In block 61, the decoder module 222 may determine a block unit from an image frame according to video data.

[0101] In at least one embodiment, the video data may be a bitstream. In at least one embodiment, the destination device 12 may receive the bitstream from an encoder such as the source device 11 through the second interface 123 of the destination device 12. The second interface 123 may provide the bitstream to the decoder module 222. The decoder module 222 may determine an image frame according to the bitstream and segment the image frame according to multiple segmentation indications in the bitstream to determine a block unit. For example, the decoder module 222 may segment the image frame to generate multiple coding tree units and also segment one of the coding tree units based on the segmentation indication in the video coding standard to determine a block unit.

[0102] In at least one embodiment, the entropy decoding unit 2221 may decode the bitstream to determine multiple prediction indications for the block unit, and then the decoder module 222 may further reconstruct the block unit based on the prediction indications. In at least one embodiment, the prediction indications may include multiple flags and multiple indexes.

[0103] In block 62, the prediction processing unit 2222 may select one of multiple candidate groups based on a set flag, and each candidate group may include one or more first merge mode with motion vector difference (MMVD) candidate lists.

[0104] In at least one embodiment, when the block unit is predicted in the MMVD mode, there may be multiple motion parameters for predicting the block unit. In at least one embodiment, the multiple motion parameters may include a base motion, a motion difference, and a prediction direction. In at least one embodiment, each of the multiple motion parameters may include multiple candidate parameters. In at least one embodiment, the decoder module 222 may select a prediction parameter from the candidate parameters for each of the multiple motion parameters. For example, one of the multiple prediction parameters may be selected from the multiple candidate parameters of the base motion.

[0105] In at least one embodiment, the inter-frame prediction unit 22222 may generate the multiple base candidates for the block unit based on adjacent motion information. In at least one embodiment, the multiple base candidates are candidate parameters for selecting the base motion. Each of the multiple base candidates includes at least one frame candidate and at least one vector candidate.

[0106] In at least one embodiment, the motion difference may further include a difference magnitude and a difference direction. In at least one embodiment, the inter-frame prediction unit 22222 may select one of the multiple base candidates and determine a motion difference to adjust the vector candidate in the selected base candidate. In at least one embodiment, multiple difference distance candidates are the multiple candidate parameters for selecting the difference magnitude, and multiple difference direction candidates are the multiple candidate parameters for selecting the difference direction.

[0107] In at least one embodiment, the inter-frame prediction unit 22222 may select at least one of the first reference list L0 and the second reference list L1 from multiple prediction direction candidates to predict the block unit. When the block unit is predicted based on one of the multiple first frame candidates in the first reference list L0 and one of the multiple second frame candidates in the second reference list L1, the block unit is a bi-directionally predicted block predicted based on the first reference list L0 and the second reference list L1.

[0108] In at least one embodiment, the plurality of candidate parameters corresponding to at least one of the plurality of motion parameters may be classified into the plurality of candidate groups. In the embodiment, other motion parameters may not be included in the plurality of candidate groups. In at least one embodiment, each of the plurality of candidate groups may include one or more first MMVD candidate lists, and each of the one or more first MMVD candidate lists has a plurality of first MMVD candidate parameters. In the embodiment, the plurality of first MMVD candidate parameters may be the plurality of candidate parameters among at least one classified motion parameter. In at least one embodiment, the number of at least one classified motion parameter may be equal to the number of one or more first MMVD candidate lists in each of the plurality of candidate groups. For example, the plurality of differential distance candidates and the plurality of differential direction candidates may be classified into the plurality of candidate groups. In the embodiment, the plurality of base candidates and the plurality of predicted direction candidates may not be included in the plurality of candidate groups. Therefore, at least one classified motion parameter is the differential magnitude and the differential direction, and the number of at least one classified motion parameter may be equal to 2. Additionally, the number of one or more first MMVD candidate lists in each of the plurality of candidate groups may be equal to two. In the embodiment, one of the two first MMVD candidate lists in each of the plurality of candidate groups may include the plurality of differential distance candidates, and the other of the two first MMVD candidate lists in each of the plurality of candidate groups may include the plurality of differential direction candidates. In at least one embodiment, the plurality of differential distance candidates may be classified into the plurality of candidate groups. In the embodiment, the plurality of base candidates, the plurality of differential direction candidates, and the plurality of predicted direction candidates may not be included in the plurality of candidate groups. Therefore, the at least one classified motion parameter is the differential magnitude, and the number of the at least one classified motion parameter may be equal to one. Additionally, the number of one or more first MMVD candidate lists in each of the plurality of candidate groups may be equal to one. In the embodiment, the first MMVD candidate list in each of the plurality of candidate groups may include differential distance candidates.

[0109] In at least one embodiment, the distribution of the plurality of candidate parameters in the plurality of candidate groups may be predefined (e.g., in the encoder module 112 and the decoder module 222). Thus, the inter-frame prediction unit 22222 may directly determine the classification result of the plurality of first MMVD candidate parameters in the plurality of candidate groups from the decoder module 222. In at least one embodiment, the inter-frame prediction unit 22222 may directly determine the plurality of candidate groups including the plurality of first MVMVD candidate parameters. In the embodiment, the plurality of first MMVD candidate parameters in each of the plurality of candidate groups may be predefined in the decoder module 222.

[0110] In at least one embodiment, the plurality of prediction indications may include a set flag indicating whether at least one of the plurality of prediction parameters of the block unit is selected from a specific one of the plurality of candidate groups. Thus, when the set flag is equal to 1, the inter prediction unit 22222 may directly select at least one of the plurality of prediction parameters of the block unit from the specific candidate group. Additionally, when the set flag is equal to zero, the inter prediction unit 22222 may directly determine that at least one of the plurality of prediction parameters of the block unit is not selected from the specific candidate group. In at least one embodiment, the set flag may be fpel_mmvd_enable_flag to indicate whether at least one of the plurality of prediction parameters of the block unit is selected from the specific candidate group. In at least one embodiment, when the set flag is included in the slice header, the set flag may be slice_fpel_mmvd_enabled_flag.

[0111] Referring to Figure 6 , at block 63, the prediction processing unit 2222 may determine a plurality of MMVD indications of the block unit according to the video data.

[0112] In at least one embodiment, the plurality of prediction indications may include the plurality of MMVD indications for the MVVD mode indication base candidate index, motion difference indication, and prediction direction index. In at least one embodiment, the plurality of MMVD indications may include at least one of the base candidate index, the motion difference indication, and the prediction direction index. In at least one embodiment, the decoder module 222 may select one of the plurality of prediction parameters from the plurality of candidate parameters for each of the plurality of motion parameters based on a corresponding one of the plurality of MMVD indications. For example, one of the plurality of prediction parameters may be selected from the plurality of candidate parameters of the base motion based on the base candidate index.

[0113] In at least one embodiment, the base candidate index may be the MMVD flag mmvd_cand_flag, which indicates the selected base candidate included in the plurality of prediction parameters. In one embodiment, when the base motion of the block unit is a bi-prediction candidate selected from the plurality of base candidates based on the base candidate index, the base motion of the block unit may indicate the first frame candidate in the first reference list L0, the first vector candidate corresponding to the first frame candidate, the second frame candidate in the second reference list L1, and the second vector candidate corresponding to the second frame candidate.

[0114] In at least one embodiment, the motion difference indication may further include a motion distance index and a motion direction index. In at least one embodiment, the motion distance index may be a first MMVD index mmvd_distance_idx indicating a difference magnitude from among the plurality of difference distance candidates, and the motion direction index may be a second MMVD index mmvd_direction_idx indicating a difference direction from among the plurality of difference direction candidates. In at least one embodiment, the motion difference indication may be a motion difference index indicating a motion difference including an X coordinate difference and a Y coordinate difference. Accordingly, the inter-frame prediction unit 22222 may determine a motion difference based on the X coordinate difference and the Y coordinate difference indicated by the motion difference index to adjust the selected vector candidate.

[0115] In at least one embodiment, the inter-frame prediction unit 22222 may select at least one of the first reference list L0 and the second reference list L1 from among a plurality of prediction direction candidates based on the prediction direction index. In at least one embodiment, when the reference list determined by the base candidate index is preset as the selected reference list for predicting the block unit, there may be no prediction direction index among the plurality of prediction indications.

[0116] In at least one embodiment, the MMVD indication may include a set index. In the embodiment, when the motion difference indication is the motion difference index, the set index may indicate at least one of the base candidate index, the motion difference index, and the prediction direction index. In the embodiment, the index indicated by the at least one selected from the base candidate index, the motion difference index, and the prediction direction index may not be included in the plurality of MMVD indications, while other indexes selected from the base candidate index, the motion difference index, and the prediction direction index that are not indicated in the set index may be directly included in the plurality of MMVD indications. In at least one embodiment, when the motion difference indication includes the motion distance index and the motion direction index, the set index may indicate at least one of the base candidate index, the motion distance index, the motion direction index, and the prediction direction index. In the embodiment, the index indicated by the at least one selected from the base candidate index, the motion distance index, the motion direction index, and the prediction direction index may not be included in the plurality of MMVD indications, while other indexes selected from the base candidate index, the motion distance index, the motion direction index, and the prediction direction index that are not indicated in the set index may be directly included in the plurality of MMVD indications. For example, when the set index only represents the motion distance index, the decoder module 222 may directly determine the difference magnitude based on the set index. In the embodiment, when the set index represents the motion distance index, the plurality of difference distance candidates may be distributed into the plurality of candidate groups. In the embodiment, the plurality of base candidates, the plurality of difference direction candidates, and the plurality of prediction direction candidates are not included in the plurality of candidate groups. Additionally, the base candidate index, the motion direction index, and the prediction direction index directly included in the plurality of MMVD indications may not be indicated by the set index.

[0117] At block 64, the decoder module 222 may select one or more first MMVD prediction parameters of the block unit from one or more first MMVD candidate lists included in the selected candidate group.

[0118] In at least one embodiment, the one or more first MMVD prediction parameters are included in the plurality of prediction parameters of the block unit. In at least one embodiment, the inter-frame prediction unit 22222 may directly determine the one or more first MMVD prediction parameters from the specific candidate group based on the set index. In at least one embodiment, the inter-frame prediction unit 22222 may select the one or more first MMVD prediction parameters from one or more first MMVD candidate lists in the selected candidate group.

[0119] In at least one embodiment, when the number of the at least one classified motion parameter is equal to one, the number of the one or more first MMVD candidate lists in the selected candidate group may be equal to one. Thus, the inter-frame prediction unit 22222 may select one first MMVD prediction parameter corresponding to the one classified motion parameter from one first MMVD candidate list in the selected candidate group. For example, the one classified motion parameter corresponding to the one first MMVD prediction parameter is the difference magnitude of the block unit. Thus, the one first MMVD candidate list may include only the plurality of difference distance candidates. In the embodiment, the inter-frame prediction unit 22222 may select the difference magnitude of the block unit from the plurality of difference distance candidates in the one first MMVD candidate list of the selected candidate group based on the set index.

[0120] In at least one embodiment, when the number of the at least one classified motion parameter is greater than one, the number of the one or more first MMVD candidate lists in the selected candidate group may be greater than one. Thus, the inter-frame prediction unit 22222 may select the plurality of first MMVD prediction parameters from the plurality of first MMVD candidate lists in the selected candidate group, and each of the plurality of first MMVD prediction parameters may correspond to one of the plurality of classified motion parameters. In at least one embodiment, since each of the plurality of classified motion parameters corresponds to one of the plurality of first MMVD candidate lists in the selected candidate group, the inter-frame prediction unit 22222 may select each of the plurality of first MMVD prediction parameters from the plurality of first MMVD candidate parameters in the corresponding one first MMVD candidate list of the plurality of first MMVD candidate lists in the selected candidate group.

[0121] In block 65, the decoder module 222 may select one or more second MMVD prediction parameters of the block unit from one or more second MMVD candidate lists excluded from the plurality of candidate groups.

[0122] In at least one embodiment, the one or more second MMVD prediction parameters are included in the plurality of prediction parameters of the block unit. In at least one embodiment, since the at least one unclassified motion parameter is excluded from the plurality of candidate groups, the plurality of candidate parameters in the at least one unclassified motion parameter may be included in the one or more second MMVD candidate lists excluded from the plurality of candidate groups. In the embodiment, each of the one or more second MMVD candidate lists is different from the one or more first MMVD candidate lists. In at least one embodiment, the plurality of candidate parameters in each of the at least one unclassified motion parameter may be included in a corresponding one of the one or more second MMVD candidate lists. For example, when the at least one unclassified motion parameter is the base motion and the difference direction, the base candidate may be included in one of the two second MMVD candidate lists, and the difference direction candidate may be included in the other of the two second MMVD candidate lists.

[0123] In at least one embodiment, the inter-frame prediction unit 22222 may determine the one or more second MMVD prediction parameters based on one or more of the plurality of MMVD indications, and each of the one or more second MMVD prediction parameters may correspond to at least one unclassified motion parameter. In at least one embodiment, each of the one or more MMVD indications is different from the set index. In at least one embodiment, the inter-frame prediction unit 22222 may select each of the one or more second MMVD prediction parameters from the plurality of second MMVD candidate parameters in the corresponding one of the one or more second MMVD candidate lists based on a corresponding one of the one or more of the plurality of MMVD indications. For example, when the at least one unclassified motion parameter is the base motion and the difference direction, the base motion of the block unit may be determined based on the base candidate index, and the difference magnitude of the block unit may be determined based on the motion distance index.

[0124] In block 66, the decoder module 222 may reconstruct the block unit based on the one or more first prediction parameters and the one or more second prediction parameters.

[0125] In at least one embodiment, the inter-frame prediction unit 22222 may generate a prediction component for one of the multiple block components in a block unit according to the one or more first prediction parameters and the one or more second prediction parameters. In at least one embodiment, the first adder 2224 may generate multiple reconstruction components based on the multiple prediction components and multiple residual components for reconstructing the block unit. In this embodiment, the first adder 2224 may receive the multiple residual components of the block unit via the entropy decoding unit 2221 and the inverse quantization / inverse transformation unit 2223. In this embodiment, the multiple residual components may be determined from a bitstream. In at least one embodiment, the decoder module 222 may reconstruct all other block units in the image frame to reconstruct the image frame and the video.

[0126] Figure 7 is a block diagram of an encoder module 712 according to an exemplary embodiment of the present application, and the encoder module 712 represents Figure 1 an exemplary embodiment of the encoder module 12 of the source device 11 in the system of. In at least one embodiment, the encoder module 712 may include a prediction processor (e.g., prediction processing unit 7121), at least one adder (e.g., first adder 7122 and second adder 7125), a transform / quantization processor (e.g., transform / quantization unit 7123), an inverse quantization / inverse transformation processor (e.g., inverse quantization / inverse transformation unit 7124), a filter (e.g., filtering unit 7126), a decoded picture buffer (e.g., decoded picture buffer 7127), and an entropy encoding unit (e.g., entropy encoding unit 7128). In at least one embodiment, the prediction processing unit 7121 of the encoder module 712 may further include a segmentation processor (e.g., segmentation unit 71211), an intra-frame prediction processor (e.g., intra-frame prediction unit 71212), and an inter-frame prediction processor (e.g., inter-frame prediction unit 71213). In at least one embodiment, the encoder module 712 may receive a source video and encode the source video to output a bitstream.

[0127] In at least one embodiment, the encoder module 712 may receive a source video including a plurality of image frames, and then segment the image frames according to an encoding structure. In at least one embodiment, each image frame may be segmented into at least one image block. At least one image block may include a luminance block having a plurality of luminance samples and at least one chrominance block having a plurality of chrominance samples. The luminance block and the at least one chrominance block may be further segmented to generate macroblocks, Coding Tree Units (CTUs), coding blocks (CBs), their sub-segments, and / or other equivalent coding units. In at least one embodiment, the encoder module 712 may perform additional fine segmentation of the source video. It should be noted that regardless of how the source video is segmented before and / or during encoding, the described embodiments are generally applicable to video encoding.

[0128] In at least one embodiment, during the encoding process, the prediction processing unit 7121 may receive a current image block of a specific one of the plurality of image frames. The current image block may be one of the luminance block and at least one chrominance block in the specific image frame. The segmentation unit 71211 may segment the current image block into a plurality of block units. The intra prediction unit 71212 may perform intra prediction encoding of the current block unit relative to one or more adjacent blocks in the same frame as the current block unit to provide spatial prediction. The inter prediction unit 71213 may perform inter prediction encoding of the current block unit relative to one or more blocks in one or more reference image blocks to provide temporal prediction.

[0129] In at least one embodiment, the prediction processing unit 7121 may select one of a plurality of encoding results generated by the intra prediction unit 71212 and the inter prediction unit 71213 based on a mode selection method such as a cost function. In at least one embodiment, the mode selection method may be a Rate-Distortion Optimization (RDO) process. The prediction processing unit 7121 may determine the selected encoding result, and provide the prediction block corresponding to the selected encoding result to the first adder 7122 to generate a residual block, and provide it to the second adder 7125 to reconstruct the encoded block unit. In at least one embodiment, the prediction processing unit 7121 may also provide syntax elements such as motion vectors, intra mode indicators, segmentation information, and other syntax information to the entropy encoding unit 7128.

[0130] In at least one embodiment, the intra prediction unit 71212 may perform intra prediction on the current block unit. In at least one embodiment, the intra prediction unit 71212 may determine an intra prediction mode pointing to a reconstructed sample adjacent to the current block unit to encode the current block unit. In at least one embodiment, the intra prediction unit 71212 may encode the current block unit using various intra prediction modes, and the intra prediction unit 71212 or the prediction processing unit 7121 may select an appropriate intra prediction mode from test modes. In at least one embodiment, the intra prediction unit 71212 may use a cross-component prediction mode to encode the current block unit to predict one of the two chrominance components of the current block unit based on the luminance component of the current block unit. Additionally, the intra prediction unit 71212 may predict the first of the two chrominance components of the current block unit based on the other of the two chrominance components of the current block unit.

[0131] In at least one embodiment, as described above, the inter prediction unit 71213 may perform inter prediction on the current block unit as an alternative to the intra prediction performed by the intra prediction unit 71212. The inter prediction unit 71213 may perform motion estimation to estimate the motion of the current block unit to generate a motion vector. The motion vector may indicate the displacement of the current block unit within the current image block relative to a reference block unit within a reference image block. In at least one embodiment, the inter prediction unit 71213 may receive at least one reference image block stored in the decoded picture buffer 7127 and estimate the motion based on the received reference image block to generate a motion vector.

[0132] In at least one embodiment, the first adder 7122 may generate a residual block by subtracting the prediction block determined by the prediction processing unit 7121 from the original current block unit. The first adder 7122 may represent one or more elements that perform this subtraction operation.

[0133] In at least one embodiment, the transform / quantization unit 7123 may apply a transform to the residual block to generate residual transform coefficients, and then may quantize the residual transform coefficients to further reduce the bit rate. In at least one embodiment, the transform may be one of a DCT, DST, AMT, MDNSST, HyGT, signal-dependent transform, KLT, wavelet transform, integer transform, sub-band transform, or a conceptually similar transform. In at least one embodiment, the transform may convert the residual information from the pixel value domain to the transform domain, such as the frequency domain. In at least one embodiment, the degree of quantization may be modified by adjusting a quantization parameter. In at least one embodiment, the transform / quantization unit 7123 may scan a matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 7128 may perform the scanning.

[0134] In at least one embodiment, the entropy coding unit 7128 may receive a plurality of syntax elements including quantization parameters, transform data, motion vectors, intra-mode, segmentation information, and other syntax information from the prediction processing unit 7121 and the transform / quantization unit 7123, and may entropy code the syntax elements into a bitstream. In at least one embodiment, the entropy coding unit 7128 entropy codes the quantized transform coefficients. In at least one embodiment, the entropy coding unit 7128 may perform CAVLC, CABAC, SBAC, PIPE coding, or another entropy coding technique to generate an encoded bitstream. In at least one embodiment, the encoded bitstream may be sent to another device (e.g., the destination device 12) or archived for later transmission or retrieval.

[0135] In at least one embodiment, the inverse quantization / inverse transform unit 7124 may apply inverse quantization and inverse transform to reconstruct the residual block in the pixel domain for later use as a reference block. In at least one embodiment, the second adder 7125 may add the reconstructed residual block to the prediction block provided from the prediction processing unit 7121 to generate a reconstructed block to be stored in the decoded picture buffer 7127.

[0136] In at least one embodiment, the filtering unit 7126 may include a deblocking filter, SAO filter, bilateral filter, and / or ALF to remove blocking artifacts from the reconstructed block. In addition to the deblocking filter, SAO filter, bilateral filter, and ALF, other filters (intra-loop or post-loop) may also be used. For the sake of brevity, such filters are not shown, but if needed, the output of the second adder 7125 may be filtered.

[0137] In at least one embodiment, the decoded picture buffer 7127 may be a reference picture memory that stores reference blocks used, for example, by the encoder module 712 to encode video in an intra- or inter-frame coding mode. The decoded picture buffer 7127 may be formed of any one of a variety of storage devices, such as DRAM including SDRAM, MRAM, RRAM, or other types of storage devices. In at least one embodiment, the decoded picture buffer 7127 may be on-chip with other elements of the encoder module 712 or off-chip relative to those elements.

[0138] In at least one embodiment, the encoder module 712 may perform a prediction method as Figure 3 shown, the prediction method being for predicting block units based on multiple candidate parameters of merge modes with motion vector differences (MMVD) in multiple candidate groups. For example, the configuration as Figure 1 and Figure 7 shown may be used to perform the Figure 3 method, and various elements of these figures are referred to when explaining the example method. In addition, in addition, Figure 3 the order of the boxes in

[0139] is merely illustrative and may be changed. Additional boxes may be added or fewer boxes may be utilized without departing from the present application.

[0139] At block 31, the encoder module 712 may determine block units from an image frame according to video data and determine multiple candidate parameters of the block units.

[0140] In at least one embodiment, the video data may be video. The source device 11 may receive video through the source module 111. The encoder module 712 may determine an image frame from the video and segment the image frame to determine block units.

[0141] In at least one embodiment, the prediction processing unit 7121 of the source device 11 may determine block units from video via the segmentation unit 71211, and then the encoder module 712 may provide multiple segmentation indications into the bitstream based on the segmentation result of the segmentation unit 71211.

[0142] In at least one embodiment, the source device 11 may determine a plurality of candidate parameters for selecting a merge mode with motion vector difference (MMVD) mode from a plurality of candidate modes for a block unit, and select a plurality of prediction parameters from the plurality of candidate parameters to predict the block unit. In this embodiment, the source device 11 may provide a plurality of prediction indications indicating the plurality of prediction parameters of the block unit in a bitstream to the target device 12. In at least one embodiment, the plurality of prediction indications may include a plurality of flags and a plurality of indexes.

[0143] In at least one embodiment, the MMVD mode may include a plurality of motion parameters. In at least one embodiment, the motion parameters may include a base motion, a motion difference, and a prediction direction. In at least one embodiment, each motion parameter has a plurality of candidate parameters. In at least one embodiment, the prediction indication may include at least one of a base candidate index, a motion difference indication, and a prediction direction index for the MMVD mode. In at least one embodiment, the prediction indication may be provided to the bitstream for each of the plurality of motion parameters. For example, the base candidate index may be provided to the bitstream for the base motion.

[0144] In at least one embodiment, when the motion difference indication includes a motion distance index and a motion direction index, the set index may represent at least one of the base candidate index, the motion distance index, the motion direction index, and the prediction direction index. For example, when the set index only represents the motion distance index, the encoder module 712 may provide the set index to the bitstream to indicate the difference magnitude.

[0145] In block 32, the encoder module 712 may determine a plurality of candidate groups, each candidate group including a plurality of group parameters selected from the plurality of candidate parameters.

[0146] In at least one embodiment, the inter-frame prediction unit 71213 may determine a plurality of previous blocks encoded before the block unit. In the embodiment, the plurality of previous blocks are predicted by a plurality of previous patterns selected from the plurality of candidate patterns. In at least one embodiment, when the plurality of previous patterns are MMVD patterns, the plurality of previous patterns may include a plurality of previous parameters selected from the plurality of candidate parameters. Accordingly, the inter-frame prediction unit 71213 may determine the plurality of previous parameters of the plurality of previous blocks, and classify the plurality of candidate parameters that are the same as or similar to the plurality of previous parameters of the plurality of prediction patterns into a first candidate group. For example, when the inter-frame prediction unit 71213 uses a set index to represent a difference magnitude, the inter-frame prediction unit 71213 compares the plurality of difference distance candidates among the plurality of candidate parameters with the plurality of difference magnitudes of the plurality of previous patterns of the plurality of previous blocks.

[0147] In at least one embodiment, the distribution of the plurality of candidate parameters in the plurality of candidate groups may be predefined (e.g., in the encoder module 712 and the decoder module 122). Accordingly, the inter-frame prediction unit 71213 may directly determine the classification result of the plurality of candidate parameters in the plurality of candidate groups from the encoder module 712. In the embodiment, the plurality of candidate parameters in each candidate group may be predefined in the encoder module 712.

[0148] In block 33, the encoder module 71 may determine whether a particular one of the plurality of candidate parameters for predicting the block unit is included in the first candidate group. If the particular candidate parameter of the block unit is included in the first candidate group, the program may proceed to block 34. If the particular candidate parameter of the block unit is not included in the first candidate group, the program may proceed to block 35.

[0149] In at least one embodiment, when the block unit is predicted in the MMVD mode, the encoder module 712 may determine whether the specific prediction candidate of the block unit is included in the first candidate group. When the encoder module 712 determines that the specific prediction candidate of the block unit is included in the first candidate group, the encoder module 712 may provide a set flag equal to one to the bitstream. When the encoder module 712 determines that the specific prediction candidate of the block unit is not included in the first candidate group, the encoder module 712 may provide a set flag equal to zero to the bitstream. In at least one embodiment, the set flag may be fpel_mmvd_enable_flag to indicate whether the specific candidate parameter of the block unit is included in the first candidate group. In at least one embodiment, when the set flag is included in the slice header, the set flag may be slice_fpel_mmvd_enabled_flag.

[0150] At block 34, the encoder module 712 may determine the plurality of prediction parameters based on the specific candidate parameter corresponding to the set index.

[0151] In at least one embodiment, each group parameter in the first candidate group may correspond to one index value of the set index. Therefore, when the specific prediction candidate is included in the first candidate group, the inter prediction unit 71213 may determine the index value corresponding to the specific prediction candidate and provide the set index to the bitstream.

[0152] At block 35, the encoder module 712 may determine the plurality of prediction parameters based on the specific candidate parameter in the remaining candidate group.

[0153] In at least one embodiment, each of the plurality of group parameters in the plurality of remaining candidate groups corresponds to one index value of the group index. Therefore, when the specific candidate parameter is not included in the first candidate group, the inter prediction unit 71213 may determine the index value corresponding to the specific candidate parameter and provide the group index to the bitstream.

[0154] At block 36, the encoder module 712 may reconstruct the block unit based on the determined prediction parameters.

[0155] In at least one embodiment, the inter-frame prediction unit 71213 may generate a prediction component for one of the plurality of block components in a block unit according to the plurality of prediction parameters. In at least one embodiment, the first adder 7122 may generate a plurality of residual components based on the plurality of prediction components and the plurality of original components for predicting an image frame. In the embodiment, the plurality of original components are directly determined from the image frame of the video. In the embodiment, the transform / quantization unit 7123 and the entropy encoding unit 7128 may encode the plurality of residual components to provide a bitstream to the destination device 12. In addition, the second adder 7125 may also reconstruct the block unit based on the plurality of residual components and the plurality of prediction components to generate a plurality of reconstructed components. Therefore, the plurality of reconstructed components of the block unit may be reference blocks for predicting the next block unit.

[0156] Figure 8 FIG. 4 is a flowchart of an exemplary prediction method for predicting a block unit based on a plurality of candidate parameters of a merge mode with motion vector difference (MMVD) in a plurality of candidate groups according to an exemplary embodiment of the present application. Since there are multiple ways to perform this method, an example method is provided only by way of example. For example, the following-described method may be performed using the configurations shown in FIGS. 5 and 6, and various elements of these figures are referred to in the explanation of the example method. Each box shown in FIG. 5 may represent one or more processes, methods, or subroutines performed in the example method. In addition, the order of the boxes is merely illustrative and may be changed. Additional boxes may be added or fewer boxes may be utilized without departing from the present application. Figure 1 and Figure 7 FIG. 6. The configurations shown in FIGS. 5 and 6 are referred to in the explanation of the example method, and various elements of these figures are referred to in the explanation of the example method. Each box shown in FIG. 5 may represent one or more processes, methods, or subroutines performed in the example method. In addition, the order of the boxes is merely illustrative and may be changed. Additional boxes may be added or fewer boxes may be utilized without departing from the present application. Figure 8 Each box shown in FIG. 5 may represent one or more processes, methods, or subroutines performed in the example method. In addition, the order of the boxes is merely illustrative and may be changed. Additional boxes may be added or fewer boxes may be utilized without departing from the present application.

[0157] In block 81, the encoder module 712 determines a block unit from an image frame according to video data.

[0158] In at least one embodiment, the video data may be a video. The source device 11 may receive the video by the source module 111. The encoder module 712 determines an image frame from the video and segments the image frame to determine a block unit. In at least one embodiment, the prediction processing unit 7121 of the source device 11 may determine a block unit from the video via the segmentation unit 71211, and then the encoder module 712 may provide a plurality of segmentation indications to the bitstream based on the segmentation result of the segmentation unit 71211.

[0159] At block 82, the encoder module 712 may determine one or more first MMVD prediction parameters of the block unit from a plurality of first MMVD candidate parameters in a plurality of candidate groups, and determine one or more second MMVD prediction parameters of the block unit from a plurality of second MMVD candidate parameters excluded from the plurality of candidate groups.

[0160] In at least one embodiment, when the block unit is predicted in an MMVD mode selected from a plurality of candidate modes, the encoder module 712 may determine a plurality of MMVD candidate parameters and select a plurality of MMVD prediction parameters from the plurality of MMVD candidate parameters.

[0161] In at least one embodiment, the MMVD mode may include a plurality of motion parameters. In at least one embodiment, the plurality of motion parameters may include a base motion, a motion difference, and a prediction direction. In at least one embodiment, the plurality of motion parameters may include the base motion, a difference magnitude, a difference direction, and the prediction direction. In at least one embodiment, each of the plurality of motion parameters may include the plurality of MMVD candidate parameters.

[0162] In at least one embodiment, the plurality of MMVD candidate parameters may include the plurality of first MMVD candidate parameters and the plurality of second MMVD candidate parameters, and the plurality of motion parameters may be separated into one or more first motion parameters and one or more second motion parameters. In at least one embodiment, the plurality of first MMVD candidate parameters for selecting the one or more first MMVD prediction parameters may be the plurality of MMVD candidate parameters corresponding to the one or more first motion parameters and distributed in the plurality of candidate groups, and the plurality of second MMVD candidate parameters for selecting the one or more second MMVD prediction parameters may be the plurality of MMVD candidate parameters corresponding to the one or more second motion parameters and excluded from the plurality of candidate groups. In at least one embodiment, each of the plurality of candidate groups may include the plurality of first MMVD candidate parameters corresponding to each of the one or more first motion parameters. For example, the one or more first motion parameters are a difference magnitude and a difference direction. In this embodiment, each of the plurality of candidate groups may include the plurality of first MMVD candidate parameters of the difference magnitude and the plurality of first MMVD candidate parameters of the difference direction. For example, the one or more first motion parameters are a difference magnitude. In this embodiment, each of the plurality of candidate groups may include the plurality of first MMVD candidate parameters of the difference magnitude.

[0163] In at least one embodiment, the distribution of the plurality of first MMVD candidate parameters in the plurality of candidate groups may be predefined in the encoder module 712 and the decoder module 122. Thus, the inter-frame prediction unit 71213 may directly determine the predefined distribution of the plurality of first MMVD candidate parameters in the plurality of candidate groups.

[0164] At block 83, the encoder module 712 may determine a set flag indicating a particular one of the plurality of candidate groups corresponding to the block unit, a set index indicating one or more first MMVD prediction parameters included in the particular candidate group, and one or more MMVD indexes indicating one or more second MMVD prediction parameters excluded by the plurality of candidate groups.

[0165] In the embodiment, the encoder module 712 may provide a plurality of prediction indications indicating the plurality of MMVD prediction parameters of the block unit to the destination device 12. In at least one embodiment, the plurality of prediction indications may include a plurality of flags and a plurality of indexes. In at least one embodiment, the plurality of prediction indications may include at least one of a base candidate index for the MMVD mode, a motion difference indication, and a prediction direction index to indicate the plurality of motion parameters.

[0166] In at least one embodiment, when the motion difference indication includes the motion distance index and the motion direction, the set index may represent at least one of the base candidate index, the motion distance index, the motion direction index, and the prediction direction index. For example, when the set index only represents the motion distance index, the encoder module 712 may provide the set index to indicate the difference magnitude.

[0167] In at least one embodiment, the encoder module 712 may determine whether one or more first MMVD prediction parameters of the block unit are included in the first of the plurality of candidate groups. When the encoder module 712 determines that one or more first MMVD prediction parameters of the block unit are included in the first candidate group, the encoder module 712 may provide a set flag equal to one. When the encoder module 712 determines that one or more first MMVD prediction parameters of the block unit are not included in the first candidate group, the encoder module 712 may provide a set flag equal to zero. In at least one embodiment, the set flag may be fpel_mmvd_enable_flag to indicate whether one or more first MMVD prediction parameters of the block unit are included in the first candidate group. In at least one embodiment, when the set flag is included in the slice header, the set flag may be slice_fpel_mmvd_enabled_flag.

[0168] In at least one embodiment, each of the plurality of first MMVD candidate parameters in the plurality of candidate groups corresponds to one of a plurality of first index values for the set index. Thus, the encoder module 712 can determine a particular one of the plurality of first index values corresponding to the one or more first MMVD prediction parameters for the set index. In at least one embodiment, each of the plurality of second MMVD candidate parameters corresponds to one of a plurality of second index values for a corresponding one of the one or more MMVD indexes. Thus, the inter prediction unit 91213 can determine a particular one of the plurality of second index values, the particular second index value corresponding to a particular one of the one or more first MMVD prediction parameters for a corresponding one of the one or more MMVD indexes.

[0169] At block 84, the encoder module 712 can provide the set flag, the set index, and the one or more MMVD indexes to the bitstream.

[0170] In at least one embodiment, the inter prediction unit 71213 can generate a prediction component for one of the plurality of block components in a block unit based on the one or more first MMVD prediction parameters and the one or more second MMVD prediction parameters. In at least one embodiment, the first summer 7122 can generate a plurality of residual components based on the plurality of prediction components and a plurality of original components for predicting an image frame. In the embodiment, the plurality of original components are directly determined from the image frame of the video. In the embodiment, the transform / quantization unit 7123 and the entropy coding unit 7128 can encode the plurality of residual components. In at least one embodiment, the encoded residual components, the set flag, the set index, and the one or more MMVD indexes can be provided to the bitstream to reach the destination device 12.

[0171] However, as described above, it is obvious that various techniques can be used to implement the concepts described in this application without departing from the scope of those concepts. In addition, having specifically described the concepts with reference to certain embodiments, those of ordinary skill in the art will recognize that changes in form and detail can be made without departing from the scope of those concepts. Thus, the described embodiments should be considered illustrative in all respects and not restrictive. It should also be understood that this application is not limited to the above specific embodiments, but that many rearrangements, modifications, and substitutions are possible without departing from the scope of this application.

Claims

1. A method for encoding video data by an electronic device, the method comprising: Determining block units from an image frame according to the video data; Selecting multiple merge mode with motion vector difference (MMVD) prediction parameters of the block units to predict the block units; Determining a set flag and multiple MMVD indications of the block units, the set flag indicating one of multiple candidate groups corresponding to the block units, and the multiple MMVD indications being used to indicate the multiple MMVD prediction parameters of the block units according to the indicated candidate group; and Encoding the set flag and the multiple MMVD indications into a bitstream, wherein: Each of the multiple candidate groups includes one or more first MMVD candidate lists, and each of the one or more first MMVD candidate lists in the one or more first MMVD candidate lists includes multiple first MMVD candidate parameters; One or more of the multiple MMVD prediction parameters are selected from the multiple first MMVD candidate parameters in the one or more first MMVD candidate lists of the indicated candidate group; and MMVD prediction parameters other than the one or more of the multiple MMVD prediction parameters are selected from multiple second candidate parameters in one or more second MMVD candidate lists, and each of the one or more second MMVD candidate lists in the one or more second MMVD candidate lists is different from the one or more first MMVD candidate lists.

2. The method according to claim 1, wherein A base candidate index, a motion difference index, and a prediction direction index are determined from the multiple MMVD indications.

3. The method according to claim 2, wherein A set index included in the multiple MMVD indications indicates one or more of the base candidate index, the motion difference index, and the prediction direction index.

4. The method according to claim 3, wherein When the number of one or more of the base candidate index, the motion difference index, and the prediction direction index indicated by the set index is equal to 1, the set index is the same as one of the base candidate index, the motion difference index, and the prediction direction index to select one of the multiple MMVD prediction parameters.

5. A method for encoding video data by an electronic device, the method comprising: Determining block units from an image frame according to the video data; Selecting multiple merge mode with motion vector difference (MMVD) prediction parameters of the block units to predict the block units; Determining a set flag indicating one of multiple candidate groups, wherein each of the multiple candidate groups includes one or more first merge mode with motion vector difference (MMVD) candidate lists; Determining a set index indicating one or more first MMVD prediction parameters of the MMVD prediction parameters of the block units from the one or more first MMVD candidate lists included in the indicated candidate group; Determining one or more MMVD indexes indicating one or more second MMVD prediction parameters of the MMVD prediction parameters of the block units from one or more second MMVD candidate lists not included in the multiple candidate groups; and Encode the set flag, the set index, and the one or more MMVD indices into a bitstream.

6. The method according to claim 5, characterized in that, Each of the one or more first MMVD prediction parameters is selected from a plurality of first MMVD candidate parameters within a corresponding one of the one or more first MMVD candidate lists of the indicated candidate group.

7. The method according to claim 5, wherein Each of the one or more second MMVD prediction parameters is selected from a plurality of second MMVD candidate parameters within a corresponding one of the one or more second MMVD candidate lists, each of the one or more second MMVD candidate lists being different from the one or more first MMVD candidate lists.

8. The method according to claim 5, wherein The base candidate index, the motion difference indication, and the prediction direction index indicating the one or more first MMVD prediction parameters and the one or more second MMVD prediction parameters are determined based on the set index and the one or more MMVD indices.

9. The method according to claim 8, wherein The set index indicates one or more indicated indices selected from the base candidate index, the motion difference indication, and the prediction direction index.

Images