Devices and methods for encoding video data

Abstract

This invention provides a method for decoding a bitstream using an electronic device. The electronic device: receives the bitstream and determines an affine enable flag corresponding to one or more image frames from the bitstream; when the affine enable flag is true, the electronic device determines a maximum index corresponding to the one or more image frames from the bitstream. The index value of the maximum index is in the range of 0 to N-K, where N is a first integer and K is a second integer less than N. When the affine enable flag is true, the electronic device determines a maximum number of zero or more sub-block-based merged motion vector prediction (MVP) candidates based on the maximum index. The electronic device reconstructs the one or more image frames based on the maximum number of zero or more sub-block-based merged MVP candidates.

Description

[0001] Cross-reference to related applications

[0002] This disclosure claims the benefit and priority of Provisional U.S. Patent Application Serial No. 62 / 992092 (hereinafter referred to as "'092 Provisional Case"), filed on March 19, 2020, entitled "High Level Syntax for MergeModes". The disclosure of '092 Provisional Case is hereby incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure relates generally to video coding, and in particular to techniques for using advanced syntax to determine the number of merging candidates. Background Technology

[0004] In conventional video coding methods, an encoder encodes video data to generate encoded data with multiple flags and multiple indices, and then provides this encoded data to a decoder. Flags indicate whether multiple coding modes are enabled. For example, the encoded data may include an affine enable flag indicating whether an affine mode is enabled for certain image frames. Furthermore, indices may indicate the maximum number of mode candidates for use in coding modes.

[0005] Although encoded data can include flags and indices for each encoding mode in each different level of syntax element, encoding efficiency may be reduced. Therefore, encoders and decoders need a way to use flags and indices more efficiently to avoid increasing the number of bits in the encoded data too much. Summary of the Invention

[0006] This disclosure relates to an apparatus and method for determining the number of merge candidates using an advanced syntax.

[0007] In a first aspect of this disclosure, a method for decoding a bitstream and an electronic device for performing the method are provided. The method includes: receiving the bitstream; determining from the bitstream an affine enable flag corresponding to one or more image frames; when the affine enable flag is true, determining from the bitstream a maximum index corresponding to one or more image frames, wherein the index value of the maximum index is in the index range of 0 to NK, where N is a first integer and K is a second integer less than N; when the affine enable flag is true, determining a maximum number of zero or more sub-block-based merged motion vector prediction (MVP) candidates based on the maximum index; and reconstructing the one or more image frames based on the maximum number of zero or more sub-block-based merged MVP candidates.

[0008] In a second aspect of this disclosure, an electronic device is provided for decoding a bitstream and for performing the method. The method includes: receiving the bitstream; determining from the bitstream a maximum index corresponding to one or more image frames; wherein the index value of the maximum index is in the index range of 0 to NK, where N is a first integer and K is a second integer less than N; determining a maximum number of zero or more sub-block-based merged motion vector prediction (MVP) candidates by subtracting the index value of the maximum index from N; and reconstructing the one or more image frames based on the maximum number of zero or more sub-block-based merged MVP candidates. Attached Figure Description

[0009] The various aspects of this disclosure are best understood when read in conjunction with the accompanying drawings in the following detailed description. The features are not drawn to scale, and their dimensions may be arbitrarily increased or decreased for clarity of discussion.

[0010] Figure 1 A block diagram of a system configured to encode and decode video data according to an embodiment of the present disclosure is shown.

[0011] Figure 2 The embodiments according to this disclosure are shown. Figure 1 The block diagram of the decoder module of the second electronic device shown is illustrated.

[0012] Figure 3 A flowchart illustrating a method for decoding video data via an electronic device according to an embodiment of the present disclosure is shown.

[0013] Figure 4 A flowchart illustrating a method for decoding video data via an electronic device according to an embodiment of the present disclosure is shown.

[0014] Figure 5 A flowchart illustrating a method for decoding video data via an electronic device according to an embodiment of the present disclosure is shown.

[0015] Figure 6 A flowchart illustrating a method for decoding video data via an electronic device according to an embodiment of the present disclosure is shown.

[0016] Figure 7 A flowchart illustrating a method for decoding video data via an electronic device according to an embodiment of the present disclosure is shown.

[0017] Figure 8 The embodiments according to this disclosure are shown. Figure 1 The diagram shows a block diagram of the encoder module of the first electronic device. Detailed Implementation

[0018] The following disclosure contains specific information relating to embodiments described herein. The accompanying drawings and corresponding detailed disclosures are illustrative of exemplary embodiments. However, this disclosure is not limited to these exemplary embodiments. Other variations and embodiments of this disclosure will occur to those skilled in the art.

[0019] Unless otherwise indicated, similar or corresponding elements in the accompanying drawings are indicated by similar or corresponding reference numerals. The drawings and illustrations are generally not drawn to scale and are not intended to correspond to actual relative dimensions.

[0020] For the purposes of consistency and ease of understanding, similar features are identified by reference numerals in the exemplary drawings (although not shown in some examples). However, features in different embodiments may differ in other respects, and should not be narrowly limited to what is shown in the drawings.

[0021] This specification uses the phrases "in one embodiment" or "in some embodiments," which may refer to one or more of the same or different embodiments. The term "coupled" is defined as a connection, whether direct or indirect through intermediate components, and is not necessarily limited to a physical connection. The term "comprising" means "including but not limited to" and specifically indicates an open-ended inclusion or subordination relationship in the combination, group, series, or equivalents described herein.

[0022] For purposes of explanation rather than limitation, specific details (such as functional entities, technologies, protocols, and standards) are presented to provide an understanding of the disclosed technologies. Detailed descriptions of well-known methods, technologies, systems, and architectures are omitted to avoid unnecessary detail that could obscure the description.

[0023] Those skilled in the art will readily recognize that any one or more of the disclosed coding functions or algorithms described herein can be implemented by hardware, software, or a combination of both. The described functions may correspond to modules, which may be software, hardware, firmware, or any combination thereof.

[0024] Software implementations may include computer-executable instructions stored on a computer-readable medium, such as memory or other types of storage devices. For example, one or more microprocessors or general-purpose computers with communication processing capabilities may be programmed with executable instructions and perform one or more of the disclosed functions or algorithms.

[0025] Microprocessors or general-purpose computers may be formed using application-specific integrated circuits (ASICs), programmable logic arrays, and / or one or more digital signal processors (DSPs). Although some of the disclosed embodiments are geared towards software installed and executed on computer hardware, alternative implementations as firmware or hardware or a combination of hardware and software are also fully within the scope of this disclosure. Computer-readable media include, but are not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, compact disc read-only memory (CD ROM), magnetic cartridges, magnetic tapes, disk storage devices, or any other equivalent medium capable of storing computer-readable instructions.

[0026] Figure 1 A block diagram of a system 100 configured to encode and decode video data according to an embodiment of the present disclosure is shown. The system 100 includes a first electronic device 110, a second electronic device 120, and a communication medium 130.

[0027] The first electronic device 110 can be a source device, including any device configured to encode video data and transmit the encoded video data to the communication medium 130. The second electronic device 120 can be a destination device, including any device configured to receive and decode the encoded video data via the communication medium 130.

[0028] The first electronic device 110 can communicate with the second electronic device 120 via a communication medium 130, either wired or wirelessly. The first electronic device 110 may include a source module 112, an encoder module 114, and a first interface 116. The second electronic device 120 may include a display module 122, a decoder module 124, and a second interface 126. The first electronic device 110 may be a video encoder, and the second electronic device 120 may be a video decoder.

[0029] The first electronic device 110 and / or the second electronic device 120 may be a mobile phone, tablet computer, desktop computer, laptop computer or other electronic device. Figure 1 An example of a first electronic device 110 and a second electronic device 120 is shown. The first electronic device 110 and the second electronic device 120 may include more or fewer components than shown, or different configurations of various components.

[0030] Source module 112 may include a video capture device for capturing new video, a video archive for storing previously captured video, and / or a video feed interface for receiving video from a video content provider. Source module 112 may generate computer graphics-based data as source video, or generate a combination of live video, archived video, and computer-generated video as source video. The video capture device may be a charge-coupled device (CCD) image sensor, a complementary metal-oxide-semiconductor (CMOS) image sensor, or a camera.

[0031] Encoder module 114 and decoder module 124 can each be implemented as any of a variety of suitable encoder / decoder circuits, such as one or more microprocessors, central processing units (CPUs), graphics processing units (GPUs), systems-on-chip (SoCs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When partially implemented in software, the device may store instructions for software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the disclosed methods. In at least one embodiment, each of encoder module 114 and decoder module 124 may be included in one or more encoders and decoders, each of which may be integrated as part of a combined encoder / decoder (CODEC) in the device.

[0032] The first interface 116 and the second interface 126 may utilize custom protocols or comply with existing or de facto standards, including but not limited to Ethernet, IEEE 802.11 or IEEE 802.15 series, wireless USB, or telecommunications standards (including but not limited to Global System for Mobile Communications (GSM), Code-Division Multiple Access (CDMA) 2000, Time Division Synchronous Code Division Multiple Access (TD-SCDMA), Worldwide Interoperability for Microwave Access (WiMAX), Third Generation Partnership Project Long-Term Evolution (3GPP-LTE), or Time-Division LTE (TD-LTE)). The first interface 116 and the second interface 126 may each include any device configured to transmit and / or store compatible video bitstreams via the communication medium 130 and to receive compatible video bitstreams via the communication medium 130.

[0033] The first interface 116 and the second interface 126 may include computer system interfaces that enable the storage of or receipt of compatible video bitstreams from a storage device. For example, the first interface 116 and the second interface 126 may include chipsets supporting Peripheral Component Interconnect (PCI) and Peripheral Component Interconnect Express (PCIe) bus protocols, proprietary bus protocols, Universal Serial Bus (USB) protocols, I2C, or any other logical and physical architecture that can be used to interconnect peer devices.

[0034] Display module 122 may include a display that uses liquid crystal display (LCD) technology, plasma display technology, organic light emitting diode (OLED) display technology, or light emitting polymer display (LPD) technology, and in other embodiments, other display technologies may be used. Display module 122 may include a high-definition display or an ultra-high-definition display.

[0035] Figure 2 The embodiments according to this disclosure are shown. Figure 1 The diagram shows a block diagram of the decoder module 124 of the second electronic device 120. The decoder module 124 includes an entropy decoder (e.g., entropy decoding unit 2241), a prediction processor (e.g., prediction processing unit 2242), an inverse quantization / inverse transform processor (e.g., inverse quantization / inverse transform unit 2243), an adder (e.g., adder 2244), a filter (e.g., filter unit 2245), and a decoded image buffer (e.g., decoded image buffer 2246). The prediction processing unit 2242 further includes an intra-frame prediction processor (e.g., intra-frame prediction unit 22421) and an inter-frame prediction processor (e.g., inter-frame prediction unit 22422). The decoder module 124 receives a bitstream and decodes the bitstream to output the decoded video.

[0036] Entropy decoding unit 2241 can be from Figure 1 The second interface 126 receives a bitstream including multiple syntax elements and performs a parsing operation on the bitstream to extract the syntax elements. As part of the parsing operation, the entropy decoding unit 2241 can entropy decode the bitstream to generate quantized transform coefficients, quantization parameters, transform data, motion vectors, intra-frame modes, partition information, and other syntax information.

[0037] The entropy decoding unit 2241 can perform context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding technique to generate quantized transform coefficients. The entropy coding unit 2241 can provide the quantized transform coefficients, quantization parameters, and transform data to the inverse quantization / inverse transform unit 2243, and provide motion vectors, intra-frame modes, partitioning information, and other syntax information to the prediction processing unit 2242.

[0038] The prediction processing unit 2242 may receive syntax elements, such as motion vectors, intra-frame modes, partitioning information, and other syntax information, from the entropy coding unit 2241. The prediction processing unit 2242 may receive syntax elements including partitioning information and divide image frames according to the partitioning information.

[0039] Each of the image frames can be divided into at least one image block based on partitioning information. The at least one image block may include a luminance block for reconstructing multiple luminance samples and at least one chrominance block for reconstructing multiple chrominance samples. The luminance block and at least one chrominance block may be further subdivided to generate macroblocks, coding tree units (CTUs), coding blocks (CBs), their subdivision units, and / or another equivalent coding unit.

[0040] During the decoding process, the prediction processing unit 2242 may receive prediction data, which includes an intra-frame mode or motion vector for a current image block in a specific image frame. The current image block may be one of the luma blocks or chroma blocks in the specific image frame.

[0041] Intra-prediction unit 22421 may perform intra-prediction coding on the current block unit relative to one or more neighboring blocks in the same frame, based on syntax elements associated with the intra-frame mode, to generate a prediction block. The intra-frame mode may specify the location of reference samples selected from neighboring blocks in the current frame. In at least one embodiment, when chroma components are reconstructed by prediction processing unit 2242, intra-prediction unit 22421 may reconstruct multiple chroma components of the current block unit based on multiple luma components of the current block unit.

[0042] When the luminance component of the current block is reconstructed by the prediction processing unit 2242, the intra-prediction unit 22421 can reconstruct multiple chrominance components of the current block unit based on multiple luminance components of the current block unit.

[0043] Inter-frame prediction unit 22422 can perform inter-frame prediction coding on the current block unit relative to one or more blocks in one or more reference image blocks based on syntax elements associated with motion vectors in order to generate a prediction block.

[0044] The motion vector indicates the displacement of the current block cell within the current image patch relative to a reference block cell within the reference image patch. The reference block cell is determined to be a block that closely matches the current block cell.

[0045] The inter-frame prediction unit 22422 can receive a reference image block stored in the decoded image buffer 2246 and reconstruct the current block unit based on the received reference image block.

[0046] The inverse quantization / inverse transform unit 2243 can apply inverse quantization and inverse transform to reconstruct the residual block in the pixel domain. The inverse quantization / inverse transform unit 2243 can apply inverse quantization to the residual quantization transform coefficients to generate residual transform coefficients, and then apply inverse transform to the residual transform coefficients to generate the residual block in the pixel domain.

[0047] The inverse transform can be applied in reverse through transform processes such as the discrete cosine transform (DCT), discrete sine transform (DST), adaptive multiple transform (AMT), mode-dependent non-separable secondary transform (MDNSST), hypercube-givens transform (HyGT), signal-correlated transform, Karhunen-Loéve transform (KLT), wavelet transform, integer transform, subband transform, or conceptually similar transforms. The inverse transform can convert residual information from the transform domain, such as the frequency domain, back to the pixel domain. The degree of inverse quantization can be modified by adjusting the quantization parameters.

[0048] Adder 2244 adds the reconstruction residual block to the prediction block provided from prediction processing unit 2242 to produce a reconstruction block.

[0049] Filtering unit 2245 may include a deblocking filter, a sample adaptive offset (SAO) filter, a bilateral filter, and / or an adaptive loop filter (ALF) to remove block artifacts from the reconstructed blocks. In addition to the deblocking filter, SAO filter, bilateral filter, and ALF, other filters (in-loop or post-loop) may be used. For simplicity, such filters are not explicitly shown, but they may filter the output of adder 2244. After filtering unit 2245 performs the filtering process on the reconstructed blocks of a specific image frame, filtering unit 2245 may output the decoded video to display module 122 or other video receiving unit.

[0050] The decoding image buffer 2246 can be a reference image memory that stores reference blocks for use by the prediction processing unit 2242 when decoding the bitstream (in inter-frame coding mode). The decoding image buffer 2246 can be formed from any of a variety of memory devices, such as dynamic random-access memory (DRAM) (including synchronous DRAM (SDRAM)), magneto-resistive RAM (MRAM), resistive RAM (RRAM)), or other types of memory devices. The decoding image buffer 2246 can be on-chip with other components of the decoder module 124 or off-chip relative to those components.

[0051] Figure 3 A flowchart is shown of a method 300 for decoding video data via an electronic device according to an embodiment of the present disclosure. Method 300 is merely an example, as there are various methods to perform video data decoding.

[0052] Method 300 can be used Figure 1 and Figure 2 The configuration shown is used to execute the method, and reference is made to the various elements in these figures for method 300. Figure 3 Each box shown may represent one or more procedures, methods, or subroutines being executed.

[0053] Figure 3 The order of the boxes in this document is illustrative only and is subject to change. Additional boxes may be added or fewer boxes may be used without departing from this disclosure.

[0054] At box 310, decoder module 124 receives video data, which may be a bitstream.

[0055] refer to Figure 1 and Figure 2 The second electronic device 120 can receive bitstreams from an encoder, such as the first electronic device 110, or other video providers via a second interface 126. The second interface 126 can provide the bitstreams to the decoder module 124.

[0056] Entropy decoding unit 2241 can decode the bitstream to determine multiple prediction indicators and multiple partition indicators for multiple video images. Decoder module 124 can then reconstruct the multiple video images based on the prediction indicators and partition indicators. The prediction indicators and partition indicators may include multiple flags and multiple indices.

[0057] At box 320, decoder module 124 determines from the video data the affine enable flag corresponding to one or more image frames. Further reference Figure 1 and Figure 2 The decoder module 124 can determine one or more image frames from the bit stream and determine a first parameter set corresponding to one or more image frames.

[0058] The first parameter set may include multiple first prediction indicators and multiple first partition indicators. The bitstream may further include a second parameter set corresponding to one or more other image frames besides one or more image frames.

[0059] Both the first and second parameter sets can be one of the following: video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), and picture header (PH). For example, when the affine enable flag is included in the SPS, the affine enable flag can be the flag `sps_affine_enabled_flag`. When the first parameter set is an SPS, each of one or more image frames is reconstructed based on the SPS.

[0060] Decoder module 124 can determine affine enable flags corresponding to one or more image frames from a first parameter set of the bitstream. The affine enable flags are used to determine whether affine model-based motion compensation is enabled for one or more image frames. The block-based syntax in the bitstream may also include block-based affine flags used to determine whether a partitioned image block from one or more image frames uses affine model-based motion compensation when the affine enable flag is equal to 1. Affine model-based motion compensation may include affine merge prediction (having multiple affine merge candidates selected from multiple inherited affine merge candidates, multiple constructed affine merge candidates, and at least one zero motion vector (MV) candidate) and affine advanced motion vector prediction (AMVP) (having multiple affine AMVP candidates selected from multiple inherited affine AMVP candidates, multiple constructed affine AMVP candidates, translation MV candidates determined from neighboring blocks, and zero MV candidates).

[0061] At box 330, decoder module 124 determines whether the affine enable flag is true. When decoder module 124 determines that the affine enable flag is true, method 300 can proceed to box 340. When decoder module 124 determines that the affine enable flag is false, method 300 can proceed to box 350.

[0062] When the affine enable flag is equal to 1, the decoder module 124 determines that the affine enable flag is true. When the affine enable flag is equal to 0, the decoder module 124 determines that the affine enable flag is false.

[0063] When the affine enable flag is included in a VPS, it is the flag `vps_affine_enabled_flag`. When it is included in an SPS, it is the flag `sps_affine_enabled_flag`. When it is included in a PPS, it is the flag `pps_affine_enabled_flag`. When it is included in a PH, it is the flag `ph_affine_enabled_flag`.

[0064] At box 340, decoder module 124 determines the maximum index corresponding to one or more image frames from the video data. Further reference Figure 1 and Figure 2 When the affine enable flag is true, the decoder module 124 can determine the maximum index from the first parameter set corresponding to one or more image frames. Therefore, the maximum index also corresponds to one or more image frames. When the affine enable flag is included in the SPS, the maximum index is included in the SPS.

[0065] When the affine projection enable flag is true, the maximum index is used to compute the maximum number of more than zero subblock-based merged motion vector prediction (MVP) candidates. The maximum number can be represented as `MaxNumSubblockMergeCand`. The index value of the maximum index can be in the range of 0 to N1-K. The quantity N1 is the first integer, and the quantity K is the second integer less than N1. When the maximum index is included in the SPS, the maximum index is index `sps_N1_minus_max_num_subblock_merge_cand`. When the maximum index is included in the SPS and the quantity N1 equals 5, the maximum index is index `sps_five_minus_max_num_subblock_merge_cand`. When the maximum index is included in the PPS and the quantity N1 equals 6, the maximum index is index `pps_six_minus_max_num_subblock_merge_cand`.

[0066] The quantity K can be either 0 or 1. When the quantity K is 0, the maximum index value can be in the range of 0 to 5. When the quantity K is 1, the maximum index value can be in the range of 0 to 4.

[0067] At box 350, decoder module 124, for one or more image frames, determines the maximum number of more than zero subblock-based merged MVP candidates based on the subblock-based temporal MVP (sbTMVP) flag and the temporal MVP (TMVP) flag, without determining the maximum index from the video data.

[0068] When the affine enable flag is false, the maximum number is calculated based on the sbTMVP and TMVP flags without determining the maximum index from the video data. The sbTMVP flag can be determined from a first set of parameters corresponding to one or more image frames. Therefore, the sbTMVP flag can also correspond to one or more image frames.

[0069] When the affine enable flag is included in the SPS, the sbTMVP flag can be the flag sps_sbtmvp_enabled_flag included in the SPS. The TMVP flag can be determined from a parameter set at another level.

[0070] For example, when the TMVP flag is included in the image header corresponding to one of at least one or more image frames, the TMVP flag can be the flag ph_temporal_mvp_enabled_flag. Therefore, the TMVP flag can correspond to one of at least one or more image frames. One of the at least one or more image frames is reconstructed based on the image header corresponding to one of the at least one or more image frames.

[0071] The `sbTMVP` flag can be either 1 or 0, and the `TMVP` flag can also be either 1 or 0. When the affine enable flag is false, the maximum number `MaxNumSubblockMergeCand` can be equal to "sbTMVP flag && TMVP flag". The logical operator "&&" is the value of two Boolean logical "and" operations. Therefore, when the affine enable flag is false, and each of the `sbTMVP` and `TMVP` flags is equal to 1, the maximum number `MaxNumSubblockMergeCand` can be equal to 1.

[0072] In other words, when the affine enable flag is false, and at least one of the sbTMVP flag and the TMVP flag is equal to 0, the maximum number MaxNumSubblockMergeCand can be equal to 0. Therefore, when the affine enable flag is false, the maximum number MaxNumSubblockMergeCand can be derived as follows: MaxNumSubblockMergeCand = sps_sbtmvp_enabled_flag && ph_temporal_mvp_enable_flag

[0073] At box 360, decoder module 124 determines the maximum number based on the maximum index. When the affine enable flag is true, the maximum number can be calculated by subtracting the maximum index from the number N1. Therefore, when the affine enable flag is true and the number N1 equals 5, the maximum number can be calculated by subtracting the maximum index from 5. In other words, when the affine enable flag is true and the number N1 equals 5, the maximum number MaxNumSubblockMergeCand can be derived as follows:

[0074] MaxNumSubblockMergeCand=5-sps_five_minus_max_num_subblock_merge_cand

[0075] When the affine mapping enable flag is true, the maximum index value can be in the range of 0 to N1-K, and the maximum quantity can be calculated by subtracting the maximum index from the quantity N1. Therefore, the value of the maximum quantity is in the range of N1-(N1-K) to N1 (e.g., the range of K to N1). When the affine mapping enable flag is true and the quantity K equals 1, the value of the maximum quantity is in the range of 1 to N1, and the index value of the maximum index is in the range of 0 to N1-1. When the affine mapping enable flag is true and the quantity K equals 0, the value of the maximum quantity can be in the range of 0 to N1, and the index value of the maximum index is in the range of 0 to N1. Additionally, when the affine mapping enable flag is true, the quantity N1 equals 5, and the quantity K equals 1, the value of the maximum quantity is in the range of 1 to 5, and the index value of the maximum index is in the range of 0 to 4. When the affine mapping enable flag is true, the quantity N1 equals 5, and the quantity K equals 0, the value of the maximum quantity can be in the range of 0 to 5, and the index value of the maximum index is in the range of 0 to 5.

[0076] refer to Figure 3 The maximum number of blocks from 330 to 360, MaxNumSubblockMergeCand, can be derived as follows:

[0077]

[0078] Decoder module 124 can determine whether the maximum quantity is included within the value range. When decoder module 124 determines that the maximum quantity is excluded from the value range, decoder module 124 can determine that the video data does not conform to decoder module 124.

[0079] Therefore, decoder module 124 can output error signals for one or more image frames. In other words, when decoder module 124 determines that the maximum number is within the value range, decoder module 124 can determine that the video data conforms to decoder module 124. Therefore, decoder module 124 can decode the bitstream to reconstruct one or more image frames.

[0080] When the affine enable flag is true, decoder module 124 can determine whether the maximum index is included within the index range. When decoder module 124 determines that the maximum index is excluded from the index range, decoder module 124 can determine that the video data does not conform to decoder module 124. Therefore, decoder module 124 can output an error signal for one or more image frames. The above determination can be one of the bitstream conformance requirements in video coding standards (such as: Versatile Video Coding (VVC)).

[0081] In other words, when decoder module 124 determines that the maximum index is contained within the index range, decoder module 124 can determine that the video data conforms to decoder module 124. Therefore, decoder module 124 can decode the bitstream to reconstruct one or more image frames.

[0082] At box 370, decoder module 124 reconstructs one or more image frames based on the maximum number of zero or more sub-block-based merged MVP candidates. Further reference. Figure 1 and Figure 2The decoder module 124 can select a prediction mode for each of multiple image blocks in one or more image frames from multiple mode candidates, including zero or more sub-block-based merged MVP candidates, and determine multiple prediction blocks based on the prediction modes. The number of zero or more sub-block-based merged MVP candidates for each image block in one or more image frames is limited to a maximum number. Therefore, the decoder module 124 reconstructs one or more image frames based on the maximum number of zero or more sub-block-based merged MVP candidates. Zero or more sub-block-based merged MVP candidates may include zero or more of multiple sbTMVP candidates, multiple affine merged prediction candidates, and at least one zero-padding candidate. The decoder module 124 can determine multiple residual components from the bitstream for each image block and add the residual components to a corresponding prediction block to reconstruct the image block. The decoder module 124 can reconstruct all image frames to reconstruct a video image.

[0083] In video coding standards such as High Efficiency Video Coding (HEVC) and VVC, multiple first search candidates for subblock-based merging patterns, including subblock-based merging MVP candidates, can be more efficient than multiple second search candidates predicted by affine AMVP. However, the number of first search candidates for subblock-based merging patterns can be limited by a maximum number, MaxNumSubblockMergeCand. Therefore, due to encoder complexity, when motion compensation based on affine patterns is enabled, the maximum number MaxNumSubblockMergeCand can be set to ensure that subblock-based merging patterns are not disabled by the maximum number. Thus, when the number K is set to be greater than 0, the maximum number MaxNumSubblockMergeCand will be greater than 1. There will be no condition where the maximum number equals 0 to disable subblock-based merging patterns; therefore, the signaling of the maximum index will not be useless or wasted.

[0084] To reduce complexity, a sub-block-based merge mode can be used when the affine mapping enable flag is true. When the affine mapping enable flag is true, the quantity K can be equal to 1 to change the range of values ​​for the maximum quantity MaxNumSubblockMergeCand. However, when the sub-block-based merge mode is disabled because any other method has been disabled, it is not necessary to retain candidates for the sub-block-based merge MVP. Therefore, when the sub-block-based merge mode is disabled, the quantity K can be equal to 0 to maintain the value range. Thus, the quantity K can be set for the sub-block-based merge mode.

[0085] Figure 4A flowchart is shown of a method 400 for decoding video data via an electronic device according to an embodiment of the present disclosure. Method 400 is merely an example, as there are various methods to perform video data decoding.

[0086] Method 400 for decoding bitstreams and reconstructing block units can be used Figure 1 and Figure 2 The configuration shown is used to execute the method, and reference is made to the various elements in these figures for method 400. Figure 4 Each box shown may represent one or more procedures, methods, or subroutines being executed.

[0087] Figure 4 The order of the boxes in this document is illustrative only and is subject to change. Additional boxes may be added or fewer boxes may be used without departing from this disclosure.

[0088] At box 410, decoder module 124 receives video data, which may be a bitstream. (See reference) Figure 1 and Figure 2 The second electronic device 120 can receive bitstreams from an encoder, such as the first electronic device 110, or other video providers via a second interface 126. The second interface 126 can provide the bitstreams to the decoder module 124.

[0089] At box 420, decoder module 124 determines the maximum index corresponding to one or more image frames from the video data. Further reference Figure 1 and Figure 2 The decoder module 124 can determine one or more image frames from the bitstream and determine a first parameter set corresponding to the one or more image frames. The bitstream may further include a second parameter set corresponding to one or more other image frames besides the one or more image frames. Both the first parameter set and the second parameter set can be one of VPS, SPS, PPS, and PH.

[0090] Decoder module 124 can determine the maximum index from a first parameter set corresponding to one or more image frames. Therefore, the maximum index can also correspond to one or more image frames. When the first parameter set is an SPS (Segmented Frames), the maximum index can be included in the SPS.

[0091] The maximum index is used to calculate the maximum number of merged motion vector prediction (MVP) candidates with zero or more values. The maximum number can be represented as MaxNumMergeCand. The index value of the maximum index can be in the range of 0 to N², where N² is an integer.

[0092] When the maximum index is included in the SPS, the maximum index can be index `sps_N2_minus_max_num_merge_cand`. When the maximum index is included in the SPS and the quantity N2 equals 6, the maximum index can be index `sps_six_minus_max_num_merge_cand`. When the maximum index is included in the PPS and the quantity N2 equals 5, the maximum index can be index `pps_five_minus_max_num_merge_cand`.

[0093] At box 430, decoder module 124 determines the maximum number of merged MVP candidates with more than zero values ​​based on the maximum index. This maximum number corresponds to one or more image frames, and the value of this maximum number is in the range including 0.

[0094] The maximum quantity can be calculated by subtracting the maximum index from the quantity N2. Therefore, when the quantity N2 equals 6, the maximum quantity can be calculated by subtracting the maximum index from 6. In other words, when the quantity N2 equals 6, the maximum quantity MaxNumMergeCand can be derived as follows:

[0095] MaxNumMergeCand=6-sps_six_minus_max_num_merge_cand

[0096] The maximum quantity's index value can be in the range of 0 to N². Therefore, the maximum quantity's value can be in the range of 0 to N², and the value range includes 0. When the quantity N² equals 6, the maximum quantity's value can be in the range of 0 to 6.

[0097] Decoder module 124 can determine whether the maximum quantity is included within the value range. When decoder module 124 determines that the maximum quantity is excluded from the value range, decoder module 124 can determine that the video data does not conform to decoder module 124. Therefore, decoder module 124 can output an error signal for one or more image frames.

[0098] Decoder module 124 can determine whether the maximum index is included within the index range. When decoder module 124 determines that the maximum number is excluded from the value range, decoder module 124 can determine that the video data does not conform to decoder module 124. Therefore, decoder module 124 can output an error signal for one or more image frames.

[0099] At box 440, decoder module 124 reconstructs one or more image frames based on a maximum number. Further reference. Figure 1 and Figure 2The decoder module 124 can select a prediction mode for each of multiple image blocks in one or more image frames from multiple mode candidates, including zero or more merged MVP candidates, and determine multiple prediction blocks based on the prediction modes. The zero or more merged MVP candidates can include zero or more of multiple spatial MVP candidates, multiple temporal MVP candidates, multiple history-based MVP candidates, multiple pairwise average MVP candidates, and multiple zero MV candidates. Spatial MVP candidates can be determined from the motion vectors of spatially adjacent blocks, temporal MVP candidates can be determined from the motion vectors of collocated blocks, and history-based MVP candidates can be determined from a first-in-first-out (FIFO) list.

[0100] Decoder module 124 can determine multiple residual components from the bitstream for each image block and add the residual components to a corresponding prediction block to reconstruct the image block. Decoder module 124 can reconstruct all image frames to reconstruct the video image.

[0101] In video coding standards such as HEVC and VVC, when inter-frame prediction mode is enabled, regular merge mode, including merging MVP candidates, can also be enabled. However, the process of decoding and constructing merge candidates can be more complex than the process used to decode regular inter-frame mode. Therefore, pipeline latency is also an issue, and regular merge mode can be directly disabled to reduce pipeline latency.

[0102] The regular merge mode can be disabled directly using advanced syntax such as VPS, SPS, and PPS. When the value range includes 0, the maximum number can be equal to 0 to directly disable the regular merge mode. Therefore, disabling the regular merge mode in one or more image frames is only allowed by the maximum index, because the first set of parameters including the maximum index corresponds to one or more image frames.

[0103] Figure 5 A flowchart is shown of a method 500 for decoding video data via an electronic device according to an embodiment of the present disclosure. Method 500 is merely an example, as there are various methods to perform video data decoding.

[0104] Method 500 for decoding bitstreams and reconstructing block units can be used Figure 1 and Figure 2 The configuration shown is used to execute the method, and various elements in these figures are referenced for method 500. Figure 5Each box shown may represent one or more procedures, methods, or subroutines being executed.

[0105] Figure 5 The order of the boxes in this document is illustrative only and is subject to change. Additional boxes may be added or fewer boxes may be used without departing from this disclosure.

[0106] At box 510, decoder module 124 receives video data, which may be a bitstream. (See reference) Figure 1 and Figure 2 The second electronic device 120 can receive bitstreams from an encoder, such as the first electronic device 110, or other video providers via a second interface 126. The second interface 126 can provide the bitstreams to the decoder module 124.

[0107] At box 520, decoder module 124 determines from the video data the regular merging enable flag corresponding to one or more image frames. Further reference Figure 1 and Figure 2 The decoder module 124 can determine one or more image frames from the bitstream and determine a first parameter set corresponding to the one or more image frames. The first parameter set may include a plurality of first prediction indicators and a plurality of first partition indicators. The bitstream may further include a second parameter set corresponding to one or more other image frames besides the one or more image frames.

[0108] Both the first and second parameter sets can be one of VPS, SPS, PPS, and PH. For example, when the regular merge enable flag is included in SPS, the regular merge enable flag can be the flag sps_regular_merge_enabled_flag.

[0109] Decoder module 124 can determine the regular merge enable flag corresponding to one or more image frames from the first parameter set of the bitstream. The regular merge enable flag is used to determine whether the regular merge mode is enabled for one or more image frames. When the regular merge enable flag is equal to 1, the block-based syntax in the bitstream can also include a block-based regular merge flag to determine whether a segment of an image block from one or more image frames uses the regular merge mode.

[0110] A typical merge pattern can include multiple merge MVP candidates. These candidates can include zero or more of the following: spatial MVP candidates, temporal MVP candidates, history-based MVP candidates, pairwise average MVP candidates, and zero zero MV candidates. Spatial MVP candidates can be determined from the motion vectors of spatially adjacent blocks, temporal MVP candidates can be determined from the motion vectors of co-located blocks, and history-based MVP candidates can be determined from a first-in-first-out (FIFO) list.

[0111] When the regular merge enable flag is absent, it is inferred to be equal to 0. In other words, when the regular merge enable flag is absent, the decoder module 124 can determine that the regular merge enable flag is false.

[0112] At box 530, decoder module 124 determines whether the regular merge enable flag is true. When decoder module 124 determines that the regular merge enable flag is true, method 500 can proceed to box 540. When decoder module 124 determines that the regular merge enable flag is false, method 500 can proceed to box 550.

[0113] When the regular merge enable flag is equal to 1, the decoder module 124 determines that the regular merge enable flag is true. When the regular merge enable flag is equal to 0, the decoder module 124 determines that the regular merge enable flag is false. When the regular merge enable flag is included in the SPS, the regular merge enable flag can be the flag sps_regular_merge_enabled_flag.

[0114] At box 540, decoder module 124 determines the maximum index corresponding to one or more image frames from the video data. Further reference Figure 1 and Figure 2 When the regular merging enable flag is true, the decoder module 124 can determine the maximum index from the first parameter set corresponding to one or more image frames. Therefore, the maximum index also corresponds to one or more image frames. When the regular merging enable flag is included in the SPS, the maximum index is included in the SPS.

[0115] When the regular merge enabled flag is true, the maximum index is used to calculate the maximum number of merge MVP candidates with more than zero entries. The maximum number can be represented as MaxNumMergeCand.

[0116] The index value of the maximum index can be in the range of 0 to N3-P. The quantity N3 can be the first integer, and the quantity P can be the second integer less than the quantity N3.

[0117] When the maximum index is included in the SPS, the maximum index can be index sps_N3_minus_max_num_merge_cand. When the maximum index is included in the SPS and the quantity N3 equals 6, the maximum index can be index sps_six_minus_max_num_merge_cand. When the maximum index is included in the PPS and the quantity N equals 5, the maximum index can be index pps_five_minus_max_num_merge_cand.

[0118] The quantity P can be a positive integer. For example, the quantity P can be equal to 1. When the quantity P is equal to 1 or the quantity N3 is equal to 6, the index value of the maximum index can be in the range of 0 to 5.

[0119] At box 550, decoder module 124 determines the maximum number of zero or more merged MVP candidates equal to 0 for one or more image frames, without determining the maximum index from the video data.

[0120] When the regular merge enable flag is false, the maximum number can be directly set to 0 without determining the maximum index from the video data. When the regular merge enable flag is false, the regular merge mode is disabled in one or more image frames. Therefore, when the regular merge enable flag is false, all merge MVP candidates can be excluded from predicting all multiple blocks divided from one or more image frames, and the maximum number can be directly set to 0 without further analyzing any flags or indices from the first parameter set.

[0121] At box 560, decoder module 124 determines the maximum number based on the maximum index. When the regular merge enable flag is true, the maximum number can be calculated by subtracting the maximum index from the number N3. Therefore, when the regular merge enable flag is true and the number N3 equals 6, the maximum number can be calculated by subtracting the maximum index from 6. In other words, when the regular merge enable flag is true and the number N3 equals 6, the maximum number MaxNumMergeCand can be derived as follows:

[0122] MaxNumMergeCand=6-sps_six_minus_max_num_merge_cand

[0123] The maximum index value can be in the range of 0 to N3-P. Therefore, when the regular merge enable flag is true, the maximum quantity value is in the range of P to N3. When the regular merge enable flag is true, the quantity N3 equals 6, and the quantity P equals 1, the maximum quantity value is in the range of 1 to 6. Therefore, when the regular merge enable flag is true, the maximum quantity can be non-zero.

[0124] refer to Figure 5 The maximum number of elements in boxes 530 to 560, MaxNumMergeCand, can be derived as follows:

[0125]

[0126] Decoder module 124 can determine whether the maximum quantity is included within the value range. When decoder module 124 determines that the maximum quantity is excluded from the value range, decoder module 124 can determine that the video data does not conform to decoder module 124. Therefore, decoder module 124 can output an error signal for one or more image frames.

[0127] Decoder module 124 can determine whether the maximum index is included within the index range. When decoder module 124 determines that the maximum index is excluded from the index range, decoder module 124 can determine that the video data does not conform to decoder module 124. Therefore, decoder module 124 can output error signals for one or more image frames.

[0128] At box 570, decoder module 124 reconstructs one or more image frames based on the maximum number of zero or more merged MVP candidates.

[0129] refer to Figure 1 and Figure 2 The decoder module 124 can select a prediction mode from multiple mode candidates, including zero or more merged MVP candidates, for each of multiple image blocks in one or more image frames, and determine multiple prediction blocks based on the prediction mode. The decoder module 124 can determine multiple residual components from the bitstream for each image block and add the residual components to the corresponding prediction block to reconstruct the image block. The decoder module 124 can reconstruct all image frames to reconstruct the video image.

[0130] Regular merge mode can be disabled directly using advanced syntax such as VPS, SPS, and PPS to reduce pipeline latency. Therefore, advanced syntax can include a regular merge enable flag to determine whether to enable merge MVP candidates in one or more image frames. Furthermore, it is not necessary to include 0 in the maximum value range to disable regular merge mode, as it can be directly disabled using the regular merge enable flag. Therefore, the quantity P can be a positive integer, such as 1.

[0131] Figure 6 A flowchart is shown of a method 600 for decoding video data via an electronic device according to an embodiment of the present disclosure. Method 600 is merely an example, as there are various methods to perform this method.

[0132] Method 600 for decoding bitstreams and reconstructing block units can be used Figure 1 and Figure 2 The configuration shown is used to execute the method, and reference is made to the various elements in these figures for method 600. Figure 6 Each box shown may represent one or more procedures, methods, or subroutines being executed.

[0133] Figure 6 The order of the boxes in this document is illustrative only and is subject to change. Additional boxes may be added or fewer boxes may be used without departing from this disclosure.

[0134] At box 610, decoder module 124 receives video data, which may be a bitstream. (See reference) Figure 1 and Figure 2 The second electronic device 120 can receive bitstreams from an encoder, such as the first electronic device 110, or other video providers via a second interface 126. The second interface 126 can provide the bitstreams to the decoder module 124.

[0135] At box 620, decoder module 124 determines the intra-block copy (IBC) enable flag from the video data. Further reference Figure 1 and Figure 2 The decoder module 124 can determine one or more image frames from the bitstream and determine a first parameter set corresponding to the one or more image frames. The first parameter set can be one of VPS, SPS, PPS, and PH. For example, when the IBC enable flag is included in the SPS, the IBC enable flag can be the flag sps_ibc_enabled_flag.

[0136] Decoder module 124 can determine the IBC enable flag from the bitstream. Decoder module 124 can determine the IBC enable flag from a first parameter set in the bitstream. Because the first parameter set corresponds to one or more image frames, the IBC enable flag can also correspond to one or more image frames.

[0137] The IBC enable flag is used to determine whether the IBC prediction mode is enabled for one or more image frames. The IBC prediction mode can be implemented as a block-level coding mode. Therefore, block matching is performed at the encoder to find the optimal block vector for each of multiple image blocks divided from one or more image frames. The block vector is used to indicate the displacement from the current image block in the current image frame to a reference image block, which has been reconstructed in the current image.

[0138] At box 630, decoder module 124 determines whether the IBC enable flag is true. When decoder module 124 determines the IBC enable flag is true, or when decoder module 124 determines the IBC enable flag is false, method 600 can proceed to box 650.

[0139] When the IBC enable flag is equal to 1, the decoder module 124 determines that the IBC enable flag is true. When the IBC enable flag is equal to 0, the decoder module 124 determines that the IBC enable flag is false. When the IBC enable flag is included in the SPS, the IBC enable flag can be the flag sps_ibc_enabled_flag.

[0140] At frame 640, decoder module 124 determines the maximum index from the video data. (See reference) Figure 1 and Figure 2 When the IBC enable flag is true, the decoder module 124 can determine the maximum index from the video data. The maximum index can be contained in one of VPS, SPS, PPS, and PH.

[0141] When the IBC enable flag is true, the maximum index can be used to compute zero or more IBC merged block vector prediction (BVP) candidates. The maximum number can be represented as MaxNumIbcMergeCand.

[0142] The index value of the maximum index can be in the range of 0 to N4. The quantity N4 can be an integer.

[0143] When the maximum index is included in the SPS, the maximum index can be index sps_N4_minus_max_ibc_num_merge_cand. When the maximum index is included in the SPS and the number N equals 6, the maximum index can be index sps_six_minus_max_ibc_num_merge_cand. When the maximum index is included in the PPS and the number N equals 5, the maximum index can be index pps_five_minus_max_ibc_num_merge_cand.

[0144] At box 650, decoder module 124 determines the maximum number of IBC merge BVP candidates equal to zero or more, without determining the maximum index from the video data. The maximum number can be calculated by subtracting the maximum index from the number N4. Therefore, when the number N4 equals 6, the maximum number can be calculated by subtracting the maximum index from 6. In other words, when the number N4 equals 6, the maximum number MaxNumIbcMergeCand can be derived as follows:

[0145] MaxNumIbcMergeCand=6-sps_six_minus_max_ibc_num_merge_cand

[0146] The index value of the maximum index can be in the range of 0 to N4. Therefore, the value of the maximum quantity can be in the range of 0 to N4, and the range includes 0. When the quantity N4 equals 6, the value of the maximum quantity can be in the range of 0 to 6.

[0147] At box 660, decoder module 124 determines the maximum quantity based on the maximum index, and the maximum quantity has a quantity value in the range including 0. When the IBC enable flag is true, the maximum quantity can be calculated by subtracting the maximum index from the quantity N4. Therefore, when the IBC enable flag is true and the quantity N4 equals 6, the maximum quantity can be calculated by subtracting the maximum index from 6. In other words, the maximum quantity MaxNumIbcMergeCand can be derived as follows:

[0148] MaxNumIbcMergeCand=6-sps_six_minus_max_ibc_num_merge_cand

[0149] The maximum index value can be in the range of 0 to N4. Therefore, when the IBC enable flag is true, the maximum quantity value can be in the range of 0 to N4. Thus, when the IBC enable flag is true, the value range still includes 0. In one implementation, when the quantity N4 equals 6, the maximum quantity value can be in the range of 0 to 6.

[0150] refer to Figure 6 The maximum number of elements in boxes 630 to 660, MaxNumIbcMergeCand, can be derived as follows:

[0151]

[0152] Decoder module 124 can determine whether the maximum quantity is included within the value range. When decoder module 124 determines that the maximum quantity is excluded from the value range, decoder module 124 can determine that the video data does not conform to decoder module 124. Therefore, decoder module 124 can output an error signal.

[0153] When the IBC enable flag is true, decoder module 124 can determine whether the maximum index is included in the index range. When decoder module 124 determines that the maximum index is excluded from the index range, decoder module 124 can determine that the video data does not conform to decoder module 124. Therefore, decoder module 124 can output an error signal.

[0154] At box 670, decoder module 124 reconstructs the image blocks divided from the image frame based on the maximum number of blocks, corresponding to the maximum number of blocks.

[0155] Further reference Figure 1 and Figure 2 The decoder module 124 can select a prediction mode for a segmented image block from one or more image frames from a plurality of mode candidates, including zero or more IBC merged BVP candidates, and determine multiple prediction blocks based on the prediction mode. Zero or more IBC merged BVP candidates may include zero or more of a plurality of spatial IBC MVP candidates, a plurality of history-based MVP candidates, and a plurality of pairwise average MVP candidates.

[0156] Spatial MVP candidates can be determined from the motion vectors of spatially adjacent blocks, and history-based MVP candidates can be determined from a FIFO list. Decoder module 124 can determine multiple residual components from the bitstream for an image block and add the residual components to the prediction block to reconstruct the image block.

[0157] Decoder module 124 can reconstruct all image frames to reconstruct the video image. In video coding standards such as HEVC and VVC, IBC prediction modes can include IBC inter-frame modes and IBC merge modes with IBC merge candidates. However, the process of decoding and constructing IBC merge BVP candidates can be more complex than the process used to decode IBC inter-frame modes. Therefore, pipeline latency is also an issue, and IBC merge modes can be directly disabled to reduce pipeline latency.

[0158] IBC merge mode can be disabled directly using advanced syntax such as VPS, SPS, and PPS. When the value range includes 0, the maximum number can be allowed to be equal to 0 to directly disable IBC merge mode. Therefore, when both the maximum index and the IBC enable flag are included in the first parameter set, IBC merge mode can be disabled in one or more image frames using only one maximum index.

[0159] For example, both the maximum index and the IBC enable flag are included in the SPS. When the syntax level of the IBC enable flag is higher than that of the maximum index, the IBC merging mode can be disabled in one or more image frames by using different maximum indices. For example, the maximum index is included in the PPS, and the IBC enable flag is included in the SPS.

[0160] Figure 7 A flowchart is shown of a method 700 for decoding video data via an electronic device according to an embodiment of the present disclosure. Method 700 is merely an example, as various methods exist for decoding video data.

[0161] Method 700 for decoding bitstreams and reconstructing block units can be used Figure 1 and Figure 2 The configuration shown is used to execute the method, and reference is made to the various elements in these figures for method 700. Figure 7 Each box shown may represent one or more procedures, methods, or subroutines being executed.

[0162] Figure 7 The order of the boxes in this document is illustrative only and is subject to change. Additional boxes may be added or fewer boxes may be used without departing from this disclosure.

[0163] At box 710, decoder module 124 receives video data, which may be a bitstream. (See reference) Figure 1 and Figure 2 The second electronic device 120 can receive bitstreams from an encoder, such as the first electronic device 110, or other video providers via a second interface 126. The second interface 126 can provide the bitstreams to the decoder module 124.

[0164] At box 720, decoder module 124 determines from the video data a generic merge enable flag indicating whether all multiple merge modes are enabled. Further reference Figure 1 and Figure 2 The decoder module 124 can determine one or more image frames from the bitstream and determine a first parameter set corresponding to the one or more image frames. The first parameter set can be one of VPS, SPS, PPS, and PH.

[0165] Decoder module 124 can determine the general merge enable flag from the bitstream. Decoder module 124 can determine the general merge enable flag from a first parameter set in the bitstream. Because the first parameter set corresponds to one or more image frames, the general merge enable flag can also correspond to one or more image frames.

[0166] The general merge enable flag can be used to determine all merge modes enabled for one or more image frames. Merge modes can include regular merge mode, affine merge mode, IBC merge mode, and other merge modes.

[0167] When the general merge enable flag is false, all merge modes can be disabled in one or more merge modes. When the general merge enable flag is true, all merge modes can be enabled in one or more merge modes. When the general merge enable flag is true, zero or more merge modes can be disabled in one or more merge modes due to any other flag. For example, when the general merge enable flag is true, the IBC merge mode can be disabled due to the IBC enable flag.

[0168] When the general merge enable flag is not present, it is inferred to be equal to 0. In other words, when the general merge enable flag is not present, the decoder module 124 can determine that the general merge enable flag is inferred to be equal to 0.

[0169] At box 730, decoder module 124 determines the maximum number of specific merge candidates (more than zero) in a given merge mode based on a general merge enable flag. Further reference. Figure 1 and Figure 2 The decoder module 124 can determine the maximum index from a first parameter set corresponding to one or more image frames for a maximum number of frames. Therefore, the maximum index can also correspond to one or more image frames. When the first parameter set is SPS, the maximum index is included in the SPS. However, when the maximum index does not exist, the maximum index can be equal to the inferred value.

[0170] The maximum index is used to calculate the maximum number of a specific merge candidate with zero or more values ​​in a given merge pattern. The index value of the maximum index can be within an index range, therefore the maximum number can also be within a value range. Decoder module 124 can compare the maximum number to the value range, or compare the maximum index to the index range, to determine if the video data conforms to decoder module 124.

[0171] In one implementation, when the specific merge pattern is a regular merge pattern, the maximum index can be used to calculate the maximum number of merge MVP candidates with zero or more MVPs. The maximum number can be represented as MaxNumMergeCand. When the maximum index is included in the SPS, the maximum index can be the index sps_N5_minus_max_num_merge_cand. The number N5 can be a positive integer.

[0172] When the general merge enable flag is true, the decoder module 124 can determine the maximum index used for the regular merge mode from the video data. When the general merge enable flag is false, the decoder module 124 can determine the inferred value of the maximum index, which is equal to the number N5, without resolving the maximum index from the video data.

[0173] When the maximum index is included in the SPS and the quantity N5 equals 6, the maximum index can be the index sps_six_minus_max_num_merge_cand. The maximum number in the normal merge mode can be calculated by subtracting the maximum index from the quantity N5. Therefore, the maximum number MaxNumMergeCand can be derived as follows:

[0174]

[0175]

[0176] In one implementation, when the specific merge mode is affine merge, the maximum index is used to calculate the maximum number of subblock-based merge MVP candidates with zero or more subblocks. The maximum number can be represented as MaxNumSubblockMergeCand. When the maximum index is included in the SPS, the maximum index can be the index sps_N6_minus_max_num_subblock_merge_cand. The number N6 can be a positive integer.

[0177] When both the affine enable flag and the general merge enable flag are true, the decoder module 124 can determine the maximum index from the video data. When either the affine enable flag or the general merge mode is false, the decoder module 124 can determine an inferred value equal to the number N6 without resolving the maximum index. When the maximum index is included in the SPS and the number N6 equals 5, the maximum index of the affine merge mode can be the index sps_five_minus_max_num_subblock_merge_cand.

[0178] When the affine merge enable flag is true, the maximum number of affine merge patterns can be calculated by subtracting the maximum index from the total number N6. When the affine merge enable flag is false, the maximum number of affine merge patterns can be calculated based on the general merge enable flag, the subblock-based temporal MVP (sbTMVP) flag, and the temporal MVP (TMVP) flag. For example, the maximum number MaxNumSubblockMergeCand can be derived as follows:

[0179]

[0180] In one implementation, when the specific merge mode is IBC merge mode, the maximum index can be used to calculate the maximum number of IBC merge BVP candidates with zero or more merging capabilities. The maximum number can be represented as MaxNumIbcMergeCand. When the maximum index is included in the SPS, the maximum index can be the index sps_N7_minus_max_ibc_num_merge_cand. The number N7 can be a positive integer.

[0181] When the IBC enable flag and the universal merge enable flag are true, the decoder module 124 can determine the maximum index from the video data. When the IBC enable flag or the universal merge mode is false, the decoder module 124 can determine an inferred value equal to the number N7 without parsing the maximum index.

[0182] When the maximum index is included in the SPS and the quantity N7 equals 6, the maximum index for the IBC merge mode can be the index `sps_six_minus_max_ibc_num_merge_cand`. When the IBC enable flag is true, the maximum number of IBC merge modes can be calculated by subtracting the maximum index from the quantity N7. When the affine enable flag is false, the maximum number of IBC merge modes can be determined based on the general merge enable flag. For example, the maximum number `MaxNumIbcMergeCand` can be derived as follows:

[0183]

[0184] In another embodiment, decoder module 124 may determine more than one maximum number for different merging modes based on at least two of the first to third embodiments of method 700, using a general merge enable flag. For example, decoder module 124 may determine the maximum number for the regular merging mode and the maximum number for the IBC merging mode based on the general merge enable flag.

[0185] At box 740, decoder module 124 reconstructs image patches divided from image frames based on the maximum number of patches. Further reference. Figure 1 and Figure 2 Decoder module 124 selects a prediction mode from multiple mode candidates containing zero or more specific merging candidates for each of multiple image blocks in one or more image frames, and determines multiple prediction blocks based on the prediction mode. Decoder module 124 can determine multiple residual components from the bitstream for each image block and add the residual components to the corresponding prediction block to reconstruct the image block. Decoder module 124 can reconstruct all image frames to reconstruct a video image.

[0186] All merge modes can be disabled directly using advanced syntax such as VPS, SPS, and PPS to reduce pipeline latency. Therefore, advanced syntax can include a general merge enable flag to determine whether all merge modes are enabled in one or more image frames. Furthermore, the maximum number of each merge mode can be controlled using the general merge enable flag to further improve encoding efficiency.

[0187] Figure 8 The embodiments according to this disclosure are shown. Figure 1The diagram shows a block diagram of an encoder module 114 of a first electronic device. The encoder module 114 may include a prediction processor (e.g., prediction processing unit 8141), at least a first adder (e.g., first adder 8142) and a second adder (e.g., second adder 8145), a transform / quantization processor (e.g., transform / quantization unit 8143), an inverse quantization / inverse transform processor (e.g., inverse quantization / inverse transform unit 8144), a filter (e.g., filter unit 8146), a decoded image buffer (e.g., decoded image buffer 8147), and an entropy encoder (e.g., entropy encoding unit 8148). The prediction processing unit 8141 of the decoder module 114 may further include a partition processor (e.g., partition unit 81411), an intra-frame prediction processor (e.g., intra-frame prediction unit 81412), and an inter-frame prediction processor (e.g., inter-frame prediction unit 81413).

[0188] Encoder module 114 can receive source video and encode the source video to output a bitstream. Encoder module 114 can receive source video comprising multiple image frames, and then divide the image frames according to the encoding structure. Each image frame can be divided into at least one image block.

[0189] At least one image block may include a luminance block having multiple luminance samples and at least one chrominance block having multiple chrominance samples. The luminance block and at least one chrominance block may be further subdivided to generate macroblocks, coding tree units (CTUs), coding blocks (CBs), their subdivision units, and / or another equivalent coding unit.

[0190] Encoder module 114 can perform further subdivision of the source video. It should be noted that these implementations are generally applicable to video encoding, regardless of how the source video data is partitioned before and / or during encoding.

[0191] During the encoding process, the prediction processing unit 8141 may receive the current image block of a specific image frame. The current image block may be one of the luma blocks or chroma blocks in the specific image frame.

[0192] Partitioning unit 81411 can divide the current image block into multiple block units. Intra-frame prediction unit 81412 can perform intra-frame prediction coding of the current block unit relative to one or more adjacent blocks in the same frame as the current block unit to provide spatial prediction. Inter-frame prediction unit 81413 can perform inter-frame prediction coding of the current block unit relative to one or more blocks in one or more reference image blocks to provide temporal prediction.

[0193] The prediction processing unit 8141 can select one of the coding results generated by the intra-prediction unit 81412 and the inter-prediction unit 81413 based on a mode selection method such as a cost function. The mode selection method can be a rate-distortion optimization (RDO) process.

[0194] The prediction processing unit 8141 can determine the selected coding result and provide the prediction block corresponding to the selected coding result to the first adder 8142 for generating the residual block, and to the second adder 8145 for reconstructing the coding block unit. The prediction processing unit 8141 can also provide syntax elements such as motion vectors, intra-frame mode indicators, partition information and other syntax information to the entropy coding unit 8148.

[0195] Intra-prediction unit 81412 can perform intra-prediction on the current block unit. Intra-prediction unit 81412 can determine the intra-prediction mode for the reconstructed samples adjacent to the current block unit so as to encode the current block unit.

[0196] Intra-prediction unit 81412 can encode the current block unit using various intra-prediction modes, and either intra-prediction unit 81412 or prediction processing unit 8141 can select an appropriate intra-prediction mode from the selected modes. Intra-prediction unit 81412 can encode the current block unit using a cross-component prediction mode to predict one of the two chrominance components of the current block unit based on the luma component. Intra-prediction unit 81412 can predict the first of the two chrominance components of the current block unit based on the second of the two chrominance components.

[0197] As an alternative to intra-prediction performed by intra-prediction unit 81412, inter-prediction unit 81413 may perform inter-prediction on the current block unit. Inter-prediction unit 81413 may perform motion estimation to estimate the motion of the current block unit to generate motion vectors.

[0198] The motion vector indicates the displacement of a current block cell within the current image block relative to a reference block cell within the reference image block. The inter-frame prediction unit 81413 may receive at least one reference image block stored in the decoded image buffer 8147 and estimate the displacement based on the received reference image block to generate the motion vector.

[0199] The first adder 8142 generates a residual block by subtracting the prediction block determined by the prediction processing unit 8141 from the original current block unit. The first adder 8142 may represent one or more components that perform this subtraction operation.

[0200] The transform / quantization unit 8143 can apply a transform to the residual block to generate residual transform coefficients, and then quantize the residual transform coefficients to further reduce the bit rate. The transform can be one of DCT, DST, AMT, MDNSST, HyGT, signal correlation transform, KLT, wavelet transform, integer transform, subband transform, or a conceptually similar transform.

[0201] Transformation can convert residual information from the pixel value domain to the transform domain, such as the frequency domain. The degree of quantization can be modified by adjusting the quantization parameters.

[0202] The transform / quantization unit 8143 can perform a scan on a matrix including quantized transform coefficients. Alternatively, the entropy encoding unit 8148 can perform a scan.

[0203] The entropy coding unit 8148 can receive multiple syntax elements from the prediction processing unit 8141 and the transform / quantization unit 8143, including quantization parameters, transform data, motion vectors, intra-frame modes, partition information, and other syntax information. The entropy coding unit 8148 can encode the syntax elements into a bitstream.

[0204] The entropy coding unit 8148 can entropy-encode the quantized transform coefficients to generate a coded bitstream by performing CAVLC, CABAC, SBAC, PIPE coding, or another entropy coding technique. The coded bitstream can then be transmitted to another device. Figure 1 The second electronic device 120 shown in the figure may be archived for later transmission or retrieval.

[0205] The inverse quantization / inverse transform unit 8144 can apply inverse quantization and inverse transform to reconstruct a residual block in the pixel domain for later use as a reference block. The second adder 8145 can add the reconstructed residual block to the prediction block provided from the prediction processing unit 8141 to produce a reconstructed block for storage in the decoded image buffer 8147.

[0206] Filtering unit 8146 may include a deblocking filter, a SAO filter, a bilateral filter, and / or an ALF to remove blocking artifacts from the reconstructed block. In addition to the deblocking filter, SAO filter, bilateral filter, and ALF, other filters (in-loop or post-loop) may be used. For simplicity, such filters are not shown, and the output of the second adder 8145 may be filtered.

[0207] The decoding image buffer 8147 may be a reference image memory that stores reference blocks for the encoder module 114 to use for encoding video, such as in intra-frame or inter-frame coding modes. The decoding image buffer 8147 may include various memory devices, such as DRAM (including SDRAM, MRAM, RRAM) or other types of memory devices. The decoding image buffer 8147 may be on-chip with other components of the encoder module 114 or off-chip relative to those components.

[0208] Encoder module 114 can receive video data and use multiple prediction modes to predict multiple image frames within the video data. The prediction modes may include multiple merging modes with multiple merging candidates. Encoder module 114 can signal based on the prediction results. Figure 3 Affine activation mark in China Figure 5 The regular merge activation flag in the middle, Figure 6 The IBC enable flag and Figure 7 Use at least one of the common merge enable flags in the image frame to determine whether to enable the corresponding mode for one or more image frames.

[0209] Encoder module 114 can send signals based on the prediction results. Figures 3 to 7 The maximum index for methods 300 to 700 is used to determine the corresponding maximum number. Additionally, refer to... Figure 3 Encoder module 114 can signal the sub-block-based time MVP flag and time MVP flag to determine the maximum number of affine merging MVP candidates. Then, encoder module 114 can add the encoded data, including the flags and indexes, to the bitstream for provision to decoder module 124.

[0210] The described embodiments should be considered illustrative rather than restrictive in all respects. It should also be understood that this disclosure is not limited to the specific embodiments described above, but many rearrangements, modifications and substitutions are possible without departing from the scope of this disclosure.

Claims

1. A method for decoding a bitstream using an electronic device, characterized in that, The method includes: Receive the bit stream; Determine the affine enable flag corresponding to one or more image frames from the bitstream; When the affine enable flag is true, the maximum index corresponding to the one or more image frames is determined from the bitstream, wherein the index value of the maximum index is in the index range of 0 to NK, wherein N is a first integer and K is a second integer less than N; When the affine enable flag is true, based on the maximum index, determine the maximum number of zero or more sub-block-based merged motion vector prediction MVP candidates, wherein K equals 1 when the affine enable flag is true and one of the zero or more sub-block-based merged MVP candidates needs to be retained; and The one or more image frames are reconstructed based on the maximum number of zero or more sub-block-based merged MVP candidates.

2. The method according to claim 1, wherein, The value of N is 5.

3. The method according to claim 1, wherein, The method further includes: When the affine enable flag is false, the maximum quantity is determined based on the sub-block-based time MVP flag and the time MVP flag, instead of determining the maximum index from the bitstream.

4. The method according to claim 3, wherein, The affine enable flag, the sub-block-based time MVP flag, and the maximum index are included in the sequence parameter set SPS of the bitstream. The time MVP flag is included in the image header of the bitstream. Each of the one or more image frames is reconstructed based on the SPS, and At least one of the one or more image frames is reconstructed based on the image header.

5. The method according to claim 1, wherein, The maximum quantity is determined by subtracting the index value of the maximum index from the N.

6. The method according to claim 5, wherein, The maximum quantity is within the range of the quantity from K to N.

7. The method according to claim 5, wherein, When K is 1, the index value of the maximum index is in the index range of 0 to N-1, and the maximum quantity is in the quantity range of 1 to N.

8. An electronic device for decoding bitstreams, characterized in that, The electronic device includes: At least one processor; and A storage device, coupled to the at least one processor and storing a plurality of instructions, which, when executed by the at least one processor, cause the at least one processor to: Receive the bit stream; Determine the affine enable flag corresponding to one or more image frames from the bitstream; When the affine enable flag is true, the maximum index corresponding to the one or more image frames is determined from the bitstream, wherein the index value of the maximum index is in the index range of 0 to NK, wherein N is a first integer and K is a second integer less than N; When the affine enable flag is true, based on the maximum index, determine the maximum number of zero or more sub-block-based merged motion vector prediction MVP candidates, wherein K equals 1 when the affine enable flag is true and one of the zero or more sub-block-based merged MVP candidates needs to be retained; and The one or more image frames are reconstructed based on the maximum number of zero or more sub-block-based merged MVP candidates.

9. The electronic device according to claim 8, wherein, The value of N is 5.

10. The electronic device according to claim 8, wherein, The maximum quantity is determined by subtracting the index value of the maximum index from the N.

11. The electronic device according to claim 10, wherein, The maximum quantity is within the range of K to N.

12. The electronic device according to claim 10, wherein, When K is 1, the index value of the maximum index is in the index range of 0 to N-1, and the maximum quantity is in the quantity range of 1 to N.

13. The electronic device according to claim 10, wherein, When the affine enable flag is false, the maximum quantity is determined based on the sub-block-based time MVP flag and the time MVP flag, instead of determining the maximum index from the bitstream.

14. A method for decoding a bitstream using an electronic device, characterized in that, The method includes: Receive the bit stream; Determine the maximum index corresponding to one or more image frames from the bitstream, wherein the index value of the maximum index is in the index range of 0 to NK, and N is a first integer, and K is a second integer less than N; The maximum number of sub-block-based merged motion vector prediction MVP candidates is determined by subtracting the index value of the maximum index from the N, wherein, Determine the affine enable flag corresponding to the one or more image frames from the bitstream, and K equals 1 when the affine enable flag is true and one of the zero or more sub-block-based merge MVP candidates must be retained; and The one or more image frames are determined based on the maximum number of zero or more sub-block-based merged MVP candidates.

15. The method according to claim 14, wherein, When K is 1, the index value of the maximum index is in the index range of 0 to N-1, and the maximum quantity is in the quantity range of 1 to N.

16. The method of claim 14, wherein, When the affine enable flag is true, the maximum index is determined from the bitstream. When the affine enable flag is false, the maximum quantity is determined based on the sub-block-based time MVP flag and the time MVP flag, instead of determining the maximum index from the bitstream.

Images