Video encoding and decoding

By optimizing the construction and sorting of the motion vector predictor candidate list in the video encoding standard VVC, the problems of low encoding efficiency and high complexity in the prior art are solved, and more efficient video encoding is achieved.

CN119013966APending Publication Date: 2024-11-22CANON KK
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202380033208.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-04-11
Filing Date
2023-04-11
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The composition and order of the motion vector predictor candidate list in the existing video encoding standard VVC has a significant impact on encoding efficiency, but the existing methods are difficult to optimize effectively, resulting in increased bit rate and increased complexity.

Method used

By adding and reordering motion vector predictor candidates to the motion vector predictor candidate list, excluding duplicate candidates, sorting based on the relative cost of candidates, to optimize the list structure and reduce bit rate and complexity.

Benefits of technology

Improve encoding efficiency and reduce complexity. By effectively sorting and filtering candidates, redundant information in the bitstream is reduced and video encoding performance is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119013966A_ABST
    Figure CN119013966A_ABST
Patent Text Reader

Abstract

Improvements in derivation and ordering of motion vector predictor candidates for predicting motion in an image portion are disclosed. A list of motion vector predictor candidates for predicting motion in an image portion is generated, in which a first set of motion vector predictor candidates is added to the list, and if the number of the first set of motion vector predictor candidates is lower than a maximum number of candidates, the motion vector predictor candidates are added to the list. If so, a second set of motion vector predictor candidates is added to the list such that the total number of candidates is equal to the maximum number of candidates. The list of candidates is reordered, and candidates of the second set are excluded from the reordering.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to video encoding and decoding. Background Art

[0002] The Joint Video Experts Group (JVET), a collaborative team consisting of MPEG and VCEG of ITU-T Study Group 16, has released a new video coding standard called Versatile Video Coding (VVC). The goal of VVC is to provide a significant improvement in compression performance over the existing HEVC standard (i.e., typically twice as good as before). Key target applications and services include, but are not limited to, 360-degree and high dynamic range (HDR) video. Particular results were shown on ultra-high definition (UHD) video test material. Therefore, for the final standard, we can expect compression efficiency improvements far exceeding the targeted 50%.

[0003] Since the end of the VVC v1 standard, JVET has started the exploration phase by building the Exploration Tool (ECM). JVET collects additional tools and improves existing tools based on the VVC standard to achieve better coding efficiency.

[0004] Compared to HEVC, VVC has, among other modifications, a modified set of "merge modes" for motion vector prediction, which achieves better coding efficiency at the cost of greater complexity. Motion vector prediction is achieved by deriving a list of "motion vector predictor candidates", where the index of the selected candidate is signaled in the bitstream. The merge candidate list is generated for each coding unit (CU). But the CU can be split into smaller blocks for decoder-side Motion Vector Refinement (DMVR) or other methods.

[0005] The composition and order of the list can have a significant impact on coding efficiency, since an accurate motion vector predictor reduces the size of the residual or distortion of the block predictor, and having such a candidate at the top of the list reduces the number of bits required to signal the selected candidate. The present invention is directed to improving at least one of these aspects.

[0006] Modifications incorporated into VVC v1 and ECM mean that there can be up to 10 motion vector predictor candidates; this achieves diversity in the candidates, but the bit rate may increase if a candidate lower in the list is selected. The present invention broadly relates to improvements in the ordering of candidates in a list of motion vector predictor candidates. In particular, the computational cost of a candidate does not represent an instance in which the candidate is likely to be selected. Reordering the list can result in improved coding efficiency, and efficient methods of reordering can result in reduced complexity. Various embodiments of the present invention achieve one or both of these advantages.

[0007] According to one aspect of the invention, there is provided a method for generating a list of motion vector predictor candidates for predicting motion in an image portion, the method comprising: adding a first set of motion vector predictor candidates to the list; in the event that the number of motion vector predictor candidates of the first set is less than a maximum number of candidates, adding a second set of motion vector predictor candidates to the list such that the total number of candidates is equal to the maximum number of candidates; reordering the list of candidates, wherein the candidates of the second set are excluded from the reordering. Optionally, the first set does not include duplicates.

[0008] In another aspect of the invention, there is provided a method for generating a list of motion vector predictor candidates for predicting motion in an image portion, the method comprising: adding a first set of motion vector predictor candidates to the list; determining duplicate candidates in the list; in the event that the number of motion vector predictor candidates of the first set is less than a maximum number of candidates, adding a second set of motion vector predictor candidates to the list such that the total number of candidates is equal to the maximum number of candidates; reordering the list of candidates, wherein the duplicate candidates are excluded from the reordering. Optionally, the second set of candidates are also excluded from the reordering.

[0009] The following statements may apply to any of the above mentioned aspects:

[0010] Optionally, the second set of motion vector predictors are zero candidates, and the candidates of the first set are not zero candidates.

[0011] The first set of candidates may be candidates derived from at least one previously decoded or encoded motion information. Optionally, the first set of candidates are candidates derived from true samples including one or more of temporal candidates, spatial candidates, historical candidates, previously used candidates, and candidates derived from other candidates.

[0012] The re-ordering may be based on the calculated relative costs of the candidates. Optionally, at least one of the candidates is derived from at least one spatially or temporally matched template, wherein templates within a delimited region are available and templates outside the delimited region are not available, and in the event that at least one template is not available, a non-zero cost is calculated for the candidate.

[0013] In yet another aspect of the present invention, a method for generating a list of motion vector predictor candidates for predicting motion in an image portion is provided, the method comprising: adding a plurality of motion vector predictor candidates to the list; calculating a cost associated with at least one candidate in the list, wherein the at least one candidate is derived from at least one spatially or temporally matched template, wherein templates within a delimited area relative to the image portion are available and templates outside the delimited area are not available, and in the event that at least one template is not available, calculating a non-zero cost for the candidate and reordering the list according to the calculated cost.

[0014] The cost may be determined based on a size of one or more templates from which the candidates were derived.

[0015] The cost may be determined based on the number of samples of the available templates and the unavailable templates. For example, the cost may be determined based on the ratio of available samples to unavailable samples.

[0016] Optionally, the division used to determine the cost approximates a shift operation.

[0017] Optionally, the cost value for the unavailable template is set equal to the cost value of the associated available template.

[0018] Optionally, in the case where no template is available, the cost is set to a maximum value. For example, in the case where the number of added motion vector predictor candidates is lower than the maximum number of candidates, the candidates for which no associated template is available may be reordered in the list immediately above the added motion vector predictor candidates so that the total number of candidates is equal to the maximum number of candidates. Optionally, in the case where the number of added motion vector predictor candidates is lower than the maximum number of candidates, the maximum value is lower than the cost assigned to the added motion vector predictor candidates so that the total number of candidates is equal to the maximum number of candidates.

[0019] In an embodiment, the cost for the candidate is assigned a maximum value.

[0020] Optionally, the at least one candidate is a bidirectional candidate having two spatially matched templates, and when determining the cost of the bidirectional candidate, the cost of the available template is used instead of the cost of the unavailable template.

[0021] Optionally, the at least one candidate is a bidirectional candidate having two spatially matching templates, and the cost of the bidirectional candidate is calculated using only available templates.

[0022] In case a template is partially available, the cost can be calculated for the available samples, or the cost for the unavailable template corresponds to the cost of the associated available template.

[0023] The method may also include applying a weight to the calculated cost of the candidate if a template is not available.

[0024] In yet another aspect of the present invention, a method for generating a list of motion vector predictor candidates for predicting motion in an image portion is provided, the method comprising: adding a plurality of motion vector predictor candidates to the list, wherein at least one candidate in the list is derived from at least one spatially or temporally matched template, wherein templates within a delimited area relative to the image portion are available and templates outside the delimited area are not available, and reordering the list unless at least one template is not available.

[0025] Optionally, the reordering is performed unless all templates are unavailable.

[0026] In another aspect of the present invention, a method for generating a list of motion vector predictor candidates is provided, wherein the motion vector predictor candidates are used to predict motion in an image portion, the method comprising: determining the availability of at least one template related to the template of the image portion; and deriving the motion vector predictor candidates based on the availability of the template.

[0027] Optionally, in case at least one template is not available, the method comprises adding at least one further temporal candidate to the list.

[0028] Optionally, the number of temporal candidates is the same regardless of the availability of the templates.

[0029] Optionally, the method further comprises reducing the maximum number of candidates.

[0030] Optionally, the method further comprises reducing a motion vector threshold before adding the further temporal candidate.

[0031] The cost of a candidate in the list may be calculated based on a comparison metric between at least one sample associated with the candidate and at least one other sample. The cost for the candidate may be calculated based on a difference between neighboring samples of a predictor block and neighboring samples of a current block. Optionally, the cost for the candidate is calculated by calculating the difference between the predictors of two blocks. Optionally, the cost for the candidate is calculated by calculating the difference with another candidate in the list. Optionally, the other candidate is the most likely candidate. Optionally, the cost is based on samples or neighboring subsamples of the predictor. Optionally, the cost is based on samples corresponding to an image according to another resolution. Optionally, the values ​​of the samples used to calculate the cost are preprocessed.

[0032] Optionally, the cost corresponds to a distortion. The distortion may be SAD, SATD, SSE or SSIM.

[0033] Optionally, the cost comprises a weight. The weight may differ between motion vector predictor candidates. The method may further comprise deriving a variable corresponding to the number of motion vector predictor candidates in the first set, and reordering the list in dependence on the variable.

[0034] In the above aspects and embodiments, optionally, the variable identifies a first candidate from the second set of motion vector predictor candidates. Individual motion vector predictor candidates in the second set may be associated with variables, and the reordering is performed according to these variables. The method optionally includes setting the non-reordered motion vector predictor candidates to the end of the list.

[0035] The method may include performing a second reordering process on the non-reordered motion vector predictor candidates.

[0036] The method may comprise performing a second reordering process on the non-reordered motion vector predictor candidates when the first set contains no more than one candidate.

[0037] The method may include performing a second reordering process on the non-reordered motion vector predictor candidates according to a coding mode.

[0038] Optionally, the second reordering is not applied for the sub-block merging mode.

[0039] Optionally, the method further comprises: when the mode has a number of candidates greater than a threshold, performing a second reordering process on the non-reordered motion vector predictor candidates.

[0040] In another aspect of the present invention, there is provided a method for encoding image data into a bitstream, the method comprising the method for generating a list of motion vector predictor candidates according to any one of the above aspects and embodiments.

[0041] In another aspect of the present invention, there is provided a method for decoding image data from a bitstream, the method comprising the method for generating a list of motion vector predictor candidates according to any one of the above aspects and embodiments.

[0042] In another aspect of the present invention, there is provided an apparatus for encoding image data into a bitstream, the apparatus being configured to perform the method for generating a list of motion vector predictor candidates according to any one of the above aspects and embodiments.

[0043] In another aspect of the present invention, there is provided an apparatus for decoding image data from a bitstream, the apparatus being configured to perform the method for generating a list of motion vector predictor candidates according to any one of the above aspects and embodiments.

[0044] In another aspect of the invention, a computer program is provided which, when executed, causes a method according to any one of the above aspects and embodiments to be performed.The computer program may be stored on a computer readable carrier medium which may be transitory or non-transitory.

[0045] Further aspects of the invention relate to corresponding encoding methods, encoding devices, decoding devices and computer programs operable to perform the decoding and / or encoding methods of the invention.

[0046] The program may be provided independently or may be on, carried by or in a carrier medium. The carrier medium may be non-transitory, such as a storage medium, in particular a computer-readable storage medium. The carrier medium may also be transient, such as a signal or other transmission medium. The signal may be transmitted via any suitable network, including the Internet. Other features of the invention are characterized by independent and dependent claims.

[0047] Any features in one aspect of the invention may be applied to other aspects of the invention in any appropriate combination. In particular, method aspects may be applied to apparatus aspects, and vice versa.

[0048] Furthermore, features implemented in hardware may be implemented in software and vice versa. Any references herein to software and hardware features should be interpreted accordingly.

[0049] Any apparatus features as described herein may also be provided as method features, and vice versa.As used herein, means-plus-function features may alternatively be expressed in terms of their corresponding structure, such as a suitably programmed processor and associated memory.

[0050] It should also be understood that specific combinations of the various features described and defined in any aspect of the present invention may be independently implemented and / or provided and / or used.

[0051] Further aspects of the invention are provided by the independent and dependent claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Reference will now be made, by way of example, to the accompanying drawings, in which:

[0053] Figure 1 is a diagram for explaining the coding structure used in HEVC;

[0054] Figure 2 is a block diagram schematically illustrating a data communication system in which one or more embodiments of the present invention may be implemented;

[0055] Figure 3 is a block diagram illustrating components of a processing device that may implement one or more embodiments of the present invention;

[0056] Figure 4 is a flow chart showing the steps of an encoding method according to an embodiment of the present invention;

[0057] Figure 5 is a flow chart showing the steps of a decoding method according to an embodiment of the present invention;

[0058] Figure 6 and 7 shows a labeling scheme for describing blocks located relative to a current block;

[0059] Figure 8 (a) and (b) illustrate affine (sub-block) patterns;

[0060] Fig. 9 (a), (b), (c), (d) illustrate geometric patterns;

[0061] Fig.10 Illustrate the first step of deriving a merge candidate list of a VVC;

[0062] Fig.11 Illustrate additional steps of deriving a merge candidate list for a VVC;

[0063] Fig.12 Illustrate the derivation of pairwise candidates;

[0064] Fig.13 Demonstrate the template matching method based on adjacent samples;

[0065] Fig.14 Example pair Fig.10Modification of the first step of the merge candidate list derivation shown;

[0066] Fig.15 Example pair Fig.11 Modifications to the additional steps of deriving the merge candidate list shown;

[0067] Fig.16 Example pair Fig.12 Modification of the derivation of the pairs of candidates shown;

[0068] Fig.17 Cost determination of instantiated list candidates;

[0069] Fig.18 Illustrate a reordering process of a list of merge mode candidates;

[0070] Fig.19 illustrating pairwise candidate derivation during a reordering process of a list of merge mode candidates;

[0071] Fig. 20 Illustrate the derivation of the merge candidate list of the present invention;

[0072] Fig.21 An example of the reordering process of the list of merge mode candidates of the present invention is illustrated;

[0073] Fig. 22 Three examples of templates for candidates outside the region are illustrated;

[0074] Fig.23 Three examples of templates for the current block outside the region are illustrated;

[0075] Fig.24 An example of illustrating a template outside the region for bi-prediction;

[0076] Fig.25 is a diagram showing a system including an encoder or a decoder and a communication network according to an embodiment of the present invention;

[0077] Fig.26 is a schematic block diagram of a computing device for implementing one or more embodiments of the present invention;

[0078] Fig. 27 is a diagram illustrating a network camera system;

[0079] Fig.28 The figure illustrates an example of a smart phone. DETAILED DESCRIPTION

[0080] Figure 1The invention relates to a coding structure used in the High Efficiency Video Coding (HEVC) video standard and the Diversified Video Coding (VVC) standard. A video sequence 1 consists of a series of digital images i. Each of these digital images is represented by one or more matrices. The matrix coefficients represent pixels.

[0081] The images 2 of the sequence may be partitioned into slices 3. In some cases, a slice may constitute the entire image. The slices are partitioned into non-overlapping coding tree units (CTUs). The coding tree unit (CTU) is the basic processing unit of the High Efficiency Video Coding (HEVC) video standard and conceptually corresponds in structure to the macroblock unit used in several previous video standards. The CTU is sometimes also referred to as the largest coding unit (LCU). The CTU has luma and chroma component parts, each of which is called a coding tree block (CTB). These different color components are not in the same Figure 1 Shown in.

[0082] For HEVC, a CTU is typically 64 pixels by 64 pixels in size, while for VVC it ​​may be 128 pixels by 128 pixels. Each CTU may be iteratively partitioned into smaller, variable-sized coding units (CUs) 5 using a quadtree decomposition.

[0083] The coding unit is the basic coding element and consists of two subunits called prediction units (PUs) and transform units (TUs). The maximum size of a PU or TU is equal to the CU size. A prediction unit corresponds to a partition of a CU for prediction of pixel values. Various different partitions of a CU into PUs are possible, as shown in 606, including a partition into 4 square PUs, and two different partitions into 2 rectangular PUs. A transform unit is a basic unit for spatial transformation using DCT. A CU can be partitioned into TUs based on a quadtree representation 607.

[0084] Each slice is embedded in a network abstraction layer (NAL) unit. In addition, the coding parameters of the video sequence are stored in a dedicated NAL unit called a parameter set. In HEVC and H.264 / AVC, two parameter set NAL units are used: first, the sequence parameter set (SPS) NAL unit, which collects all parameters that remain unchanged during the entire video sequence. Generally, it handles the encoding profile, the size of the video frame, and other parameters. Second, the picture parameter set (PPS) NAL unit, which includes parameters that can be changed from one image (or frame) of the sequence to other images (or frames). HEVC also includes a video parameter set (VPS) NAL unit, which contains parameters describing the overall structure of the bitstream. VPS is a type of parameter set defined in HEVC and is applied to all layers of the bitstream. A layer can contain multiple temporal sublayers, and all version 1 bitstreams are limited to a single layer. HEVC has certain hierarchical extensions for scalability and multi-view, and these extensions will allow multiple layers with a backward compatible version 1 base layer.

[0085] Other ways of splitting an image have been introduced in VVC including sub-pictures, which are independently coded groups of one or more slices.

[0086] Figure 2 The data communication system in which one or more embodiments of the present invention can be implemented is illustrated. The data communication system includes a transmission device (in this case, a server 201) which is operable to transmit data packets of a data stream to a receiving device (in this case, a client terminal 202) via a data communication network 200. The data communication network 200 can be a wide area network (WAN) or a local area network (LAN). Such a network can be, for example, a wireless network (Wifi / 802.11a or b or g), an Ethernet network, an Internet network, or a hybrid network consisting of several different networks. In a specific embodiment of the present invention, the data communication system can be a digital television broadcasting system, in which the server 201 sends the same data content to multiple clients.

[0087] The data stream 204 provided by the server 201 may consist of multimedia data representing video and audio data. In some embodiments of the present invention, the audio and video data streams may be captured by the server 201 using a microphone and a camera, respectively. In some embodiments, the data streams may be stored on the server 201 or received by the server 201 from other data providers, or generated at the server 201. The server 201 is provided with an encoder for encoding the video and audio streams, in particular for providing a compressed bit stream for transmission, which is a more compact representation of the data presented as input to the encoder.

[0088] In order to obtain a better ratio of quality of transmitted data to the amount of transmitted data, the video data may be compressed, for example, according to the HEVC format or the H.264 / AVC format or the VVC format.

[0089] The client 202 receives the transmitted bitstream and decodes the reconstructed bitstream to reproduce the video image on the display device and reproduce the audio data using the speaker.

[0090] Despite Figure 2 A streaming scenario is considered in the example of , but it will be appreciated that in some embodiments of the invention, data communication between the encoder and the decoder may be carried out using, for example, a media storage device (such as an optical disc).

[0091] In one or more embodiments of the present invention, a video image is transmitted along with data representing a compensating offset to be applied to reconstructed pixels of the image to provide filtered pixels in the final image.

[0092] Figure 3 A processing device 300 configured to implement at least one embodiment of the present invention is schematically illustrated. The processing device 300 may be a device such as a microcomputer, a workstation, or a light portable device. The device 300 includes a communication bus 313, which is connected to:

[0093] - a central processing unit 311 denoted as CPU, such as a microprocessor or the like;

[0094] - a read-only memory 306, denoted ROM, for storing a computer program implementing the invention;

[0095] - a random access memory 312, represented as a RAM, for storing executable codes of the method of the embodiment of the present invention, and registers suitable for recording variables and parameters required for implementing the method for encoding a digital image sequence and / or the method for decoding a bit stream according to the embodiment of the present invention; and

[0096] A communication interface 302 connected to a communication network 303, via which digital data to be processed are transmitted or received.

[0097] Optionally, the device 300 may further include the following components:

[0098] - a data storage component 304, such as a hard disk, for storing a computer program for implementing the method of one or more embodiments of the present invention and data used or generated during the implementation of one or more embodiments of the present invention;

[0099] a disk drive 305 for a disk 306, which is suitable for reading data from the disk 306 or writing data to said disk;

[0100] - A screen 309 for displaying data and / or serving as a graphical interface for interaction with a user by means of a keyboard 310 or any other pointing means.

[0101] Device 300 may be connected to various peripheral devices such as digital camera 320 or microphone 308 , each of which is connected to an input / output card (not shown) to provide multimedia data to device 300 .

[0102] The communication bus provides communication and interoperability between the various elements included in or connected to the device 300. The representation of a bus is not limiting, and in particular, the central processing unit is operable to communicate instructions to any element of the device 300 directly or via other elements of the device 300.

[0103] The disk 306 may be replaced by any information medium, such as a rewritable or non-rewritable compact disk (CD-ROM), a ZIP disk or a memory card, and, in general, by an information storage component that can be read by a microcomputer or a microprocessor, the disk 306 being integrated into the device or not, possibly removable and suitable for storing one or more programs whose execution enables the implementation of the method for encoding a digital image sequence and / or the method for decoding a bit stream according to the present invention.

[0104] The executable code may be stored in a read-only memory 306, on a hard disk 304 or on a removable digital medium such as, for example, a disk 306 as previously described, etc. According to a variant, the executable code of the program may be received via the interface 302 by means of a communication network 303 to be stored in one of the storage means of the device 300, such as a hard disk 304, etc., before being executed.

[0105] The central processing unit 311 is adapted to control and direct the execution of instructions or parts of software code of one or more programs according to the invention, the execution of instructions stored in one of the above-mentioned storage means. At power-on, one or more programs stored in non-volatile memory (e.g., on hard disk 304 or in read-only memory 306) are transferred to random access memory 312 (which then contains the executable code of one or more programs) and registers for storing variables and parameters necessary for implementing the invention.

[0106] In this embodiment, the device is a programmable device that implements the invention using software. Alternatively, however, the invention may be implemented in hardware (for example in the form of an application specific integrated circuit or ASIC).

[0107] Figure 4Block diagram illustrating an encoder according to at least one embodiment of the invention. The encoder is represented by connected modules, each module being adapted to implement, e.g. in the form of programming instructions executed by the CPU 311 of the apparatus 300, at least one corresponding step of a method for implementing at least one embodiment of encoding an image in a sequence of images according to one or more embodiments of the invention.

[0108] The encoder 400 receives as input a raw sequence 401 of digital images i0 to in. Each digital image is represented by a collection of samples (sometimes also called pixels) (hereinafter, they are referred to as pixels).

[0109] After implementing the encoding process, the encoder 400 outputs a bitstream 410. The bitstream 410 includes a plurality of coding units or slices, each slice including a slice header for transmitting an encoding value of an encoding parameter used for slice encoding, and a slice body including encoded video data.

[0110] Module 402 divides the input digital images i0 to in 401 into blocks of pixels. Blocks correspond to image parts and can have variable sizes (e.g., 4×4, 8×8, 16×16, 32×32, 64×64, 128×128 pixels, and several rectangular block sizes can also be considered). A coding mode is selected for each input block. Two families of coding modes are provided: coding modes based on spatial prediction coding (intra-frame prediction) and coding modes based on temporal prediction (inter-frame coding, merge, skip). Possible coding modes are tested.

[0111] Module 403 implements an intra prediction process in which a given block to be encoded is predicted by a predictor calculated from neighboring pixels of the block to be encoded. If intra coding is selected, an indication of the selected intra predictor and the difference between the given block and its predictor is encoded to provide a residual.

[0112] Temporal prediction is implemented by a motion estimation module 404 and a motion compensation module 405. First, a reference image from a reference image set 416 is selected, and a portion of the reference image (also referred to as a reference region or image portion) is selected by the motion estimation module 404, which is the region closest to a given block to be encoded (in terms of pixel value similarity). The motion compensation module 405 then uses the selected region to predict the block to be encoded. The difference between the selected reference region and the given block (also referred to as a residual block) is calculated by the motion compensation module 405. The selected reference region is indicated using a motion vector.

[0113] Thus, in both cases (spatial and temporal prediction), the residual is calculated by subtracting the predictor from the original block.

[0114] In the intra prediction implemented by module 403, the prediction direction is encoded. In the inter prediction implemented by modules 404, 405, 416, 418, 417, at least one motion vector or data identifying such a motion vector is encoded for temporal prediction.

[0115] If inter prediction is selected, information about the motion vector and the residual block is encoded. To further reduce the bit rate, the motion is assumed to be homogeneous and the motion vector is encoded by the difference relative to the motion vector predictor. The motion vector predictor in the set of motion information predictor candidates is obtained from the motion vector field 418 by the motion vector prediction and encoding module 417.

[0116] The encoder 400 further includes a selection module 406 for selecting a coding mode by applying a coding cost criterion such as a rate-distortion criterion, etc. To further reduce redundancy, a transform such as DCT is applied to the residual block by a transform module 407, and then the obtained transform data is quantized by a quantization module 408 and entropy encoded by an entropy encoding module 409. Finally, the encoded residual block of the current block being encoded is inserted into a bitstream 410.

[0117] The encoder 400 also performs decoding of the encoded images to generate reference images (e.g., reference images in reference image / picture 416) for motion estimation of subsequent images. This enables the encoder and decoder receiving the bitstream to have the same reference frame (using reconstructed images or image portions). The inverse quantization (dequantization) module 411 performs inverse quantization (dequantization) of the quantized data, followed by an inverse transform by the inverse transform module 412. The intra prediction module 413 uses the prediction information to determine which predictor to use for a given block, and the motion compensation module 414 actually adds the residual obtained by module 412 to the reference area obtained from the reference image set 416.

[0118] Post filtering is then applied by module 415 to filter the reconstructed pixel frame (image or image portion). In an embodiment of the present invention, a SAO loop filter is used, wherein a compensation offset is added to the pixel values ​​of the reconstructed pixels of the reconstructed image. It should be understood that post filtering is not always necessary. Furthermore, any other type of post filtering may be performed instead of or in addition to SAO loop filtering.

[0119] Figure 5 A block diagram of a decoder 60 according to an embodiment of the present invention is shown, which can be used to receive data from an encoder. The decoder is represented by connected modules, each module being suitable for implementing the corresponding steps of the method implemented by the decoder 60, for example in the form of programming instructions to be executed by the CPU 311 of the device 300.

[0120] The decoder 60 receives a bitstream 61 including coding units (e.g., data corresponding to a block or a decoding unit), each coding unit consisting of a header containing information related to the encoded parameters and a body containing the encoded video data. Figure 4 As illustrated, for a given block, the coded video data is entropy encoded on a predetermined number of bits and the index of the motion vector predictor is encoded. The received coded video data is entropy decoded by module 62. The residual data is then dequantized by module 63, after which an inverse transform is applied by module 64 to obtain pixel values.

[0121] Mode data indicating the encoding mode is also entropy decoded, and based on the mode, the encoding block (unit / set / group) of the image data is intra-type decoded or inter-type decoded.

[0122] In the case of intra mode, intra prediction module 65 determines the intra predictor based on the intra prediction mode specified in the bitstream.

[0123] If the mode is inter, motion prediction information is extracted from the bitstream to find (identify) the reference region used by the encoder. The motion prediction information includes a reference frame index and a motion vector residual. The motion vector predictor is added to the motion vector residual by the motion vector decoding module 70 to obtain the motion vector. Figures 6 to 10 The various motion predictor tools used in VVC are discussed in more detail.

[0124] The motion vector decoding module 70 applies motion vector decoding to each current block encoded by motion prediction. Once the index of the motion vector predictor for the current block has been obtained, the actual value of the motion vector associated with the current block can be decoded and used to apply motion compensation by module 66. The reference image portion indicated by the decoded motion vector is extracted from the reference image 68 to apply motion compensation 66. The motion vector field data 71 is updated with the decoded motion vector for prediction of subsequently decoded motion vectors.

[0125] Finally, a decoded block is obtained. Where appropriate, post-filtering is applied by a post-filtering module 67. The decoder 60 finally obtains and provides a decoded video signal 69.

[0126] Motion prediction (inter) mode

[0127] HEVC uses 3 different inter modes: inter mode (Advanced Motion Vector Prediction (AMVP)), "classic" merge mode (i.e. "non-affine merge mode" or also called "regular" merge mode) and "classic" merge skip mode (i.e. "non-affine merge skip" mode or also called "regular" merge skip mode). The main difference between these modes lies in the signaling of data in the bitstream. For motion vector coding, the current HEVC standard includes a competition-based scheme for motion vector prediction, which did not exist in earlier versions of the standard. This means that several candidates compete with the rate-distortion criterion on the encoder side to find the best motion vector predictor or best motion information for inter or merge mode (i.e. "classic / regular" merge mode or "classic / regular" merge skip mode), respectively. The index corresponding to the best predictor or best candidate for the motion information is then inserted into the bitstream together with a "residual" representing the difference between the predicted value and the actual value. The decoder can derive the same set of predictors or candidates and use the best one according to the decoded index. Using the residual, the decoder can then recreate the original value.

[0128] In the picture content extension of HEVC, a new coding tool called intra block copy (IBC) is signaled as any of these three inter modes, and the difference between IBC and the equivalent inter mode is determined by checking whether the reference frame is the current frame. This can be achieved, for example, by checking the reference index of list L0, and if it is the last frame in the list, it is inferred that this is an intra block copy. Another way is to compare the picture order counts of the current frame and the reference frame: if they are equal, this is an intra block copy.

[0129] The design of the derivation of predictors and candidates is important to achieve optimal coding efficiency without a disproportionate impact on complexity. In HEVC, two motion vector derivations are used: one for inter mode (Advanced Motion Vector Prediction (AMVP)) and one for merge mode (Merge Derivation Process (for Classic Merge Mode and Classic Merge Skip Mode)). The various motion predictor modes used in VVC are described below.

[0130] Figure 6 The marking scheme used herein to describe the blocks located between frames relative to the current block (i.e., the block currently being encoded / decoded) is shown ( Figure 6 ).

[0131] VVC Merge Mode

[0132] Compared to HEVC, several inter modes have been added in VVC. In particular, a new merge mode has been added to the normal merge mode of HEVC.

[0133] Affine mode (sub-block mode)

[0134] In HEVC, only the translational motion model is applied for motion compensated prediction (MCP). In the real world, there are many kinds of motions, such as zoom-in / zoom-out, rotation, perspective motion, and other irregular motions.

[0135] In JEM, a simplified affine transform motion compensated prediction is applied, and the general principles of the affine mode are described below based on an extract from the document JVET-G1001 presented at the JVET conference held in Torino, July 13-21, 2017. This entire document is incorporated herein by reference insofar as it describes other algorithms used in JEM.

[0136] like Figure 8 As shown in (a), the affine motion field of the block is described by two control point motion vectors.

[0137] Affine mode is a motion compensation mode like inter mode (AMVP, "classic" merge or "classic" merge skip). Its principle is to generate one motion information per pixel based on 2 or 3 neighboring motion information. In JEM, affine mode derives one motion information for each 4×4 block, such as Figure 8 (a) (each square is a 4×4 block, and Figure 8 The entire block in (a) is a 16×16 block, which is partitioned into 16 blocks of such 4×4 sized squares (each 4×4 square block has a motion vector associated with it). By enabling the affine mode with a flag, the affine mode can be used in AMVP mode and merge mode (i.e., the classic merge mode also referred to as "non-affine merge mode" and the classic merge skip mode also referred to as "non-affine merge skip mode").

[0138] In the VVC specification, affine mode is also called sub-block mode; these terms are used interchangeably in this specification.

[0139] The sub-block merge mode of VVC contains a sub-block based temporal merge candidate that inherits the motion vector field of the block in the previous frame pointed to by the spatial motion vector candidate. If the neighboring block has been coded using the inter-frame affine mode of sub-block merging, and then some affine candidates are derived as constructed before a certain zero Mv candidate, then the sub-block candidate is followed by the inherited affine motion candidate.

[0140] CIIP

[0141] In addition to the normal merge mode and the sub-block merge mode, the VVC standard also includes a combined inter merge / intra prediction (CIIP), also known as the Multi-Hypothesis Intra Inter (MHII) merge mode.

[0142] Combined Inter Merge / Intra Prediction (CIIP) merging can be considered as a combination of the regular merge mode and the intra mode and is referred to below as Fig.10 The block predictor for the current block (1001) of this mode is the average between the merged predictor block and the intra predictor block, such as Fig.10 As shown. The merge predictor block is obtained using exactly the same processing as the merge mode, so it is a dual predictor for a time block (1002) or two time blocks. Therefore, the merge index is signaled for this mode in the same way as the conventional merge mode. The intra predictor block is obtained based on the neighboring samples (1003) of the current block (1001). However, the number of available intra modes for the current block is limited compared to the intra block. In addition, there is no chroma intra predictor block signaled for the CIIP block. The chroma predictor is equal to the luminance predictor. As a result, 1, 2 or 3 bits are used to signal the intra predictor for the CIIP block.

[0143] The CIIP block predictor is obtained by merging a weighted average of the block predictor and the intra block predictor. The weights of the weighted average depend on the block size and / or the selected intra predictor block.

[0144] The obtained CIIP predictor is then added to the residual of the current block to obtain the reconstructed block. It should be noted that the CIIP mode is enabled only for non-skipped blocks. In fact, skipping using CIIP usually results in a loss of compression performance and an increase in encoder complexity. This is because the CIIP mode usually has a block residual that is opposite to other skip modes. Therefore, its signal notification for the skip mode increases the bit rate. When the current CU is skipped, CIIP is avoided. The result of this restriction is that the CIIP block cannot have a residual containing only 0 values, because it is impossible to encode a VVC block residual equal to 0. In fact, in VVC, for merge mode, the only way to signal that the block residual is equal to 0 is to use skip mode, because for merge mode, the CU CBF flag is inferred to be equal to true. And when the CBF flag is true, the block residual cannot be equal to 0.

[0145] In this manner, in this specification, CIIP should be interpreted as a mode that combines inter-frame and intra-frame prediction features, and not necessarily a label given to a specific mode.

[0146] CIIP uses the same motion vector candidate list as regular merge mode.

[0147] MMVD

[0148] The MMVD merge mode is a specific regular merge mode candidate derivation. It can be considered as an independent merge candidate list. For the current CU, the selected MMVD merge candidate is obtained by adding the offset value of one motion vector component (mvx or mvy) to the initial regular merge candidate. The offset value is added to the motion vector of the first list L0 or the motion vector of the second list L1, depending on the configuration of these reference frames (both are backward, both are forward, or forward and backward). The initial merge candidate is signaled using an index. The offset value is signaled using a distance index between 8 possible distances (1 / 4 pixel, 1 / 2 pixel, 1 pixel, 2 pixels, 4 pixels, 8 pixels, 16 pixels, 32 pixels) and a direction index that gives the x or y axis and the offset sign.

[0149] In VVC, only the first two candidates of the regular merge list are used for MMVD derivation and signaled by a flag.

[0150] Geometric partitioning mode

[0151] The geometric (GEO) merge mode is a special bi-prediction mode. Fig. 9 exemplifies this particular block predictor generation. The block predictor contains one triangle (901 or 911) from the first block predictor and a second triangle (902 or 912) from the second block predictor. However, as Fig. 9 Several other possible splits of the block are possible, as described by (c) and (d). In this specification, geometric merging should be interpreted as a pattern of combining features of two inter non-square predictors, and not necessarily a label given to a specific pattern.

[0152] exist Fig. 9 In the example of (a), each partition (901 or 902) has a motion vector candidate as a unidirectional candidate. And for each partition, an index is signaled to obtain the corresponding motion vector candidate in the unidirectional candidate list at the decoder. And the first partition and the second partition cannot use the same candidate. The candidate list comes from a conventional merge candidate list, where for each candidate, one of the 2 components (L0 or L1) has been removed.

[0153] IBC

[0154] In VVC, the intra block copy (IBC) merge mode can also be enabled. IBC has an independent merge candidate derivation process.

[0155] Other motion information improvements

[0156] DMVR

[0157] Decoder-side motion vector derivation (DMVR) in VVC increases the accuracy of MVs in merge mode. For this method, decoder-side motion vector correction based on bilateral matching (BM) is applied. In this bi-prediction operation, the corrected MV is searched around the initial MV in reference picture list L0 and reference picture list L1. The BM method calculates the distortion between two candidate blocks in reference picture list L0 and list L1.

[0158] BDOF

[0159] VVC also integrates a bidirectional optical flow (BDOF) tool. BDOF (formerly known as BIO) is used to correct the dual prediction signal of a CU at the 4×4 sub-block level. BDOF is applied to a CU if the CU meets several conditions, in particular if the distances from two reference pictures to the current picture (i.e., the picture order count (POC) difference) are the same. As the name suggests, the BDOF mode is based on the concept of optical flow assuming that the motion of objects is smooth. For each 4×4 sub-block, the motion correction (v_x, v_y) is calculated by minimizing the difference between the L0 and L1 prediction samples. The motion correction is then used to adjust the dual prediction sample values ​​in the 4×4 sub-block.

[0160] PROF

[0161] Similarly, Prediction refinement with optical flow (PROF) is used for the affine mode.

[0162] AMVR and hpelIfIdx

[0163] VVC also includes adaptive motion vector resolution (AMVR). AMVR allows the motion vector difference of the CU to be encoded with different precisions. For example, for AMVP mode, consider a quarter brightness sample, half brightness sample, full brightness sample, or four brightness samples. The following table of the VVC specification gives the AMVR shift based on different syntax elements.

[0164]

[0165] AMVR may have an impact on the encoding of modes other than those that use motion vector difference encoding as a different merge mode. In fact, for some candidates, the parameter hpelIfIdx (which represents the index of the half-pixel precision luma interpolation filter) is propagated for some merge candidates. For example, for AMVP mode, hpelIfIdx is derived as follows:

[0166] hpelIfIdx=AmvrShift==3?1:0

[0167] Bi-prediction with CU-level weights (BCW)

[0168] In VVC, the bi-prediction mode with CU-level weights (BCW) is extended beyond simple averaging (as done in HEVC) to allow weighted averaging of two prediction signals P0 and P1 according to the following equation.

[0169] P bi-pred =((8-w)*P0+w*P1+4)>>3

[0170] Five weights are allowed in the weighted average bi-prediction, where w∈{-2, 3, 4, 5, 10}.

[0171] For a non-merged CU, the weight index bcwIndex is signaled after the motion vector difference.

[0172] For a merged CU, the weight index is inferred from neighboring blocks based on the merge candidate index.

[0173] BCW is only used for CUs with 256 or more luma samples. In addition, for low-latency pictures, all 5 weights are used. For non-low-latency pictures, only 3 weights (w∈{3, 4, 5}) are used.

[0174] General merge list export

[0175] In VVC, Fig.10 and Fig.11 Derive the regular merge list. First, add the spatial candidates B1(1002), A1(1006), B0(1010), A0(1014) (as Figure 7 and performing partial redundancy (1007) between the motion information of A1 and B1 to add A1 (1008), performing partial redundancy (1011) between the motion information of B0 and B1 to add B0 (1012), and performing partial redundancy (1015) between the motion information of A0 and A1 to add A0 (1016).

[0176] As merge candidates are added, the variable cnt is incremented (1015, 1009, 1013, 1017, 1023, 1027, 1115, 1108).

[0177] If the number of candidates in the list (cnt) is strictly less than 4 (1018), candidate B2 (1019) is added (1022) if it does not have the same motion information (1021) as A1 and B1.

[0178] Then add the temporal candidates. If the bottom right candidate (1024) is available (1025), add it (1026), otherwise if the center temporal candidate (1028) exists (1029), add the center temporal candidate (1026).

[0179] Then, the history-based (HMVP) is added (1101) if they do not have the same motion information as A1 and B1 (1103). In addition, the number of history-based candidates cannot exceed the maximum number of candidates in the merge candidate list minus 1 (1102). Therefore, in the merge candidate list, after the history-based candidate, at least one position is missing.

[0180] Then, if the number of candidates in the list is at least 2, a pair of candidates is constructed (1106) and added to the merge candidate list (1107).

[0181] Then, if there is an empty position in the merge candidate list (1109), a zero candidate is added (1110).

[0182] For spatial and history-based candidates, the parameters BCWidx and useAltHpelIf are set equal to the candidate's relevant parameters. For temporal and zero candidates, they are set equal to the default value of 0. These default values ​​essentially disable the method.

[0183] For paired candidates, BCCidx is set equal to 0, and hpelIfIdxp is set equal to the hpelIfIdxp of the first candidate (if the hpelIfIdxp of the first candidate is equal to the hpelIfIdxp of the second candidate, otherwise it is set to 0).

[0184] Paired Candidate Export

[0185] according to Fig.12 The algorithm constructs paired candidates (1106). As depicted, when 2 candidates are in the list (1201), hpelIfIdxp is derived as previously described (1204, 1202, 1203). The inter-frame direction (interDir) is then set equal to 0 (1205). For each list L0 and L1, if at least one reference frame is valid (different from -1) (1207), the parameter will be set. If both are valid (1208), the mv information of the candidate is derived (1209) and set equal to the reference frame of the first candidate, and the motion information is the average between the two motion vectors of the list, and the variable interDir is incremented. If only one of the candidates has motion information of the list (1210), the motion information of the paired candidates is set equal to the candidate (1212, 1211), and the inter-frame direction variable interDir is incremented.

[0186] ECM

[0187] Since the end of the VVC v1 standard, JVET has started the exploration phase by building the Exploration Software (ECM). The exploration phase collects additional tools and improves existing tools on top of the VVC standard to achieve better coding efficiency. Different additional tools compared to VVC are described in JVET-X2025.

[0188] ECM Merge Mode

[0189] Among all the tools added, some additional merge modes were added. Affine MMVD signals the offset of the merged affine candidate encoded by MVVD as the regular merge mode. Similarly, GEO MMVD was also added. CIIP PDPC is an extension of CIIP. And two template matching merge modes were added: regular template matching and GEO template matching.

[0190] Conventional template matching is based on template matching estimation, such as Fig.13 As depicted. At the decoder side, for the candidate corresponding to the relevant merge index and for the two lists (L0, L1) available, motion estimation is performed based on the neighboring samples of the current block (1301) and based on the neighboring samples of multiple corresponding block positions, the cost is calculated and the motion information that minimizes the cost is selected. The motion estimation is limited by the search range, and several restrictions on the search range are also used to reduce complexity.

[0191] In ECM, the regular template matching candidate list is based on the regular merge list, but with some additional steps and parameters added, which means that different merge candidate lists for the same block can be generated. In addition, only 4 candidates are available for the regular merge candidate list for template matching, compared to 10 candidates for the regular merge candidate list in ECM with the common test conditions defined by JVET.

[0192] Rule Merge List Export in ECM

[0193] In ECM, updated the general merge list export. Fig.14 and Fig.15 Shown based on Fig.10 and Fig.11 However, for clarity, the module of history-based candidates (1101) has been outlined in (1501).

[0194] In this Fig.15In , a new type of merge candidate is added: non-adjacent candidates (1540). These candidates come from blocks that are spatially located in the current frame rather than blocks in adjacent frames, because adjacent are spatial candidates. These candidates are selected based on distance and direction. For history-based candidates, a list of neighboring candidates can be added until the list reaches the maximum number of candidates minus 1, so that pairs of candidates can still be added.

[0195] Zero Candidate

[0196] If the list still has not reached the maximum number of candidates (Maxcand), a zero candidate is added to the list. Zero candidates are added based on possible reference frames or reference frame pairs. The following pseudo code gives the derivation of such candidates:

[0197]

[0198] This pseudo code can be summarized as: For each reference frame index (unidirectional) or reference index pair (bi-prediction), add a zero candidate. When all are added, only zero candidates with reference frame index 0 are added until the number of candidates reaches its maximum value. In this way, the merge list can include multiple zero candidates. In fact, it has been surprisingly found that this happens frequently in real video sequences, especially at the beginning of slices or frames and sequences.

[0199] In the latest modification of the derivation of merge candidates, the number of candidates in the list can be greater than the maximum number of candidates in the final list Maxcand. However, this number of candidates in the initial list (MaxCandInitialList) is used for derivation. Therefore, zero candidates are added until MaxCandInitialList, instead of until Maxcand.

[0200] BM Merge Mode

[0201] BM merge is a merge mode dedicated to the adaptive decoder-side motion vector refinement method, which is an extension of the multi-pass DMVR of ECM. As described in JVET-X2025, this mode is equivalent to 2 merge modes that refine the MV in only one direction. Therefore, one merge mode is for L0 and one merge mode is for L1. Therefore, BM merge is enabled only when DMVR conditions can be enabled. For these two merge modes, only one list of merge candidates is derived, and all candidates comply with DMVR conditions.

[0202] Merge candidates for BM merge mode are derived from spatially neighboring coding blocks, TMVP, non-neighboring blocks, HMVP, pairwise candidates in a similar manner as for regular merge mode. The difference is that only those that meet the DMVR condition are added to the candidates. The merge index is encoded in a similar manner as for regular merge mode.

[0203] AMVP Merge Mode

[0204] The AVMP merge mode (also called dual predictor) is defined in JVET-X2025 as follows: It consists of an AMVP predictor in one direction and a merge predictor in the other direction. This mode can be enabled for a coding block when the selected merge predictor and AMVP predictor meet the DMVR condition, where there is at least one reference picture from the past and one reference picture from the future relative to the current picture, and the distances from these two reference pictures to the current picture are the same, and bilateral matching MV refinement is applied to the merged MV candidate and AMVP MVP as the starting point. Otherwise, if the template matching function is enabled, the template matching MV refinement is applied to the merged predictor or AMVP predictor with a higher template matching cost.

[0205] The AMVP part of the mode is signaled as regular unidirectional AMVP, ie, the reference index and MVD are signaled, and if template matching is used, the AMVP part has the derived MVP index, or the MVP index is signaled when template matching is disabled.

[0206] For the AMVP direction LX (where X can be 0 or 1), the merged part in the other direction (1-LX) is implicitly derived by minimizing the bilateral matching cost between the AMVP predictor and the merge predictor (i.e., for the pair of AMVP and merge motion vector). For each merge candidate in the merge candidate list with the motion vector in the other direction (1-LX), the bilateral matching cost is calculated using the merge candidate MV and the AMVP MV. The merge candidate with the minimum cost is selected. Bilateral matching refinement is applied to the coding block with the selected merge candidate MV and AMVP MV as starting points.

[0207] The third pass of 8x8 sub-PU BDOF refinement as multi-pass DMVR in multi-pass DMVR is enabled for AMVP merge mode coding blocks.

[0208] The mode is indicated by a flag, and if the mode is enabled, the AMVP direction LX is further indicated by a flag.

[0209] MVD Sign Prediction

[0210] The symbol prediction method is described in JVET-X0132. If the motion vector difference contains a non-zero component, the motion vector difference symbol prediction can be applied in the conventional inter-frame mode. In the current ECM version, symbol prediction is applied for AMVP, affine MVD and SMVD modes. The possible MVD symbol combinations are sorted according to the template matching cost, and the index corresponding to the true MVD symbol is derived, and the index is encoded with the context model. On the decoder side, the MVD symbol is derived as follows: 1 / parse the size of the MVD component; 2 / parse the MVD symbol prediction index of the context encoding; 3 / construct MV candidates by creating a combination between possible symbols and absolute MVD values, and add them to the MV predictor; 4 / derive the MVD symbol prediction cost for each derived MV based on the template matching cost and sorting; 5 / use the MVD symbol prediction index to select the true MVD symbol; 6 / add the true MVD to the MV predictor of the final MV.

[0211] TIMD

[0212] Intra prediction (fusion for template-based intra mode derivation (TIMD)) is described in JVET-X2025 as follows: For each intra prediction mode in MPM, the SATD between the prediction sample and the reconstructed sample of the template is calculated. The first two intra prediction modes with the smallest SATD are selected as TIMD modes. These two TIMD modes are fused with weights after applying PDPC processing, and such weighted intra prediction is used to encode the current CU. Position-dependent intra prediction combination (PDPC) is included in the derivation of TIMD mode.

[0213] Duplicate Check

[0214] exist Fig.14 and Fig.15 In, a duplication check is added for each candidate (1440, 1441, 1442, 1443, 1444, 1445 and 1530). However, repetition is also for non-adjacent candidates (1540) and history-based candidates (1501). The duplication check consists in comparing the motion information of the current candidate of index cnt with the motion information of the previous candidate. When the motion information is equal, it is considered to be a duplicate and the variable cnt is not incremented. Of course, the motion information means the inter-frame direction, reference frame index and motion vector for each list (L0, L1). Note that zero candidates corresponding to different reference frames are not considered to be duplicates.

[0215] MVTH

[0216] In ECM, a motion vector threshold is added to this duplicate check. This parameter changes the equality check in the following way: two motion vectors are considered equal if their absolute difference for each component is less than or equal to the motion vector threshold MvTh. For normal merge mode, MvTh is set equal to 1, which corresponds to the traditional duplicate check without a motion vector threshold.

[0217] As defined below, for the normal merge mode of template matching, MvTh is equal to a value that depends on the number of luma samples (nbSamples) in the current CU:

[0218] if(nbSamples<64)MvTh=1< <MV_FRACTIONAL=16

[0219] else if(nbSamples<256)MvTh=2< <MV_FRACTIONAL=32

[0220] else MvTh=4< <MV_FRACTIONAL=64

[0221] Where MV_FRACTIONAL corresponds to the inter-frame resolution of the codec. Therefore, in the current ECM, the 16th resolution is used, so MV_FRACTIONAL is equal to 4. << is a left shift operator. And where nbSamples = Height × Width (the height and width of the current block).

[0222] For example, as described below, there is another threshold MvThBDMVRMvdThreshold used for GEO merge derivation and also for duplicate checking of non-adjacent candidates:

[0223] if(nbSamples<64)

[0224] MvThBDMVRMvdThreshold=(1<<MV_FRACTIONAL)> >2=4

[0225] else if(nbSamples<256)

[0226] MvThBDMVRMvdThreshold=(1<<MV_FRACTIONAL)> >1=8

[0227] else

[0228] MvThBDMVRMvdThreshold=(1<<MV_FRACTIONAL)> >0=16

[0229] AMRC

[0230] In ECM, in order to reduce the number of bits of the merge index, adaptive reordering of merge candidates with template matching (AMRC) is added. Fig.13 The candidates are reordered based on the cost of each candidate, using the template matching cost calculated in . In this method, only one cost is calculated for each candidate. This method is applied after the list is derived and is only applied to the first 5 candidates of the regular merge candidate list. It should be understood that the number 5 is selected to balance the complexity of the reordering process with the potential benefits, so that a larger number (e.g., all candidates) can be reordered.

[0231] Fig.18 An example of this method is given for a conventional merge candidate list containing 10 candidates as in CTC.

[0232] The method also applies to sub-block merging modes other than temporal candidates, and to regular TM mode for all 4 candidates.

[0233] In the proposal, the method is also extended to reorder and select candidates to be included in the final list of merge mode candidates. For example, in JVET-X0087, all possible non-adjacent candidates (1540) and history-based candidates (1501) are considered together with temporal non-adjacent candidates to obtain a list of candidates. The list of candidates is constructed without considering the maximum number of candidates. The list candidates are then reordered. Only the correct number of candidates from the list are added to the final list of merge candidates. The correct number of candidates corresponds to the first N candidates in the list. In this example, the correct number is the maximum number of candidates minus the number of spatial and temporal candidates already in the final list. In other words, non-adjacent candidates and history-based candidates are processed separately from adjacent spatial and temporal candidates. The processed list is used to supplement the adjacent spatial and temporal merge candidates already present in the merge candidate list to generate the final merge candidate list.

[0234] In JVET-X0091, ARMC is used to select a temporal candidate from 3 temporal candidates bi-dir, L0 or L1. The selected candidate is added to the merge candidate list.

[0235] In JVET-X0133, a merged temporal candidate is selected from several temporal candidates reordered using ARMC. In the same way, all possible neighboring candidates are subjected to ARMC, and up to 9 of these candidates can be added to the merge candidate list.

[0236] All of these proposed methods use classical ARMC to reorder the final list of merge candidates. JVET-X0087 reuses the costs computed during the reordering of non-adjacent and history-based candidates to avoid additional computational costs. JVET-X0133 applies systematic reordering to all candidates on the final list of merge candidates.

[0237] New AMRC

[0238] Since the first implementation of ARMC, the method has been added to several other modes. ARMC is additionally applied to regular and template matching merge modes, sub-block merge mode, and also to IBC, MMVD, affine MMVD, CIIP, CIIP with template matching, and BM merge modes. In addition, the principle of ARMC to re-order the list of candidates based on template matching cost is also applied to AMVP merge candidate derivation and for intra method TIMD to select the most likely predictor.

[0239] Furthermore, this principle is added for the signed residual prediction methods of the affine MVD, AMVP and SMVD methods.

[0240] For GEO merge mode and GEO with template matching and reference frame index prediction, there are additional tests for the use of this reordering.

[0241] Besides this almost systematic use of ARMC, there are several additional modifications to the derivation process for some candidates.

[0242] For example, ECM4.0 export includes cascaded AMRC processing for candidate export of conventional, TM and BM merge modes, such as Fig.19 as depicted in for conventional and TM merging modes.

[0243] Compared to the previous merge candidate derivation, first, 10 temporal positions are checked and added to the list of temporal candidates after non-repetitive checking. This temporal list can contain up to 9 candidates. Furthermore, since the MV threshold is always 1 in contrast to the motion threshold used for merge candidate derivation and does not depend on the merge mode, there is a specific threshold for this temporal list. Based on the list of up to the first 9 non-repetitive positions, the ARMC process is applied and the first temporal candidate is only added in the traditional list of merge candidates if it is non-repetitive compared to the previous candidate.

[0244] In the same way, non-adjacent spatial candidates are derived in 59 positions. A first non-repeating candidate list of up to 18 candidates is derived. But the motion threshold is different from the one used for temporal derivation and for the rest of the list and is independent of the regular or template merge mode. The motion threshold is set equal to the mv of the BDMVR. Up to 18 non-adjacent candidates are reordered and only the first 9 non-adjacent candidates are retained and added in the merge candidate list. Further candidates are then added unless the list already contains the maximum number of candidates. In addition, for the TM merge mode, the maximum number of candidates in the list MaxCandInitialList is higher than the maximum number of candidates Maxcand that the final list can contain.

[0245] Then, if Fig.19 As depicted, the ARMC process is applied to all candidates of the intermediate list including MaxCandInitialList. A maximum number of candidates Maxcand is set in the final list of candidates.

[0246] As for the template matching merge mode, similar modifications also apply to the BM merge mode.

[0247] ARMC template cost algorithm

[0248] Fig.17 The template cost calculation of the ARMC method is illustrated. The number of candidates considered in this process, NumMergeCandInList, is greater than or equal to the maximum number Maxcand (1712) that the list can contain.

[0249] For each candidate in the list (1701), if no cost was calculated during the first ARMC process for temporal and non-adjacent candidates (1702), the cost is set equal to 0 (1703). In an implementation, the uncalculated cost mergeList[i].cost associated with candidate "i" is set equal to the maximum value MAXVAL. If the upper template for the current block is available (1704), the distortion compared to the current block template is calculated (1705) and added to the current cost (1706). Then or otherwise, if the left template of the current block is available (1707), the distortion compared to the current block template is calculated (1708) and added to the current cost (1709). The cost of the current merge candidate mergeList[i].cost is then set equal to the calculated cost (1710), and the list is updated (1711). In this example, we consider the position of the current candidate i being set with respect to its cost compared to the costs of the other candidates. When all candidates have been processed, the number of candidates in the list, NumMergeCandInList, is set equal to the maximum number of possible candidates in the list, Maxcand.

[0250] Multiple Hypothesis Prediction (MHP)

[0251] Multiple hypothesis prediction (MHP) has also been added to ECM. With this approach, up to four motion compensated prediction signals for each block can be used (instead of two as in VVC). These individual prediction signals are superimposed to form an overall prediction signal. The motion parameters of the individual additional prediction hypotheses can be signaled explicitly by specifying the reference index, motion vector predictor index and motion vector difference, or implicitly by specifying the merge index. A separate multiple hypothesis merge flag distinguishes between these two signaling modes.

[0252] For spatial candidates, non-neighboring merge candidates, and history-based merge candidates, the multiple hypothesis parameter value "addHypNeighbours" is inherited from the candidate.

[0253] For temporal candidates, null candidates and paired candidates, the multiple hypothesis parameter values ​​"addHypNeighbours" are not kept (they are clear).

[0254] LIC

[0255] In ECM, local illumination compensation (LIC) is added. LIC is based on a linear model of illumination variation. The linear model is calculated using neighboring samples of the current block and neighboring samples of the previous block.

[0256] In ECM, LIC is enabled only for unidirectional prediction. LIC is signaled by means of a flag. For merge mode, the LIC flag is not transmitted but inherited from the merge candidate in the following manner.

[0257] For spatial candidates, non-adjacent merge candidates and history-based merge candidates, the value of the LIC flag is inherited.

[0258] For temporal candidates and zero candidates, the LIC flag is set equal to 0.

[0259] For paired candidates, such as Fig.16 The LIC flag is set as shown in the figure. Fig.12, and modules 1620 and 1621 are added, and modules 1609, 1612, and 1611 are updated. The variable averageUsed is set equal to false (1620), if for the current list, the average for the paired candidates has been calculated, the LIC flag LICFlag[cnt] for the paired candidates is set equal to false, and the variable averageUsed is equal to true (1609). If only the candidate has motion information for the list (16121611), if averaging is not used, the LIC flag is updated. And it is set equal to the OR operation of its current value and the value of the candidate's LICflag.

[0260] And when the pair candidate is Bidir (e.g. equal to 3), the LIC flag is equal to false.

[0261] However, if Fig.16 The algorithm shown allows LICflag to be equal to a value other than true only if both candidates have motion information for one list, and each candidate has its own list. For example, candidate 0 has motion information for only L0, and candidate 1 has motion information for only L1. In this case, the LIC flag can be equal to other values ​​other than 0, but this will never happen since LIC is only used in one direction. Therefore, the LIC flag of the paired candidate is always equal to false. Therefore, the paired candidate cannot use LIC when it may be needed. Therefore, this reduces the efficiency of the candidates and avoids the propagation of LIC for subsequent coding blocks, thereby reducing coding efficiency.

[0262] In addition, duplicate checks in ECM software lead to some inefficiencies. Fig.14 and Fig.15 As depicted, each candidate is added to the list, and repeated checks (1440, 1441, 1442, 1443, 1444, 1445, and 1530) have only an effect on the incrementing of the variable cnt (1405, 1409, 1413, 1417, 1423, 1427, 1508). Fig.16 As described, the variable BCCidx is not initialized for the paired candidates. Therefore, if the last candidate added in the list is a duplicate candidate, the value BCCidx of the paired candidate is the value of the previous duplicate candidate. This is not the case in VVC, because when candidates are considered duplicates, these candidates are not added.

[0263] Example

[0264] As described above, reordering the motion vector predictor candidates in the list generally means that the most likely predictor is positioned higher in the list and therefore requires fewer bits to encode. However, it has been noted that this does not always happen. The following examples aim to improve the reordering process by defining certain situations or candidate categories that do not use, reduce, and / or perform a second reordering process.

[0265] In the following embodiments, the lists of candidates or predictors will be reordered according to the calculated costs. For example, as described above, several lists of merge candidates or motion vector predictors are reordered according to template cost and intra predictor.

[0266] In the following examples, the list of candidates or predictors may be a list of intra or inter blocks or predictors to derive something other than a predictor. For example, in the above description, the MVD symbol prediction method reorders the list of possible MVD symbol indices. The present disclosure focuses primarily on merging candidate derivation.

[0267] Exclude candidate classes from the re-ranking process

[0268] In one example, the list of candidates or predictors will be reordered according to the calculated costs. The list derived before reordering can be from previously decoded or encoded motion information. For example, spatial position, temporal position, or spatial / temporal non-adjacent position, or from a list of previously decoded candidates, or candidates derived from other candidates, or candidates derived from other decoded samples or estimated samples. Such candidates are selected based on the fact that their corresponding samples may be related to the samples to be encoded / decoded.

[0269] The list of candidates does not always reach the maximum number of candidates (Maxcand) that the final list can contain. The maximum number of candidates in the list corresponds to the maximum index value that the decoder can decode. However, the list of candidates before reordering may contain more than this maximum value, and due to Fig.17 The cost-based reordering algorithm described in can remove additional candidates from the list. Thus, during derivation, the maximum candidate can be higher than the final maximum number of candidates after reordering.

[0270] When the list of candidates does not reach the maximum number of candidates, some additional candidates are added.

[0271] When candidates added to reach the maximum number of candidates are not considered during the reordering process (i.e., excluded from the reordering process), coding efficiency is improved. This is mainly because these candidates are repeated, and when the reordering process places them in an early position (earlier than the selected / best candidate), several consecutive merge indices are used to signal the same candidate, which increases the merge index rate or avoids the selection of the best candidate. Another reason is that, on average, these candidates are inefficient because they have no relevance to the current block, especially when the list of candidates is large.

[0272] Including these as part of the reordering process is unlikely to provide a benefit, and will therefore unnecessarily increase complexity (especially for large lists).

[0273] In particularly favorable cases, zero candidates added to the list are not considered for the reordering process. This has the advantage of reduced complexity, since the number of cost calculations is limited.

[0274] An additional advantage of this embodiment is improved coding efficiency, since zero candidates are generally inefficient and when they are reordered they may take the place of useful candidates, and when they are repeated zero candidates, several consecutive indices at the beginning or in the middle of the list represent exactly the same candidates, and the signaling of other candidates after these candidates has a higher rate than required.

[0275] It has been found that this scenario is more likely to occur at the beginning of a video sequence.

[0276] The reordering process may be applied only in some coding modes. In a particularly advantageous example, the reordering process is applied on the full list for one mode. The advantage is an improvement in coding efficiency. In fact, when the zero candidate is interesting in terms of coding efficiency, there is at least one mode that is represented with a minimum number of bits.

[0277] An example of such a mode that does not perform a reordering process on the full list is the sub-block merge mode. Thus, for the normal merge, TM merge, BM merge, and IBC merge modes, no reordering process is applied to the candidates that complete the list as zero candidates.

[0278] In another example, the pattern for which the reordering process is applied to the full list is a pattern in which the number of candidates is below a threshold (i.e., the number of candidates is small) and / or a pattern in which zero candidates are often in the list but the number of zero candidates is low (e.g., below a threshold). A suitable threshold is 4. In other words, for a pattern with a number of candidates above a threshold and / or a number of zero candidates above a threshold, the reordering process is not applied to the candidates that are set to complete the list as zero candidates.

[0279] Determine the cost function that defines the reordering process

[0280] The accuracy of the process of determining the relative "cost" of candidates at the decoder side is a determining factor in how effective the re-ordering process is. The following examples provide improvements to the cost determination process, which results in more accurate lists, lower complexity in calculating the cost, or both.

[0281] It is important to note that the algorithm used to perform the reordering can be changed to prioritize the accuracy or complexity (speed) of the reordering process.

[0282] The following examples are intended to produce an accurate indication of relative costs while minimizing the overall complexity of the operation.

[0283] In one example, the re-ranking is based on a cost including a metric between samples. A sample associated with each candidate is compared to another sample to generate a relative cost.

[0284] For example, the cost of the candidate can be calculated based on the neighboring samples of the predictor block and the neighboring samples of the current block. Such samples are easily available at the decoder side.

[0285] The cost may be calculated between samples of the 2 block predictors corresponding to the candidate in the list. For example, when the candidate is a bi-prediction candidate, the cost may be the distortion between the two block predictors. Bi-prediction candidates are discussed in more detail below.

[0286] The cost can also be calculated by comparing with another candidate. For example, one other candidate can be the most likely candidate or predictor. The cost of a candidate is calculated by comparing its samples with the samples of the most likely candidate.

[0287] The cost may be calculated for a subset of neighboring samples or a subset of the samples of the predictor. For example, if there are multiple neighboring samples that can be used to determine the cost, these samples are sampled to reduce the complexity of the calculation.

[0288] The cost may be calculated based on samples corresponding to an image according to another resolution. A high similarity to an image from a higher resolution is a good indication of a low cost (ie a good predictor).

[0289] The values ​​of the samples used to calculate the cost may be pre-processed. Assuming only relative cost values ​​are needed (i.e., only order is important), pre-processing the values ​​means simpler calculations and is less likely to significantly affect the efficacy of the re-ordering process. Based on the pre-processing, the calculated costs improve the re-ordering process.

[0290] In the above examples, the cost may be a measure of distortion such as the sum of absolute differences (SAD), the sum of absolute transformed differences (SATD), the sum of squared errors (SSE), or the structural similarity index measure (SSIM).

[0291] Alternatively, the cost is a measure of distortion and a weight can be applied to the distortion. The rate or estimated rate can also be considered. Or a threshold.

[0292] The cost can also be a weighted cost, where the weights differ depending on the type of predictor or candidate or the initial position of the candidate in the list.

[0293] Determine which candidates are re-ranked (signaled)

[0294] When some candidates in the list are not reordered, both the encoder and the decoder need to determine which candidates are excluded from reordering (or which candidates are reordered). One way to achieve this is to notify / derive a variable corresponding to the number of reordered candidates via additional signaling.

[0295] In one example, before candidates added to complete the candidate list are added to the candidate list, the variable numMaxNonZeroCand is set equal to the current number of candidates. Reordering is then limited to the value of this variable, rather than the maximum number that the list can contain. The variable can be the difference between the current number of candidates and Maxcand.

[0296] In one embodiment, the variable is encoded in the header at the sequence, set of pictures, picture, slice level. For better coding efficiency, the variable can be set equal to the average value of numMaxNonZeroCand. In this signaled case, the variable is preferably the difference between the current number of candidates and Maxcand. This may require fewer bits to encode because the difference (i.e., the number of candidates excluded from reordering) may be lower than the current number of candidates (i.e., the number of candidates to which reordering is applied).

[0297] This does not limit the computational cost when the maximum number of candidates for the list is greater than or equal to the maximum number of candidates that the final list can contain.

[0298] A variation of this is to identify the first candidate added to complete the list by a variable, and all candidates after the variable are not reordered.

[0299] This value can be used to avoid reordering of zero candidates or at least avoid computing the cost for zero candidates.

[0300] Fig. 20 This embodiment is illustrated for zero candidates. Fig.15 In this figure, numMaxNonZeroCand is set equal to the current number of candidates before adding zero candidates (2001). In addition, in this figure, zero candidates are added until the maximum number of candidates in the initial list MaxCandInitialList (2009)(2010), rather than Maxcand. In fact, in the latest ECM versions, the initial list of candidates can be higher than the final list of candidates.

[0301] Then, if Fig.21 As shown, the value numMaxNonZeroCand is used to avoid calculating costs for these candidates (2102). Fig.21 based on Fig.17 .

[0302] Another implementation would be to change the value of NumMergeCandInList through the variable numMaxNonZeroCand in module 2101 or 1701. This is only an implementation issue and depends on the initialization of the variable.

[0303] In an alternative example, a variable is utilized to identify individual candidates added to complete the candidate list (e.g., the zero candidate). The variable is used to apply or not apply the cost calculation during the reordering process of all candidates of the list. In this way, the same reordering process is applied, but the non-reordered candidates are ignored as they have no associated costs.

[0304] Handling un-reranked candidates

[0305] In advantageous implementations of these various examples, the candidate used to complete the candidate list (e.g., as the zero candidate) is placed at the end of the list. In one example, the candidates are sorted according to the order in which they were derived in the initial list. This property gives the coding efficiency as described above. For example, in Fig.21 In the process 2110, when the current number of candidates is greater than the value numMaxNonZeroCand (2113) corresponding to the zero candidate, the cost remains equal to the maximum value MAXVAL. Therefore, during the update candidate list process 2110, the zero candidate is located at the end of the list.

[0306] To provide additional coding efficiency, costs are calculated for candidates that are used as candidates to complete the candidate list (e.g., zero candidates), and these candidates are subjected to a separate second reordering process and are placed at the end of the list. This example improves coding efficiency, particularly when the number of such candidates is high.

[0307] In one example, when the list contains no more than one candidate, no reordering is applied except for the candidate that is set to complete the list (e.g., the zero candidate). Fig.21 is illustrated in module (2114).

[0308] Repeat candidate

[0309] In one example, predictors that are repeated in the list are not reordered during the reordering process. One implementation of this is to change the derivation of candidates. For example, in the case of duplicate checking, zero candidates are added (when zero candidates are all added or during the process of adding), and the variable numNonDuplicateCand is set equal to the current number of candidates in the list. Zero candidates are then added to complete the list without duplicate checking. During the reordering process, all candidates with higher indexes are not reordered and are retained at the end of the list. All previous features, such as cost calculation, signaling, and processing of non-reordered candidates, can be applied.

[0310] The advantage of this is improved coding efficiency, since several consecutive indices will not be used at the beginning of the list when the zero candidate has a small cost, which is particularly inefficient. This scenario is surprisingly common, so this change may have a significant impact on coding efficiency (especially for short video sequences).

[0311] The reordering process can be changed dynamically according to the encoding mode. For example, for some modes except one mode where only repeated candidates are not reordered, the candidates used to complete the list are not reordered in the list.

[0312] Template matching cost calculation

[0313] The following examples address some issues related to template availability for computational cost. These examples focus on template matching methods. Template availability means that the template is outside the reference frame or outside the delimited region. Delimited regions can be defined to reduce memory access for this method.

[0314] Fig. 22 Three cases where the template for the predictor is not available are illustrated. In the figure, 2201, 2202, 2203 represent the boundaries of the reference frame or the delimited region. For the first case, the left template (2204) is not available, but the upper template is available. For the second case, the upper template of the predictor block is outside the reference frame or the restricted region (2202), and the left template is available (2207).

[0315] When a template is missing, its associated cost is by default set equal to 0. Therefore, these candidates have more chances to be selected compared to other candidates.

[0316] In the third case, both templates (2208, 2209) are not available. Candidates associated with these cases should have block predictors outside the reference frame, so these candidates usually produce worse block predictors than blocks from the real reference frame. In fact, when samples are outside the current frame, the missing samples are replaced by samples from the boundary rows or columns of the reference frame.

[0317] In order to improve the problem of calculating the cost of a non-existent template, when one template of the predictor block does not exist, the cost that cannot be calculated is replaced by a value based on the cost value of the relevant available template. The advantage of this example is that the coding efficiency is improved. In fact, the cost will correspond to the value that is closest to the value that can be obtained when both templates are available. Compared to the existing methods, the candidate in this case will not be set to an early position because it has been calculated with fewer samples than other candidates.

[0318] Alternatively, the cost is calculated to be based on the size of the template, e.g. proportional to the size of the template. The advantage is that the cost of the unavailable template will be statistically closest to the true cost, and the comparison with the costs of other candidates is fairer. The method can also be computationally simpler.

[0319] The ratio may depend on the number of samples of usable templates and unusable templates. Fig. 22 In case 1, the cost costLeft of the unavailable left template (CAO4) is set equal to the cost costUP of the upper template (2205) divided by the width of the block (width) and multiplied by the height of the block (height) as follows:

[0320] costLeft=(costUp / width)×height

[0321] Therefore, the cost of the current candidate is:

[0322] cost=costUp+(costUp / width)×height

[0323] Fig. 22 Specific cases 1 and 2 are illustrated, but for some cases, only a portion of the template is missing. In this case, the partial cost can be calculated for all available samples.

[0324] In this case, the ratio depends on the number of available samples and unavailable samples. For example, if we consider the number of available samples numAvailableS, the number of unavailable samples numNonAvailableS, and the cost of available samples costAvailableS, the cost of a candidate sample can be:

[0325] cost=costAvailableS+(costAvailableS / numAvailableS)×numNonAvailableS

[0326] This can be applied to determine the cost for one template or for all templates.

[0327] Compared to the previous embodiments, the advantage is a better estimate of the true cost of the candidate.

[0328] To reduce complexity, the ratio is calculated using a shift operation. For example, the shift operation that is closest to the actual calculation can be used. For example, the denominator is approximated to the nearest power of 2.

[0329] The advantage is simplicity, especially for hardware implementation, since no division is used. (Shift operations are no more complicated than division).

[0330] In an alternative example, the cost value for the unavailable template is set equal to the cost value for the available template. Fig. 22 For case 1, the cost is:

[0331] cost=costUp+costUp

[0332] is equivalent to:

[0333] cost=costUp<<1

[0334] Where << is a shift operation

[0335] This is mathematically equivalent to:

[0336] cost=costUp×2

[0337] The advantage is further simplification and less impact on coding efficiency because many blocks are square blocks.

[0338] It has been noted that for unavailable templates, the cost may be underestimated. To improve this, a weight w can be applied to the formula to penalize the estimated cost of the candidate. In this case, the basic formula for calculating the cost is:

[0339] cost=costUp+w×(costUp / width)×height

[0340] where w is greater than 1 to penalize the estimated cost.

[0341] The advantage of this embodiment is the improved coding efficiency, in fact, if there is a missing template, the relevant block predictor is unlikely to be very relevant. Therefore, the weight will incur a higher cost and sometimes place the candidate at the lowest position in the final list.

[0342] In the above examples, we considered the upper and left templates, where one row of samples as in ECM. But longer templates or more templates can be considered. As an example, the upper right template or the lower left template can be considered.

[0343] In the same way, further rows or lines of the template may be considered to calculate the cost. And further, for each of the above examples, bi-prediction may also be considered.

[0344] Fig. 22 The third case illustrates the case where there is no template available for the current predictor (2208, 2209). In this case, any cost can be calculated.

[0345] In case no template is available for the predictor, the value is set equal to the maximum value. This ensures that the relevant candidates are not set at the beginning of the list.

[0346] The advantage of this is that the coding efficiency is improved since these candidates are not at the beginning of the list compared to when the cost is set equal to 0. In practice, these candidates are not very efficient since they contain estimated samples instead of true samples.

[0347] In a variant, the maximum value is less than the maximum value set for the candidates for completing the list (eg, zero candidates). In fact, these candidates are more efficient than those added to meet the requirement of having a certain number of candidates.

[0348] In another variant, candidates without available templates are placed at the end of the list and before candidates added to meet the requirement of having a certain number of candidates (e.g., zero candidates). In fact, as described above, this can be achieved by setting the cost values ​​of these candidates equal to the maximum and highest maximum values ​​for zero candidates.

[0349] In another example, when there is a missing template, the cost for the candidate is set equal to the maximum value so that the candidate is at the end of the list. This has the advantage of reduced complexity because the cost does not need to be calculated for the available templates, nor does it need to be adjusted. Therefore, the coding efficiency is lower than the previous embodiment with one available template, but the complexity is lower.

[0350] Advantageously, the maximum value is smaller than the maximum value set for the candidates for completing the list (eg, the zero candidate). In fact, these candidates are more efficient than such candidates.

[0351] When there are several candidates in this case, these candidates keep the same order as in the initial list. For example, if candidate 1 is before candidate 2 in the initial list, then in the reordered list, candidate 1 is before candidate 2.

[0352] If only one template is missing for a certain candidate, but more templates are missing for other candidates, the candidate with only one missing template is placed before the candidate with all unavailable templates.

[0353] Limit reordering based on template availability

[0354] Fig.23 Three cases are illustrated where no template is available for the current block (case 6) or the template is partially available (case 4 and case 5). These cases are surprisingly common when the bit rate is low, and for small sequences or small strips.

[0355] In one example, when at least one template for the current block is not available, the list is not reordered. This can be achieved by not calculating the cost for any candidate.

[0356] The advantage is that the coding efficiency is less affected while the complexity is reduced.

[0357] To improve the situation when at least one template is not available, the number of temporal candidates is increased. Fig.19 In , the number of temporal candidates is not 1 but a higher value. For example, this can be all temporal candidates. In the example of the current ECM implementation, this can be 9 temporal candidates. Note that in this case, there are no AMRCs for temporal and non-adjacent candidates during the first ARMC process.

[0358] In addition, in order to be able to derive different additional temporal candidates, the number of temporal positions used to derive temporal candidates may be increased.

[0359] Alternatively, the number of temporal candidates is the same as when templates are available.

[0360] The advantage of this embodiment is that the coding efficiency is increased because when no template is available (case 6), there are no spatial candidates, no history-based templates, and no non-adjacent candidates. And in the case of only one temporal candidate, no paired candidates can be derived. Therefore, if more temporal candidates are added, this increases the coding efficiency. This is particularly efficient at low bit rates and for small sequences or small strips (as often happens in cases 4, 5, 6).

[0361] In an alternative or additional example, the maximum number of candidates for these blocks is reduced. For example, if only time can be considered, the maximum number of candidates should be 1. In this case, there is no cost associated with the current block.

[0362] Additionally, some modes may be avoided when there is no time available due to temporal candidates being prohibited or for the first inter frame of a GOP, since all inter modes without motion vector residuals act as merge modes.

[0363] The advantage is improved coding efficiency for correlated blocks.

[0364] In another example, when no reordering is possible, the motion vector threshold used is changed. Since there are few possible spatial candidates, the motion vector threshold is reduced (if possible) to get more candidates in the list. This allows additional candidates to be added to the list without being considered duplicates and potentially removed (and replaced by zero candidates).

[0365] This increases coding efficiency because the list contains more candidates and fewer less useful zero candidates.

[0366] Bidirectional Candidate

[0367] When the candidates are bidirectional, there are two possibilities to calculate the cost associated with the template. The first possibility is to calculate the cost of each template independently for the first list and the second list, and add these costs. If a single prediction is possible in the candidate set, the cost is divided by 2.

[0368] Due to our example, a second possibility is to apply bidirectional computation to obtain only one template for each upper and left template, and compute the cost of these bidirectional templates with the template of the current block. This second possibility is particularly efficient when bi-prediction with CU-level weights (BCW) is enabled.

[0369] Fig.24 Bidirectional prediction is illustrated when the left template of the block predictor in L0 is not available.

[0370] In one embodiment, when the cost of each template is calculated independently for a bi-prediction candidate and one template is not available for one direction and available for a second direction, the cost of the available template is used instead of the cost of the unavailable template. Fig.24 If the cost costLeftL0 cannot be calculated, the final cost is set equal to:

[0371] cost=costLeftL1<<1+costTopL0+costTopL1

[0372] Where costLeftL1 is the cost of the left template in L1 (2406), costTopL0 is the cost of the top template in L0, and costTopL1 is the cost of the top template in L1. And << is the left shift operator. So costLeftL1<<1 means the cost costLeftL1 is multiplied by 2.

[0373] For reference, for the case where all templates are available, the cost is calculated by the following formula:

[0374] cost=costLeftL0+costLeftL1+costTopL0+costTopL1

[0375] Advantages: Improved coding efficiency.

[0376] In a related example, when one template is partially available, the cost is calculated for the available samples, and the cost of the unavailable template corresponds to the cost of the associated available template. The advantage is that the cost closest to the actual cost is calculated.

[0377] In a similar manner as described above, a penalty may be applied to a cost in place of the current cost or to a final cost.

[0378] It should be understood that more templates, rows or lines of templates may be considered.

[0379] In a related but alternative example, when the candidate is bidirectional and one template is not available for one direction and is available for a second direction, the cost is calculated for the available template without deriving a bidirectional prediction. Fig.24 In , if the left template of L0 is not available, only the single prediction of the left template is considered to calculate the cost of the left template.

[0380] The advantage is improved coding efficiency.

[0381] In a similar manner as described above, when one template is partially available, bi-prediction is calculated for samples available in both directions, and uni-prediction is considered for available samples in one direction (when not available for the second direction).

[0382] The advantage is that the cost closest to the actual cost is calculated.

[0383] In a similar manner as discussed above, a penalty may be applied to a cost in place of the current cost or to the final cost.

[0384] Also more templates, rows or lines of templates are conceivable.

[0385] Unless explicitly stated otherwise, all of these embodiments can be combined. In fact, many combinations are synergistic and can produce efficiency improvements that are greater than the sum of their parts.

[0386] Implementation of the invention

[0387] Fig.25 A system 191, 195 according to an embodiment of the present invention is shown, which includes at least one of the encoder 150 or the decoder 100 and a communication network 199. According to an embodiment, the system 195 is used to process and provide content (e.g., video and audio content for display / output or streaming video / audio content) to a user, who accesses the decoder 100, for example, through a user terminal including the decoder 100 or a user interface of a user terminal that can communicate with the decoder 100. Such a user terminal can be a computer, a mobile phone, a tablet computer, or any other type of device capable of providing / displaying (provided / streamed) content to a user. The system 195 obtains / receives the bitstream 101 (in the form of a continuous stream or signal (e.g., when displaying / outputting earlier video / audio)) via the communication network 199. According to an embodiment, the system 191 is used to process content and store processed content, such as video and audio content processed for display / output / streaming at a later time. The system 191 obtains / receives content including an original image sequence 151, which is received and processed by the encoder 150 (including filtering using a deblocking filter according to the present invention), and the encoder 150 generates a bitstream 101 to be transmitted to the decoder 100 via the communication network 191. The bitstream 101 is then transmitted to the decoder 100 in a variety of ways, for example, it can be pre-generated by the encoder 150 and stored as data in a storage device in the communication network 199 (for example, on a server or cloud storage device) until the user requests the content (i.e., bitstream data) from the storage device, at which time the data is transmitted / streamed from the storage device to the decoder 100. The system 191 may also include a content providing device for providing / streaming to the user (for example, by transmitting data of a user interface to be displayed on a user terminal) content information (for example, the title of the content and other meta / storage location data for identifying, selecting and requesting the content) of the content stored in the storage device, and for receiving and processing user requests for content so that the requested content can be transmitted / streamed from the storage device to the user terminal. Alternatively, the encoder 150 generates the bitstream 101 and transmits / streams it directly to the decoder 100 when the user requests the content. The decoder 100 then receives the bitstream 101 (or signal) and filters it using the deblocking filter according to the present invention to obtain / generate a video signal 109 and / or an audio signal, which is then used by the user terminal to provide the requested content to the user.

[0388] Any step of the method / processing according to the present invention or the function described herein may be implemented in hardware, software, firmware or any combination thereof. If implemented in software, the step / function may be stored as one or more instructions or codes or programs or computer-readable media on one or more hardware-based processing units or sent via one or more hardware-based processing units, and executed by one or more hardware-based processing units, such as a programmable computing machine, which may be a PC ("personal computer"), a DSP ("digital signal processor"), a circuit, a circuit system, a processor and memory, a general-purpose microprocessor or central processing unit, a microcontroller, an ASIC ("application-specific integrated circuit"), a field programmable logic array (FPGA) or other equivalent integrated or discrete logic circuit system. Therefore, the term "processor" as used herein may refer to any of the aforementioned structures or any other structures suitable for implementing the technology described herein.

[0389] Embodiments of the present invention may also be implemented by various devices or apparatuses, including wireless handsets, integrated circuits (ICs), or JC collections (e.g., chipsets). Various components, modules, or units are described herein to illustrate functional aspects of the apparatus / device configured to perform these embodiments, but do not necessarily need to be implemented by different hardware units. Instead, the various modules / units may be combined in a codec hardware unit or provided by a collection of interoperable hardware units, which include one or more processors in conjunction with appropriate software / firmware.

[0390] Embodiments of the present invention can be implemented by a computer of a system or device that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium to perform one or more modules / units / functions in the above-mentioned embodiments and / or includes one or more processing units or circuits for performing one or more functions in the above-mentioned embodiments, and can be implemented by a method performed by a computer of a system or device, for example, reading out and executing computer executable instructions from a storage medium to perform one or more functions in the above-mentioned embodiments and / or controlling one or more processing units or circuits to perform one or more functions in the above-mentioned embodiments. The computer may include a network of separate computers or separate processing units to read out and execute computer executable instructions. Computer executable instructions may be provided to a computer from a computer-readable medium such as a communication medium, for example, via a network or a tangible storage medium. The communication medium may be a signal / bit stream / carrier. A tangible storage medium is a “non-transitory computer-readable storage medium” which may include, for example, a hard disk, a random access memory (RAM), a read-only memory (ROM), a storage device of a distributed computing system, an optical disk (such as a compact disk (CD), a digital versatile disk (DVD), or a Blu-ray disk (BD) TM ), one or more of a flash memory device, a memory card, etc. At least some steps / functions may also be implemented in hardware by a machine or dedicated components such as an FPGA (“field programmable gate array”) or an ASIC (“application-specific integrated circuit”).

[0391] Fig.263600 is a schematic block diagram of a computing device 3600 for implementing one or more embodiments of the present invention. The computing device 3600 may be a device such as a microcomputer, a workstation, or a light portable device. The computing device 3600 includes a communication bus connected to the following: - a central processing unit (CPU) 3601, such as a microprocessor; - a random access memory (RAM) 3602 for storing executable code of the method of the embodiment of the present invention and registers suitable for recording variables and parameters required to implement the method for encoding or decoding at least a part of an image according to the embodiment of the present invention, and its storage capacity can be expanded, for example, by an optional RAM connected to an expansion port; - a read-only memory (ROM) 3603 for storing a computer program for implementing an embodiment of the present invention; - a network interface (NET) 3604, which is usually connected to a communication network through which the digital data to be processed is transmitted or received, and the network interface (NET) 3604 may be a single network interface, or composed of a group of different network interfaces (for example, wired and wireless interfaces, or different kinds of wired or wireless interfaces), which are connected to the CPU when running. 3601, data packets are written to the network interface for transmission or read from the network interface for reception; - a user interface (UI) 3605, which can be used to receive input from a user or display information to a user; - a hard disk (HD) 3606, which can be set as a mass storage device; - an input / output module (IO) 3607, which can be used to receive / send data from / to an external device (such as a video source or a display, etc.). The executable code can be stored in ROM 3603, on HD 3606, or on a removable digital medium such as a disk. According to a variant, the executable code of the program can be received via NET 3604 by means of a communication network to be stored in one of the storage components (such as HD 3606, etc.) of the computing device 3600 before being executed. CPU 3601 is suitable for controlling and directing the execution of instructions or parts of the software code of one or more programs according to an embodiment of the present invention, which instructions are stored in one of the aforementioned storage components. For example, after power-up, CPU 3601 is capable of executing those instructions relating to a software application from main RAM memory 3602 after loading instructions from program ROM 3603 or HD 3606. Such a software application, when executed by CPU 3601, causes the steps of a method according to the invention to be performed.

[0392] It should also be understood that according to other embodiments of the present invention, a decoder according to the above embodiments is provided in a user terminal such as a computer, a mobile phone (cellular phone), a tablet or any other type of device (e.g., a display device) capable of providing / displaying content to a user. According to yet another embodiment, an encoder according to the above embodiments is provided in an image capture device, which also includes a camera, a video camera or a webcam (e.g., a closed-circuit television or video surveillance camera) for capturing and providing content for encoding by the encoder. See below Fig. 27 and Fig.28 Two such examples are provided.

[0393] Fig. 27 is a diagram illustrating a network camera system 3700 including a network camera 3702 and a client device 202 .

[0394] The network camera 3702 includes an imaging unit 3706 , an encoding section 3708 , a communication unit 3710 , and a control unit 3712 .

[0395] The network camera 3702 and the client device 202 are connected to each other via the network 200 to be able to communicate with each other.

[0396] The camera unit 3706 includes a lens and an image sensor (eg, a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS)), and captures an image of a subject and generates image data based on the image. The image may be a still image or a video image.

[0397] The encoding section 3708 encodes the image data by using the encoding method described above or a combination of the encoding methods described above.

[0398] The communication unit 3710 of the network camera 3702 transmits the encoded image data encoded by the encoding section 3708 to the client device 202 .

[0399] Furthermore, the communication unit 3710 receives commands from the client device 202. The commands include commands for setting parameters for encoding by the encoding section 3708.

[0400] The control unit 3712 controls other units in the network camera 3702 according to the command received by the communication unit 3710 .

[0401] The client device 202 includes a communication unit 3714 , a decoding unit 3716 , and a control unit 3718 .

[0402] The communication unit 3714 of the client device 202 transmits a command to the network camera 3702 .

[0403] Furthermore, the communication unit 3714 of the client device 202 receives the encoded image data from the web camera 3702 .

[0404] The decoding unit 3716 decodes the encoded image data by using the decoding method described above or a combination of the decoding methods described above.

[0405] The control unit 3718 of the client device 202 controls other units in the client device 202 according to a user operation or command received by the communication unit 3714 .

[0406] The control unit 3718 of the client device 202 controls the display device 2120 to display the image decoded by the decoding section 3716 .

[0407] The control unit 3718 of the client device 202 also controls the display device 2120 to display a GUI (Graphical User Interface) for specifying values ​​of parameters of the network camera 3702 (including parameters for encoding by the encoding section 3708 ).

[0408] The control unit 3718 of the client device 202 also controls other units in the client device 202 according to user operation input to the GUI displayed by the display device 2120 .

[0409] The control unit 3718 of the client device 202 controls the communication unit 3714 of the client device 202 according to the user operation input to the GUI displayed by the display device 2120 to transmit a command for specifying the value of the parameter of the network camera 3702 to the network camera 3702 .

[0410] Fig.28 The diagram illustrates an example of a smart phone 3800 .

[0411] The smartphone 3800 includes a communication unit 3802 , a decoding / encoding section 3804 , a control unit 3806 , and a display unit 3808 .

[0412] The communication unit 3802 receives the encoded image data via the network 200 .

[0413] The decoding / encoding section 3804 decodes the encoded image data received by the communication unit 3802 .

[0414] The decoding / encoding section 3804 decodes / encodes the encoded image data by using the decoding method described above.

[0415] The control unit 3806 controls other units in the smartphone 3800 according to user operations or commands received by the communication unit 3802 .

[0416] For example, the control unit 3806 controls the display unit 3808 to display the image decoded by the decoding / encoding section 3804. The smartphone 3800 may also include a sensor 3812 and an image recording device 3810. In this way, the smartphone 3800 may record an image and encode the image (using the method described above).

[0417] The smartphone 3800 may then decode the encoded image (using the method described above) and display it via the display unit 3808 , or transmit it to another device via the communication unit 3802 and the network 200 .

[0418] Replacements and modifications

[0419] Although the present invention has been described with reference to the embodiments, it should be understood that the present invention is not limited to the disclosed embodiments. It will be appreciated by those skilled in the art that various changes and modifications may be made without departing from the scope of the present invention as defined in the appended claims. All features disclosed in this specification (including any appended claims, abstracts and drawings) and / or all steps of any disclosed method or process may be combined in any combination, except for at least some mutually exclusive combinations of such features and / or steps. Unless otherwise expressly stated, each feature disclosed in this specification (including any appended claims, abstracts and drawings) may be replaced by alternative features for the same, equivalent or similar purposes. Therefore, unless otherwise expressly stated, each feature disclosed is only an example of a general series of equivalent or similar features.

[0420] It should also be understood that any results of the above-described comparison, determination, evaluation, selection, execution, proceeding, or consideration (e.g., selections made during an encoding or filtering process) may be indicated in data in the bitstream (e.g., a flag or data indicating the result) or may be determined / inferred from data in the bitstream, such that the indicated or determined / inferred result may be used for processing rather than actually being compared, determined, evaluated, selected, executed, performed, or considered, for example, during a decoding process.

[0421] In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be used to advantage.

[0422] Reference signs appearing in the claims are by way of illustration only and shall have no limiting effect on the scope of the claims.

Claims

1. A method for generating a list of motion vector predictor candidates for predicting motion in a picture portion, the method include: adding a first set of motion vector predictor candidates to the list; In case the number of motion vector predictor candidates of the first set is lower than a maximum number of candidates, adding motion vector predictor candidates of the second set to the list such that the total number of candidates is equal to the maximum number of candidates; reordering said list of candidates, Wherein, the candidates of the second set are excluded from the reordering.

2. The method according to claim 1, in, The first set does not include duplicates.

3. A method for generating a list of motion vector predictor candidates for predicting motion in a picture portion, the method include: adding a first set of motion vector predictor candidates to the list; determining duplicate candidates in the list; In case the number of motion vector predictor candidates of the first set is lower than a maximum number of candidates, adding motion vector predictor candidates of the second set to the list such that the total number of candidates is equal to the maximum number of candidates; reordering said list of candidates, Wherein, the duplicate candidates are excluded from the reordering.

4. The method according to claim 3, in, The second set of candidates is also excluded from the reordering.

5. The method according to any one of the preceding claims, in, The motion vector predictor candidates of the second set are zero candidates, and the candidates of the first set are not zero candidates.

6. The method according to any one of the preceding claims, in, The candidates of the first set are candidates derived from at least one previously decoded or encoded motion information.

7. The method according to claim 6, in, The candidates of the first set are candidates derived from true samples corresponding to one or more of temporal candidates, spatial candidates, historical candidates, previously used candidates, and candidates derived from other candidates.

8. The method according to any one of the preceding claims, in, The re-ranking is based on the calculated relative costs of the candidates.

9. The method according to claim 8, in, at least one of the candidates is derived from at least one spatially or temporally matched template, wherein the templates within the delimited area are available, and the templates outside the delimited area are unavailable, and In the event that at least one template is not available, a non-zero cost is calculated for the candidate.

10. A method for generating a list of motion vector predictor candidates for predicting motion in a picture portion, the method include: adding a plurality of motion vector predictor candidates to a list; calculating a cost associated with at least one candidate in the list, wherein the at least one candidate is derived from at least one spatially or temporally matched template, wherein templates within a delimited region relative to the image portion are available, and templates outside the delimited region are not available, and In the event that at least one template is unavailable, a non-zero cost is calculated for the candidate, and the list is reordered according to the calculated cost.

11. The method according to claim 9 or 10, in, The cost is determined based on a size of one or more templates from which the candidate is derived.

12. The method according to any one of claims 9 to 11, in, The cost is determined based on the number of samples of the available templates and the unavailable templates.

13. The method according to claim 12, in, The cost is determined based on a ratio of usable samples to unusable samples.

14. The method according to claims 11 to 13, in, The division used to determine the cost approximates a shift operation.

15. The method according to any one of claims 9 to 10, setting the cost value for an unavailable template equal to the cost value of an associated available template.

16. The method according to any one of claims 9 to 23, in, In the event that no template is available, the cost is set to a maximum value.

17. The method according to claim 16, in, In case the number of added motion vector predictor candidates is lower than the maximum number of candidates, the candidates for which no relevant template is available are reordered in the list immediately above the added motion vector predictor candidates so that the total number of candidates is equal to the maximum number of candidates.

18. The method according to claim 16 or 17, in, In case the number of added motion vector predictor candidates is lower than the maximum number of candidates, the maximum value is lower than the cost assigned to the added motion vector predictor candidates so that the total number of candidates is equal to the maximum number of candidates.

19. The method according to any one of claims 9 to 18, in, The cost for the candidate is assigned the maximum value.

20. The method according to any one of claims 9 to 19, in, The at least one candidate is a bidirectional candidate having two spatially matched templates, and When determining the cost of the bidirectional candidate, the cost of the available template is used instead of the cost of the unavailable template.

21. The method according to any one of claims 9 to 19, in, The at least one candidate is a bidirectional candidate having two spatially matched templates, and Only available templates are used to calculate the cost of the bidirectional candidates.

22. The method according to claim 20 or 21, in, In case a template is partially available, the cost is calculated for the available samples, or the cost for an unavailable template corresponds to the cost of the associated available template.

23. The method of any one of claims 9 to 22, further comprising applying a weight to the calculated cost of the candidate if a template is not available.

24. A method for generating a list of motion vector predictor candidates for predicting motion in a portion of an image, the method include: Add multiple motion vector predictor candidates to the list, wherein at least one candidate in the list is derived from at least one spatially or temporally matched template, wherein templates within a delimited region relative to the image portion are available, and templates outside the delimited region are not available, and Unless at least one template is unavailable, the list is reordered.

25. The method according to claim 24, in, The reordering is performed unless all templates are unavailable.

26. A method for generating a list of motion vector predictor candidates for predicting motion in a portion of an image, the method include: determining availability of at least one template related to the template of the image portion; as well as The motion vector predictor candidates are derived according to the availability of the template.

27. The method according to claim 26, in, In case at least one template is not available, the method comprises adding at least one further temporal candidate to the list.

28. The method according to claim 26 or 27, in, The number of temporal candidates is the same regardless of the availability of the template.

29. The method according to any one of claims 26 to 28, include: Reduce the maximum number of candidates.

30. The method according to any one of claims 26 to 29, include: Reduce the motion vector threshold.

31. The method according to any one of claims 8 to 30, in, A cost is calculated for the candidate based on a comparison metric between at least one sample associated with the candidate and at least one other sample in the list.

32. The method according to claim 31, in, The cost for a candidate is calculated based on the difference between neighboring samples of the predictor block and neighboring samples of the current block.

33. The method according to claim 31, in, The cost for a candidate is calculated by computing the difference of the two block predictors.

34. The method according to claim 31, in, The cost for a candidate is calculated by calculating the difference from another candidate in the list.

35. The method according to claim 34, in, The other candidate is the candidate with the highest probability.

36. The method according to claim 31, in, The cost is based on a sample or adjacent sub-samples of the predictor.

37. The method according to claim 31, in, The cost is based on samples corresponding to an image according to another resolution.

38. A method according to any one of claims 31 to 37, in, The values ​​of the samples used to calculate the cost are pre-processed.

39. The method according to any one of claims 31 to 38, in, The cost corresponds to the distortion.

40. The method according to claim 39, in, The distortion is SAD, SATD, SSE or SSIM.

41. The method according to any one of claims 8 to 40, in, The cost includes a weight.

42. The method according to claim 41, in, The weights differ between motion vector predictor candidates.

43. A method according to any one of the preceding claims, include: A variable corresponding to the number of motion vector predictor candidates in the first set is derived, and the reordering of the list is performed according to the variable.

44. The method according to claim 43, in, The variable identifies a first candidate from the second set of motion vector predictor candidates.

45. The method according to claim 1 or 2, in, Each motion vector predictor candidate in the first set is associated with a variable, and the reordering is performed according to these variables.

46. ​​A method according to any one of the preceding claims, include: The non-reordered motion vector predictor candidates are set to the end of the list.

47. The method according to claim 46, include: The second reordering process is performed on the non-reordered motion vector predictor candidates.

48. The method according to claim 46, include: When the first set contains no more than one candidate, a second reordering process is performed on the non-reordered motion vector predictor candidates.

49. The method according to claim 46, include: The non-reordered motion vector predictor candidates are subjected to a second reordering process according to the encoding mode.

50. The method according to claim 49, in, The second reordering is not applied for sub-block merging mode.

51. The method according to claim 46, include: When the mode has a number of candidates greater than a threshold, a second reordering process is performed on the non-reordered motion vector predictor candidates.

52. A method for encoding image data into a bitstream, the method comprising a method for generating a list of motion vector predictor candidates according to any one of claims 1 to 51.

53. A method for decoding image data from a bitstream, the method comprising a method for generating a list of motion vector predictor candidates according to any one of claims 1 to 51.

54. An apparatus for encoding image data into a bitstream, the apparatus being configured to perform the method for generating a list of motion vector predictor candidates according to any one of claims 1 to 51.

55. An apparatus for decoding image data from a bitstream, the apparatus being configured to perform the method for generating a list of motion vector predictor candidates according to any one of claims 1 to 51.

56. A computer program which, when executed, causes the method according to any one of claims 1 to 51 to be performed.

57. A computer readable carrier medium having stored thereon a computer program according to claim 56.

Citation Information

Cited By

  • Method and apparatus of MV field derivation based on temporal motion vectors and TMVP refinement in video coding systems

    WO2026153487A1