Construction of a template-matching-based advanced motion vector predictor (AMVP) candidate list using non-adjacent candidates and AMVP index signaling.

Abstract

An exemplary device includes a memory configured to store video data, and one or more processors implemented in a circuit and communicatively coupled to the memory. The one or more processors are configured to determine at least one of a temporal candidate or a history-based candidate, and determine at least one non-adjacent candidate, where the at least one non-adjacent candidate is from a unit that is not adjacent to a current prediction unit (PU). The one or more processors are configured to determine an advanced motion vector predictor (AMVP) candidate list including at least one of the temporal candidate or the history-based candidate and the at least one non-adjacent candidate. The at least one non-adjacent candidate is added to the AMVP candidate list after the temporal candidate or before the history-based candidate. The one or more processors are configured to code the current PU based on the AMVP candidate list.

Description

Technical Field

[0001]

[0001] This application claims priority to U.S. Patent Application No. 17 / 704,689, filed Mar. 25, 2022, and U.S. Provisional Application No. 63 / 167,555, filed Mar. 29, 2021, the entire contents of which are incorporated herein by reference. U.S. Patent Application No. 17 / 704,689, filed Mar. 25, 2022, claims the benefit of U.S. Provisional Application No. 63 / 167,555, filed Mar. 29, 2021.

[0002]

[0002] This disclosure relates to video encoding and video decoding.

Background Art

[0003]

[0003] Digital video capabilities can be incorporated into a wide range of devices, including digital televisions, digital direct broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video gaming devices, video game consoles, cellular or satellite radiotelephones, so-called “smartphones,” video teleconferencing devices, video streaming devices, etc. Digital video devices implement video coding techniques described in standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, Advanced Video Coding (AVC), ITU-T H.265 / High Efficiency Video Coding (HEVC), and extensions of such standards, as well as proprietary video codec / formats such as AOMedia Video 1 (AV1) developed by the Alliance for Open Media. By implementing such video coding techniques, video devices can transmit, receive, encode, decode, and / or store digital video information more efficiently.

[0004]

[0004] Video coding techniques include spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to reduce or eliminate redundancy inherent in video sequences. In block-based video coding, a video slice or video tile (e.g., a video picture or a portion of a video picture) may be divided into video blocks, which may also be called coding blocks, superblocks, coding tree units (CTUs), coding units (CUs), and / or coding nodes. A video block in an intra-coded (I) slice of a picture is coded using spatial predictions for reference samples in adjacent blocks within the same picture. A video block in an inter-coded (P or B) slice of a picture may use spatial predictions for reference samples in adjacent blocks within the same picture, or temporal predictions for reference samples in other reference pictures. A picture may be called a frame, and a reference picture may be called a reference frame. [Overview of the Initiative]

[0005]

[0005] Generally speaking, this disclosure describes techniques for constructing Advanced Motion Vector Predictor (AMVP) candidate lists and signaling AMVP indices. Such techniques may be applied to any existing video codec such as HEVC (High Efficiency Video Coding), VVC (Variable-Purpose Video Coding), or Essential Video Coding (EVC), or may be efficient coding tools for future video coding standards as well as for proprietary video codecs / formats. For example, an AMVP candidate list may include one or more non-adjacent candidates derived from units / blocks that are not adjacent to the current prediction unit (PU). Including one or more non-adjacent candidates in the AMVP candidate list can make the motion information of the candidates in the AMVP candidate list more diverse, which can improve coding efficiency. A unit or block, for example, a prediction unit from which a candidate (equipped with motion information of the candidate, such as a motion vector) in the AMVP candidate list is derived or determined, may also be referred to herein as a candidate unit or PU.

[0006]

[0006] In one example, the method includes determining at least one of time-based candidates or history-based candidates; determining at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein the at least one non-adjacent candidate unit determines an advanced motion vector predictor (AMVP) candidate list comprising at least one of time-based candidates or history-based candidates and at least one non-adjacent candidate that is not adjacent to the current predictor unit (PU); wherein the at least one non-adjacent candidate codes the current PU based on the AMVP candidate list, which is added to the AMVP candidate list at least one of after the time-based candidate or before the history-based candidate.

[0007]

[0007] In another example, the device includes a memory configured to store video data and one or more processors implemented in the circuit and communicatively coupled to the memory, wherein the one or more processors are configured to determine at least one of time-based candidates or history-based candidates; to determine at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein the at least one non-adjacent candidate unit determines an advanced motion vector predictor (AMVP) candidate list comprising at least one of time-based candidates or history-based candidates and at least one non-adjacent candidate that is not adjacent to the current predictor unit (PU), wherein the at least one non-adjacent candidate codes the current PU based on the AMVP candidate list, which is added to the AMVP candidate list at least one of after the time-based candidate or before the history-based candidate.

[0008]

[0008] In another example, a non-temporary computer-readable storage medium stores instructions that, when executed, cause one or more processors to determine at least one of time-based candidates or history-based candidates; determine at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein the at least one non-adjacent candidate unit determines an advanced motion vector predictor (AMVP) candidate list comprising at least one time-based candidate or history-based candidate and at least one non-adjacent candidate that is not adjacent to the current predictor unit (PU); wherein the at least one non-adjacent candidate codes the current PU based on the AMVP candidate list, which is added to the AMVP candidate list at least one of after the time-based candidate or before the history-based candidate.

[0009]

[0009] In another example, the device includes means for determining at least one of time-based candidates or history-based candidates; means for determining at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein the at least one non-adjacent candidate unit is means for determining an advanced motion vector predictor (AMVP) candidate list comprising at least one of time-based candidates or history-based candidates and at least one non-adjacent candidate, wherein the at least one non-adjacent candidate is added to the AMVP candidate list at least one of after time-based candidates or before history-based candidates, and means for coding the current PU based on the AMVP candidate list.

[0010]

[0010] In another example, the method includes determining an advanced motion vector predictor (AMVP) candidate, wherein at least one candidate in the AMVP candidate list is derived from a non-adjacent candidate unit, wherein the non-adjacent candidate unit is not adjacent to the current predictor unit (PU), and coding the current PU based on the AMVP candidate list.

[0011]

[0011] In another example, the device includes a memory configured to store video data and one or more processors implemented in the circuit and communicatively coupled to the memory, the one or more processors being configured to perform any of the techniques of the present disclosure.

[0012]

[0012] In another example, the device includes at least one means for carrying out any of the techniques of the present disclosure.

[0013]

[0013] In another example, a computer-readable storage medium is encoded with instructions that cause a programmable processor to perform any of the techniques of the present disclosure when executed.

[0014]

[0014] Details of one or more examples are described in the accompanying drawings and the following description. Other features, purposes, and advantages will become apparent from the description, drawings, and claims. [Brief explanation of the drawing]

[0015] [Figure 1]

[0015] A block diagram showing an exemplary video coding and decoding system capable of implementing the techniques of the present disclosure. [Figure 2A]

[0016] A conceptual diagram illustrating an exemplary quad-tree binary tree (QTBT) structure. [Figure 2B] A conceptual diagram showing the corresponding coding tree unit (CTU). [Figure 3]

[0017] A block diagram showing an exemplary video encoder capable of implementing the techniques of this disclosure. [Figure 4]

[0018] A block diagram illustrating an exemplary video decoder capable of implementing the techniques of this disclosure. [Figure 5]

[0019] A conceptual diagram showing exemplary spatially adjacent motion vector candidates for the Advanced Motion Vector Predictor (AMVP) mode. [Figure 6A]

[0020] A conceptual diagram showing an exemplary candidate for a time-motion vector predictor. [Figure 6B] A conceptual diagram illustrating the scaling of motion vectors. [Figure 7]

[0021] A conceptual diagram showing an exemplary decoding flowchart for history-based motion vector prediction. [Figure 8]

[0022] A conceptual diagram illustrating template matching performed on the search area around the initial motion vector. [Figure 9]

[0023] A conceptual diagram showing the locations of non-adjacent AMVP candidate units. [Figure 10]

[0024] A flowchart illustrating an exemplary AMVP candidate list technique according to one or more aspects of the present disclosure. [Figure 11]

[0025] A flowchart showing an exemplary method for encoding a current block according to the techniques of the present disclosure. [Figure 12]

[0026] A flowchart showing an exemplary method for decoding a current block according to the techniques of the present disclosure.

Mode for Carrying Out the Invention

[0016]

[0027] Template matching (TM) can be used to refine advanced motion vector predictor (AMVP) candidates after the AMVP candidate list has been constructed. However, in some exemplary systems, the AMVP candidates are derived or determined from history-based motion information obtained from spatial neighbor blocks, collocated temporal blocks, bottom-right temporal blocks, or recently decoded prediction units (PUs). Thus, it is quite possible that all of these potential AMVP candidates contain very similar motion information, which can be detrimental to the diversity of the AMVP candidate list. Further, in some exemplary systems, the AMVP candidate list contains a maximum of two candidates. Thus, the ability to include diverse AMVP candidates is further reduced or limited.

[0017]

[0028] This specification describes techniques for constructing AMVP candidate lists and signaling AMVP indices that can improve candidate diversity in AMVP candidate lists and result in more efficient coding. For example, a video coder (e.g., an encoder or decoder) may construct an AMVP candidate list containing one or more non-adjacent candidates. One or more non-adjacent candidates may be located in the AMVP candidate list after any time candidate and / or before any history-based candidate. In some examples, the length of an AMVP candidate list may be greater than that of two candidates, such as the length of five candidates. By including one or more non-adjacent candidates in an AMVP candidate list, the diversity of motion information of candidates in the AMVP candidate list can be increased, which can result in more efficient coding.

[0018]

[0029] Figure 1 is a block diagram illustrating an exemplary video coding and decoding system 100 capable of implementing the techniques of the present disclosure. The techniques of the present disclosure generally concern coding (encoding and / or decoding) video data. Generally, video data includes any data for processing video. Thus, video data may include raw, unencoded video, encoded video, decoded (e.g., reconstructed) video, and video metadata such as signaling data.

[0019]

[0030] As shown in Figure 1, system 100 includes, in this example, a source device 102 that provides encoded video data to be decoded and displayed by a destination device 116. In particular, the source device 102 provides the video data to the destination device 116 via a computer-readable medium 110. The source device 102 and the destination device 116 may comprise any of a wide range of devices, including desktop computers, notebook (i.e., laptop) computers, mobile devices, tablet computers, set-top boxes, telephone handsets such as smartphones, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, broadcast receiver devices, and the like. In some cases, the source device 102 and the destination device 116 may be equipped for wireless communication and are therefore sometimes referred to as wireless communication devices.

[0020]

[0031] In the example in Figure 1, the source device 102 includes a video source 104, memory 106, a video encoder 200, and an output interface 108. The destination device 116 includes an input interface 122, a video decoder 300, memory 120, and a display device 118. According to this disclosure, the video encoder 200 of the source device 102 and the video decoder 300 of the destination device 116 may be configured to apply techniques for building an AMVP candidate list and signaling an AMVP index. Thus, the source device 102 represents an example of a video encoding device, and the destination device 116 represents an example of a video decoding device. In other examples, the source and destination devices may include other components or arrangements. For example, the source device 102 may receive video data from an external video source, such as an external camera. Similarly, the destination device 116 may interface with an external display device rather than including an integrated display device.

[0021]

[0032] The system 100 shown in Figure 1 is merely an example. In general, any digital video coding and / or decoding device may perform techniques for building AMVP candidate lists and signaling AMVP indices. Source device 102 and destination device 116 are merely examples of coding devices such that source device 102 generates coded video data for transmission to destination device 116. This disclosure refers to a “coding” device as a device that performs coding (encoding and / or decoding) of data. Thus, video encoder 200 and video decoder 300 represent examples of coding devices, in particular, a video encoder and a video decoder, respectively. In some examples, source device 102 and destination device 116 may operate substantially symmetrically such that each of source device 102 and destination device 116 includes video coding components and video decoding components. Thus, system 100 may support one-way or two-way video transmission between source device 102 and destination device 116 for, for example, video streaming, video playback, video broadcasting, or video telephony.

[0022]

[0033] Generally, the video source 104 represents a source of video data (i.e., raw, unencoded video data), providing the video encoder 200 with a sequence of pictures (also called "frames") of video data, which the video encoder 200 then encodes for the pictures. The video source 104 of the source device 102 may include video capture devices, such as a video camera, a video archive containing previously captured raw video, and / or a video feed interface for receiving video from a video content provider. As a further alternative, the video source 104 may generate computer graphics-based data as source video, or a combination of live video, archived video, and computer-generated video. In each case, the video encoder 200 encodes the captured video data, pre-captured video data, or computer-generated video data. The video encoder 200 may rearrange the pictures from the reception order (sometimes called the "display order") to the coding order for encoding. The video encoder 200 may generate a bitstream containing the encoded video data. The source device 102 may then output the encoded video data onto a computer-readable medium 110 via the output interface 108 for reception and / or retrieval by the input interface 122 of the destination device 116, for example.

[0023]

[0034] Memory 106 of source device 102 and memory 120 of destination device 116 represent general-purpose memory. In some examples, memories 106 and 120 may store raw video data, for example, raw video from video source 104 and raw decoded video data from video decoder 300. Additional or alternative, memories 106 and 120 may store, for example, software instructions executable by video encoder 200 and video decoder 300, respectively. Although memories 106 and 120 are shown separately from video encoder 200 and video decoder 300 in this example, it should be understood that video encoder 200 and video decoder 300 may also include internal memory for functionally similar or equivalent purposes. Furthermore, memories 106 and 120 may store encoded video data, for example, output from video encoder 200 and input to video decoder 300. In some examples, portions 106 and 120 of memory may be allocated as one or more video buffers to store, for example, raw decoded and / or encoded video data.

[0024]

[0035] The computer-readable medium 110 may represent any type of medium or device capable of transporting encoded video data from the source device 102 to the destination device 116. For example, the computer-readable medium 110 may represent a communication medium that enables the source device 102 to directly transmit encoded video data to the destination device 116 in real time, for example, over a radio frequency network or a computer-based network. The output interface 108 may modulate the transmission signal containing the encoded video data, and the input interface 122 may demodulate the received transmission signal according to a communication standard such as a wireless communication protocol. The communication medium may comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful in facilitating communication from the source device 102 to the destination device 116.

[0025]

[0036] In some examples, the source device 102 may output encoded data to the storage device 112 via the output interface 108. Similarly, the destination device 116 may access encoded data from the storage device 112 via the input interface 122. The storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray® disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0026]

[0037] In some examples, the source device 102 may output the encoded video data to a file server 114 or another intermediate storage device capable of storing the encoded video data generated by the source device 102. The destination device 116 may access the stored video data from the file server 114 via streaming or download.

[0027]

[0038] The file server 114 can be any type of server device capable of storing encoded video data and transmitting that encoded video data to the destination device 116. The file server 114 may represent a web server (for example, for a website), a server configured to provide file transfer protocol services (such as the File Transfer Protocol (FTP) or File Delivery over Unidirectional Transport (FLUTE) protocol), a Content Delivery Network (CDN) device, a Hypertext Transfer Protocol (HTTP) server, a Multimedia Broadcast Multicast Service (MBMS) or Enhanced MBMS (eMBMS) server, and / or a Network Attached Storage (NAS) device. The file server 114 may, in addition or alternatively, implement one or more HTTP streaming protocols, such as Dynamic Adaptive Streaming over HTTP (DASH), HTTP Live Streaming (HLS), Real-Time Streaming Protocol (RTSP), or HTTP Dynamic Streaming.

[0028]

[0039] The destination device 116 may access the encoded video data from the file server 114 through any standard data connection, including an internet connection. This may include wireless channels (e.g., Wi-Fi® connection), wired connections (e.g., digital subscriber line (DSL), cable modem, etc.), or a combination of both, which are suitable for accessing the encoded data stored in the file server 114. The input interface 122 may be configured to operate according to one or more of the various protocols described above for retrieving or receiving media data from the file server 114, or any other such protocols for retrieving media data.

[0029]

[0040] The output interface 108 and input interface 122 may represent a wireless transmitter / receiver, a modem, a wired networking component (e.g., an Ethernet® card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples where the output interface 108 and input interface 122 include wireless components, the output interface 108 and input interface 122 may be configured to transfer data such as encoded video data according to cellular communication standards such as 4G, 4G-LTE® (Long-Term Evolution), LTE Advanced, or 5G. In some examples where the output interface 108 includes a wireless transmitter, the output interface 108 and input interface 122 may be configured to transfer data such as encoded video data according to other wireless standards such as the IEEE 802.11 specification, the IEEE 802.15 specification (e.g., ZigBee®), or the Bluetooth® standard. In some examples, the source device 102 and / or destination device 116 may include their respective system-on-chip (SoC) devices. For example, the source device 102 may include an SoC device for performing functions associated with the video encoder 200 and / or the output interface 108, and the destination device 116 may include an SoC device for performing functions associated with the video decoder 300 and / or the input interface 122.

[0030]

[0041] The techniques of this disclosure can be applied to video coding that supports any of a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, internet streaming video transmission such as Dynamic Adaptive Streaming over HTTP (DASH), digital video encoded on a data storage medium, decoding of digital video stored on a data storage medium, or other applications.

[0031]

[0042] The input interface 122 of the destination device 116 receives an encoded video bitstream from a computer-readable medium 110 (e.g., a communication medium, a storage device 112, a file server 114, etc.). The encoded video bitstream may include signaling information defined by the video encoder 200, which is also used by the video decoder 300, such as syntax elements having values ​​that describe the characteristics and / or processing of video blocks or other coded units (e.g., slices, pictures, picture groups, sequences, etc.). The display device 118 displays the decoded picture of the decoded video data to the user. The display device 118 may represent any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0032]

[0043] Although not shown in Figure 1, in some examples, the video encoder 200 and video decoder 300 may be integrated with an audio encoder and / or audio decoder, respectively, and may include a suitable MUX-DEMUX unit or other hardware and / or software to handle multiplexed streams containing both audio and video in a common data stream. Where applicable, the MUX-DEMUX unit may comply with the ITU H.223 multiplexer protocol or other protocols such as the User Datagram Protocol (UDP).

[0033]

[0044] The video encoder 200 and video decoder 300 can each be implemented as one or more suitable encoder and / or decoder circuits, or any combination thereof, including one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, etc. When the technique is partially implemented in software, the device may store software instructions in a suitable non-temporary computer-readable medium and use one or more processors to execute those instructions in hardware to implement the technique of the Disclosure. Each of the video encoder 200 and video decoder 300 may comprise one or more encoders or decoders, any of which may be integrated as part of a composite encoder / decoder (CODEC) in each device. A device comprising the video encoder 200 and / or video decoder 300 may include an integrated circuit, a microprocessor, and / or a wireless communication device such as a cellular telephone.

[0034]

[0045] The video encoder 200 and video decoder 300 may operate in accordance with video coding standards such as ITU-T H.265, also known as High Efficiency Video Coding (HEVC), or its extensions such as the Multiview and / or Scalable Video Coding Extension. Alternatively, the video encoder 200 and video decoder 300 may operate in accordance with other proprietary or industry standards, such as ITU-T H.266, also known as General Purpose Video Coding (VVC). A draft of the VVC standard is described in Bross et al., "Versatile Video Coding (Draft 10)," JVET-T2001-v1 (hereinafter, "VVC Draft 10"), 20th meeting of the Joint Video Expert Team (JVET) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, held remotely from 7-16 October 2020. However, the techniques of this disclosure are not limited to any particular coding standard.

[0035]

[0046] Generally, the video encoder 200 and video decoder 300 may perform block-based coding of pictures. The term “block” generally refers to a structure containing data to be processed (e.g., to be encoded, decoded, or otherwise used in the encoding and / or decoding process). For example, a block may contain a two-dimensional matrix of samples of luminance and / or chrominance data. Generally, the video encoder 200 and video decoder 300 may code video data represented in YUV (e.g., Y, Cb, Cr) format. That is, rather than coding red, green, and blue (RGB) data for the samples of a picture, the video encoder 200 and video decoder 300 may code luminance and chrominance components, where the chrominance component may include both red and blue chrominance components. In some examples, the video encoder 200 converts the received RGB format data to a YUV representation before encoding, and the video decoder 300 converts the YUV representation to RGB format. Alternatively, pre-processing and post-processing units (not shown) may perform these conversions.

[0036]

[0047] This disclosure may refer to coding a picture (e.g., encoding and decoding) to include, in general, the process of encoding or decoding the data of a picture. Similarly, this disclosure may refer to coding a block of a picture to include, for example, the process of encoding or decoding data for a block, for example, predictive and / or residual coding. An encoded video bitstream generally contains a set of values ​​for syntax elements that represent coding decisions (e.g., coding modes) and divisions of the picture into blocks. Thus, references to coding a picture or a block should generally be understood as coding the values ​​of the syntax elements that make up the picture or block.

[0037]

[0048] HEVC defines various blocks, including coding units (CUs), prediction units (PUs), and transformation units (TUs). According to HEVC, a video coder (such as video encoder 200) divides the coding tree units (CTUs) into CUs according to a quad-tree structure. That is, the video coder divides the CTUs and CUs into four equal, non-overlapping squares, and each node in the quad-tree has either zero or four child nodes. Nodes without child nodes are sometimes called "leaf nodes," and the CU of such a leaf node may contain one or more PUs and / or one or more TUs. The video coder may further divide the PUs and TUs. For example, in HEVC, the residual quad-tree (RQT) represents a division of the TUs. In HEVC, PUs represent intra-predicted data, while TUs represent residual data. Intra-predicted CUs contain intra-predicted information, such as intra-mode indications.

[0038]

[0049] As another example, a video encoder 200 and a video decoder 300 may be configured to operate according to VVC. According to VVC, a video coder (such as the video encoder 200) divides a picture into multiple CTUs. The video encoder 200 may divide the CTUs according to a tree structure, such as a quad-tree binary tree (QTBT) structure or a multi-type tree (MTT) structure. The QTBT structure eliminates the concept of multiple division types, such as the separation between CUs, PUs, and TUs in HEVC. The QTBT structure includes two levels: a first level divided according to a quad-tree division and a second level divided according to a binary tree division. The root node of the QTBT structure corresponds to a CTU. The leaf nodes of the binary tree correspond to a CU.

[0039]

[0050] In an MTT partitioning structure, blocks can be partitioned using quad-tree (QT) partitions, binary-tree (BT) partitions, and one or more types of triple-tree (TT) (also called terminally-tree (TT)) partitions. A triple-tree or terminally-tree partition is a partition in which a block is split into three sub-blocks. In some examples, a triple-tree or terminally-tree partition divides a block into three sub-blocks without splitting the original block through a center. The partitioning types in an MTT (e.g., QT, BT, and TT) can be symmetric or asymmetric.

[0040]

[0051] In some examples, the video encoder 200 and video decoder 300 may use a single QTBT or MTT structure to represent the luminance component and the chrominance component, respectively, while in other examples, the video encoder 200 and video decoder 300 may use two or more QTBT or MTT structures, such as one QTBT / MTT structure for the luminance component and another QTBT / MTT structure for both chrominance components (or two QTBT / MTT structures for each chrominance component).

[0041]

[0052] The video encoder 200 and video decoder 300 may be configured to use a quad-tree segment, QTBT segment, MTT segment, or other segmentation structure that conforms to HEVC. For illustrative purposes, the description of the techniques of this disclosure is presented in relation to the QTBT segment. However, it should be understood that the techniques of this disclosure may also be applicable to video coders configured to use a quad-tree segment, or similarly other types of segmentation.

[0042]

[0053] In some examples, the CTU includes a coding tree block (CTB) of a lumen sample, two corresponding CTBs of a chroma sample of a picture having three sample arrays, or a CTB of a sample of a monochrome picture, or a picture coded using three separate color planes and syntax structures used to code the sample. The CTB can be an N×N block of samples, for some value N such that the division of components into the CTB is a partition. The components are an array or a single sample from one of three arrays (lumen and two chroma) that constitute the picture in a 4:2:0, 4:2:2, or 4:4:4 color format, or an array or a single sample of an array that constitutes the picture in a monochrome format. In some examples, the coding block is an M×N block of samples, for some values ​​M and N such that the division of the CTB into the coding block is a partition.

[0043]

[0054] Blocks (e.g., CTUs or CUs) can be grouped in various ways within a picture. For example, a brick may refer to a rectangular area of ​​a row of CTUs within a particular tile in a picture. A tile can be a rectangular area of ​​CTUs within a particular tile column and a particular tile row in a picture. A tile column refers to a rectangular area of ​​CTUs having a height equal to the height of the picture and a width specified by a syntax element (e.g., in a picture parameter set). A tile row refers to a rectangular area of ​​CTUs having a height specified by a syntax element (e.g., in a picture parameter set) and a width equal to the width of the picture.

[0044]

[0055] In some examples, a tile may be divided into multiple bricks, each of which may contain one or more CTU rows. A tile that is not divided into multiple bricks may also be called a brick. However, a brick that is a true subset of a tile may not be called a tile.

[0045]

[0056] Bricks within a picture can also be placed within a slice. A slice can be an integer number of bricks in a picture, which may be contained exclusively within a single Network Abstraction Layer (NAL) unit. In some examples, a slice may contain either several complete tiles or just a continuous sequence of complete bricks of a single tile.

[0046]

[0057] This disclosure allows the terms "N×N(NxN)" and "N×N(N by N)" to be used interchangeably to refer to the sample dimensions of a block (such as a CU or other video block) with respect to vertical and horizontal dimensions, for example, 16×16 samples or 16×16 samples. Generally, a 16×16 CU has 16 samples vertically (y=16) and 16 samples horizontally (x=16). Similarly, an N×N CU generally has N samples vertically and N samples horizontally, where N represents a non-negative integer. Samples in a CU can be arranged in rows and columns. Furthermore, a CU does not necessarily have to have the same number of samples horizontally as vertically. For example, a CU may have N×M samples, where M is not necessarily equal to N.

[0047]

[0058] The video encoder 200 encodes video data about CUs, which represent prediction and / or residual information, as well as other information. The prediction information indicates how the CUs should be predicted in order to form prediction blocks. The residual information generally represents the sample-by-sample difference between the CU samples before encoding and the prediction blocks.

[0048]

[0059] To predict a CU, the video encoder 200 may generally form prediction blocks for the CU through inter-prediction or intra-prediction. Inter-prediction generally refers to predicting the CU from data of a previously coded picture, while intra-prediction generally refers to predicting the CU from previously coded data of the same picture. To perform inter-prediction, the video encoder 200 may generate prediction blocks using one or more motion vectors. The video encoder 200 may generally perform motion search to identify a reference block that exactly matches the CU or coding block, for example, with respect to the difference between the CU or coding block and a reference block. The video encoder 200 may compute a difference metric using absolute difference sum (SAD), squared difference sum (SSD), mean absolute difference (MAD), mean squared difference (MSD), or other such difference calculations to determine whether a reference block exactly matches the current CU. In some examples, the video encoder 200 may predict the current CU using unidirectional or bidirectional prediction.

[0049]

[0060] Some examples of VVC also offer an affine motion compensation mode, which can be considered an interpredictive mode. In affine motion compensation mode, the video encoder 200 may determine two or more motion vectors representing non-translational motion, such as zooming in or out, rotation, perspective motion, or other anomalous motion types.

[0050]

[0061] To perform intra-prediction, the video encoder 200 may select an intra-prediction mode to generate a prediction block. Several examples of VVCs offer 67 intra-prediction modes, including various directional modes, as well as planar and DC modes. Generally, the video encoder 200 selects an intra-prediction mode that describes adjacent samples to the current block (e.g., a block of CUs) from which samples of the current block should be predicted. Such samples could generally be above, to the upper left, or to the left of the current block in the same picture as the current block, assuming the video encoder 200 codes CTUs and CUs in raster scan order (left to right, top to bottom).

[0051]

[0062] The video encoder 200 encodes data representing the prediction mode of the current block. For example, in interprediction mode, the video encoder 200 may encode data representing which of the various available interprediction modes is used, as well as motion information for the corresponding mode. For example, in unidirectional or bidirectional interprediction, the video encoder 200 may encode motion vectors using AMVP or merge mode. The video encoder 200 may use similar modes to encode motion vectors for affine motion compensation mode.

[0052]

[0063] Following predictions such as intra-prediction or inter-prediction of a block, the video encoder 200 may compute residual data for the block. Residual data, such as residual blocks, represents the sample-by-sample difference between the block and the predicted block for the block, formed using the corresponding prediction mode. The video encoder 200 may apply one or more transformations to the residual blocks to produce transformation data in the transformation domain rather than the sample domain. For example, the video encoder 200 may apply a discrete cosine transform (DCT), integer transform, wavelet transform, or a conceptually similar transform to the residual video data. Furthermore, the video encoder 200 may apply a quadratic transform following a first transform, such as a mode-dependent inseparable quadratic transform (MDNSST), signal-dependent transform, or Carunenlobe transform (KLT). The video encoder 200 produces transformation coefficients following the application of one or more transformations.

[0053]

[0064] As described above, following any transformation to produce the transformation coefficients, the video encoder 200 may perform quantization of the transformation coefficients. Quantization generally refers to the process of further compression in which the transformation coefficients are quantized to reduce the amount of data used to represent them as much as possible. By performing the quantization process, the video encoder 200 may reduce the bit depth associated with some or all of the transformation coefficients. For example, the video encoder 200 may truncate an n-bit value to an m-bit value during quantization, where n is greater than m. In some examples, to perform quantization, the video encoder 200 may perform a bitwise right shift of the value to be quantized.

[0054]

[0065] Following quantization, the video encoder 200 may scan the transformation coefficients to create a one-dimensional vector from a two-dimensional matrix containing the quantized transformation coefficients. The scan may be designed to place higher-energy (and therefore lower-frequency) transformation coefficients at the beginning of the vector and lower-energy (and therefore higher-frequency) transformation coefficients at the end. In some examples, the video encoder 200 may utilize a predefined scan order to scan the quantized transformation coefficients to create a serialized vector, and then entropically encode the quantized transformation coefficients of the vector. In other examples, the video encoder 200 may perform adaptive scanning. After scanning the quantized transformation coefficients to form a one-dimensional vector, the video encoder 200 may entropically encode the one-dimensional vector, for example, according to context-adaptive binary arithmetic coding (CABAC). The video encoder 200 may also entropically encode the values ​​of syntax elements describing metadata associated with the encoded video data for use by the video decoder 300 when decoding the video data.

[0055]

[0066] To perform CABAC, the video encoder 200 may assign a context within a context model to the symbols to be transmitted. This context may relate, for example, to whether the symbol's adjacency value is zero. Probability decisions may be based on the context assigned to the symbols.

[0056]

[0067] The video encoder 200 may further generate syntax data for the video decoder 300, such as block-based syntax data, picture-based syntax data, and sequence-based syntax data, in other syntax data such as picture headers, block headers, slice headers, or sequence parameter sets (SPS), picture parameter sets (PPS), or video parameter sets (VPS). The video decoder 300 may similarly decode such syntax data to determine how the corresponding video data should be decoded.

[0057]

[0068] In this way, the video encoder 200 can generate a bitstream containing encoded video data, for example, a bitstream containing syntax elements that describe the division of a picture into blocks (e.g., CUs) and predictive and / or residual information about the blocks. Finally, the video decoder 300 can receive the bitstream and decode the encoded video data.

[0058]

[0069] Generally, the video decoder 300 performs the reverse process of what the video encoder 200 did to decode the encoded video data of the bitstream. For example, the video decoder 300 may decode values ​​for syntax elements of the bitstream using CABAC in a substantially similar manner to, but in reverse, the CABAC encoding process of the video encoder 200. The syntax elements may define partitioning information for partitioning a picture into CTUs, and partitions for each CTU, following a corresponding partitioning structure such as a QTBT structure, in order to define the CUs of the CTUs. The syntax elements may further define prediction and residual information for blocks of video data (e.g., CUs).

[0059]

[0070] Residual information may be represented, for example, by quantized transformation coefficients. The video decoder 300 may dequantize and inverse transform the quantized transformation coefficients of a block in order to reconstruct the residual block for the block. The video decoder 300 uses a signaled prediction mode (intra or inter-prediction) and associated prediction information (for example, motion information for inter-prediction) to form a prediction block for the block. The video decoder 300 may then combine the prediction block and the residual block (sample by sample) to reconstruct the original block. The video decoder 300 may perform additional processing, such as performing a deblocking process to reduce visual artifacts along the block boundaries.

[0060]

[0071] According to the techniques of the present disclosure, the method includes determining at least one of time-based candidates or history-based candidates; determining at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein the at least one non-adjacent candidate determines an advanced motion vector predictor (AMVP) candidate list comprising at least one of time-based candidates or history-based candidates and at least one non-adjacent candidate that are not adjacent to the current predictor unit (PU); wherein the at least one non-adjacent candidate codes the current PU based on the AMVP candidate list, which is added to the AMVP candidate list at least one of after the time-based candidate or before the history-based candidate.

[0061]

[0072] According to the techniques of the present disclosure, the device includes a memory configured to store video data and one or more processors implemented in the circuit and communicatively coupled to the memory, wherein the one or more processors are configured to determine at least one of time-based candidates or history-based candidates; to determine at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein the at least one non-adjacent candidate unit determines an advanced motion vector predictor (AMVP) candidate list comprising at least one time-based candidate or history-based candidate and at least one non-adjacent candidate that is not adjacent to the current predictor unit (PU), wherein the at least one non-adjacent candidate codes the current PU based on the AMVP candidate list, which is added to the AMVP candidate list at least one of after the time-based candidate or before the history-based candidate.

[0062]

[0073] According to the techniques of the present disclosure, a non-temporary computer-readable storage medium stores instructions that, when executed, cause one or more processors to determine at least one of time-based candidates or history-based candidates; determine at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein the at least one non-adjacent candidate unit determines an advanced motion vector predictor (AMVP) candidate list comprising at least one time-based candidate or history-based candidate and at least one non-adjacent candidate that is not adjacent to the current predictor unit (PU); wherein the at least one non-adjacent candidate codes the current PU based on the AMVP candidate list, which is added to the AMVP candidate list at least one of after the time-based candidate or before the history-based candidate.

[0063]

[0074] According to the techniques of the present disclosure, the device includes means for determining at least one of time-based candidates or history-based candidates; means for determining at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein the at least one non-adjacent candidate unit is means for determining an advanced motion vector predictor (AMVP) candidate list comprising at least one of time-based candidates or history-based candidates and at least one non-adjacent candidate that is not adjacent to the current predictor unit (PU); wherein the at least one non-adjacent candidate is added to the AMVP candidate list at least one of after the time-based candidate or before the history-based candidate, and means for coding the current PU based on the AMVP candidate list.

[0064]

[0075] According to the technique of the present disclosure, the method includes determining an AMVP candidate list, wherein at least one candidate in the AMVP candidate list is derived from a non-adjacent candidate unit, wherein the non-adjacent candidate unit is not adjacent to the current PU, and coding the current PU based on the AMVP candidate list.

[0065]

[0076] According to the techniques of the present disclosure, the device includes a memory configured to store video data and one or more processors implemented in the circuit and communicatively coupled to the memory, the one or more processors being configured to implement any of the techniques of the present disclosure.

[0066]

[0077] According to the techniques of this disclosure, the device includes at least one means for carrying out any of the techniques of this disclosure.

[0067]

[0078] According to the techniques of this disclosure, a computer-readable storage medium is encoded with instructions that, when executed, cause a programmable processor to perform any of the techniques of this disclosure.

[0068]

[0079] This disclosure may generally refer to “signaling” or “parsing” certain information, such as syntax elements. The term “signaling” may generally refer to the communication of values ​​for syntax elements and / or other data used to decode encoded video data. That is, the video encoder 200 may signal values ​​for syntax elements in a bitstream. Generally, signaling refers to generating values ​​in a bitstream. As described above, the source device 102 may transport the bitstream to the destination device 116, either substantially in real time or not in real time, when storing the syntax elements in the storage device 112 for later retrieval by the destination device 116. The term “parsing” refers to determining the values ​​of the signaled syntax elements and / or other data. For example, the video decoder 300 may parse the signaled syntax elements to determine the values ​​of the signaled syntax elements.

[0069]

[0080] Figures 2A and 2B are conceptual diagrams showing an exemplary QTBT structure 130 and its corresponding CTU 132. Solid lines represent quad-tree splitting, and dotted lines represent binary tree splitting. At each split (i.e., non-leaf) node of the binary tree, one flag is signaled to indicate which splitting type (i.e., horizontal or vertical) is used, where in this example, 0 indicates horizontal splitting and 1 indicates vertical splitting. In quad-tree splitting, the quad-tree node splits a block horizontally and vertically into four subblocks of equal size, so there is no need to indicate the splitting type. Thus, the video encoder 200 can encode (i.e., solid lines) syntax elements (such as splitting information) for the region tree level of the QTBT structure 130, and the video decoder 300 can decode (i.e., dashed lines) syntax elements (such as splitting information) for the prediction tree level of the QTBT structure 130. The video encoder 200 can encode video data, such as prediction and transformation data, for CUs represented by terminal leaf nodes of the QTBT structure 130, and the video decoder 300 can decode it.

[0070]

[0081] Generally, the CTU132 in Figure 2B can be associated with parameters that define the size of the blocks corresponding to the nodes of the QTBT structure 130 at the first and second levels. These parameters may include the CTU size (representing the size of the CTU132 in the sample), the minimum quad tree size (MinQTSize, representing the minimum allowable quad tree leaf node size), the maximum binary tree size (MaxBTSize, representing the maximum allowable binary tree root node size), the maximum binary tree depth (MaxBTDepth, representing the maximum allowable binary tree depth), and the minimum binary tree size (MinBTSize, representing the minimum allowable binary tree leaf node size).

[0071]

[0082] The root node of a QTBT structure corresponding to a CTU may have four child nodes at the first level of the QTBT structure, each of which may be subdivided according to a quad-tree partition. That is, a node at the first level is either a leaf node (without child nodes) or has four child nodes. An example of QTBT structure 130 represents a node that includes a parent node and child nodes with solid lines for branching. If the node at the first level is not larger than the maximum allowable binary tree root node size (MaxBTSize), the node may be further subdivided by its respective binary tree. Binary tree splitting of a node may be repeated until the nodes resulting from the split reach the minimum allowable binary tree leaf node size (MinBTSize) or the maximum allowable binary tree depth (MaxBTDepth). An example of QTBT structure 130 represents a node with dashed lines for branching. Binary tree leaf nodes are called coding units (CUs), and CUs are used for prediction (e.g., intra-picture or inter-picture prediction) and transformation without further subdivision. As explained above, CU is sometimes called a "video block" or "block".

[0072]

[0083] In one example of a QTBT partition structure, the CTU size is set to 128×128 (a chroma sample and two corresponding 64×64 chroma samples), MinQTSize is set to 16×16, MaxBTSize is set to 64×64, MinBTSize (for both width and height) is set to 4, and MaxBTDepth is set to 4. The quadtree partition is first applied to the CTU to generate quadtree leaf nodes. Quadtree leaf nodes can have sizes ranging from 16×16 (i.e., MinQTSize) to 128×128 (i.e., CTU size). If a quadtree leaf node is 128×128, the leaf quadtree node is not further split by the binary tree because its size exceeds MaxBTSize (i.e., 64×64 in this example). In other cases, the quadtree leaf node is further partitioned by the binary tree. Therefore, a quad-tree leaf node is also the root node for a binary tree and has a binary tree depth of 0. When the binary tree depth reaches MaxBTDepth (4 in this example), no further splitting is allowed. A binary tree node with a width equal to MinBTSize (4 in this example) implies that no further vertical splitting (i.e., splitting of width) is allowed for that binary tree node. Similarly, a binary tree node with a height equal to MinBTSize implies that no further horizontal splitting (i.e., splitting of height) is allowed for that binary tree node. As mentioned above, leaf nodes of a binary tree are called CUs and are further processed according to prediction and transformation without further division.

[0073]

[0084] Figure 3 is a block diagram illustrating an exemplary video encoder 200 capable of implementing the techniques of this disclosure. Figure 3 is provided for illustrative purposes only and should not be considered as limiting the techniques broadly illustrated and described in this disclosure. For illustrative purposes, this disclosure describes the video encoder 200 in accordance with the VVC (ITU-T H.266) and HEVC (ITU-T H.265) techniques. However, the techniques of this disclosure may be implemented by video encoding devices configured to other video coding standards or proprietary coding techniques.

[0074]

[0085] In the example shown in Figure 3, the video encoder 200 includes a video data memory 230, a mode selection unit 202, a residual generation unit 204, a conversion processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse conversion processing unit 212, a reconstruction unit 214, a filter unit 216, a decoded picture buffer (DPB) 218, and an entropy coding unit 220. Any or all of the video data memory 230, the mode selection unit 202, the residual generation unit 204, the conversion processing unit 206, the quantization unit 208, the inverse quantization unit 210, the inverse conversion processing unit 212, the reconstruction unit 214, the filter unit 216, the DPB 218, and the entropy coding unit 220 may be implemented in one or more processors or processing circuits. For example, a unit of the video encoder 200 may be implemented as one or more circuit or logic elements, as part of a hardware circuit configuration, or as part of a processor, ASIC, or FPGA. Furthermore, the video encoder 200 may include additional or alternative processors or processing circuits to perform these and other functions.

[0075]

[0086] The video data memory 230 can store video data to be encoded by the components of the video encoder 200. The video encoder 200 can receive video data stored in the video data memory 230 from, for example, a video source 104 (Figure 1). The DPB 218 can act as a reference picture memory that stores reference video data for use in predicting subsequent video data by the video encoder 200. The video data memory 230 and the DPB 218 can be formed by any of various memory devices, such as dynamic random access memory (DRAM) including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM®), or other types of memory devices. The video data memory 230 and the DPB 218 can be provided by the same memory device or by separate memory devices. In various examples, the video data memory 230 may be on-chip with the other components of the video encoder 200, as shown in the figure, or off-chip relative to those components.

[0076]

[0087] In this disclosure, references to video data memory 230 should not be interpreted as being limited to memory inside the video encoder 200 unless specifically described so, nor should they be interpreted as being limited to memory outside the video encoder 200 unless specifically described so. Rather, references to video data memory 230 should be understood as reference memory that stores video data received by the video encoder 200 for encoding (for example, video data about the current block to be encoded). Memory 106 in Figure 1 may also provide temporary storage for outputs from various units of the video encoder 200.

[0077]

[0088] The various units in Figure 3 are shown to help understand the operations performed by the video encoder 200. The units may be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits refer to circuits that provide a specific function and are pre-configured with respect to the operations they may perform. Programmable circuits refer to circuits that can be programmed to perform various tasks and to provide flexible functionality in the operations they may perform. For example, a programmable circuit may execute software or firmware that operates the programmable circuit in a manner defined by software or firmware instructions. Fixed-function circuits may execute software instructions (e.g., to receive or output parameters), but the type of operation performed by a fixed-function circuit is generally immutable. In some examples, one or more of the units may be separate circuit blocks (fixed-function or programmable), and in some examples, one or more of the units may be integrated circuits.

[0078]

[0089] The video encoder 200 may include a programmable core formed from an arithmetic logic unit (ALU), an EFU, digital circuits, analog circuits, and / or programmable circuits. In an example where the operation of the video encoder 200 is carried out using software executed by the programmable circuits, memory 106 (Figure 1) may store software instructions (e.g., object code) that the video encoder 200 receives and executes, or another memory (not shown) within the video encoder 200 may store such instructions.

[0079]

[0090] The video data memory 230 is configured to store the received video data. The video encoder 200 can retrieve a picture of the video data from the video data memory 230 and provide the video data to the residual generation unit 204 and the mode selection unit 202. The video data in the video data memory 230 may be raw video data to be encoded.

[0080]

[0091] The mode selection unit 202 includes a motion estimation unit 222, a motion compensation unit 224, and an intra-prediction unit 226. The mode selection unit 202 may include additional functional units for performing video prediction according to other prediction modes. For example, the mode selection unit 202 may include a pallet unit, an intra-block copy unit (which may be part of the motion estimation unit 222 and / or the motion compensation unit 224), an affine unit, a linear model (LM) unit, and the like.

[0081]

[0092] The mode selection unit 202 generally coordinates multiple coding paths to test combinations of coding parameters and generates rate distortion values ​​for such combinations. The coding parameters may include the division of the CTU to the CU, the prediction mode for the CU, the transformation type for the residual data of the CU, and quantization parameters for the residual data of the CU. The mode selection unit 202 may ultimately select a combination of coding parameters that has a better rate distortion value than other tested combinations.

[0082]

[0093] The video encoder 200 divides the picture retrieved from the video data memory 230 into a series of CTUs, and may encapsulate one or more CTUs within a slice. The mode selection unit 202 may divide the CTUs of a picture according to a tree structure, such as the HEVC QTBT structure or quad-tree structure described above. As described above, the video encoder 200 may form one or more CUs from dividing the CTUs according to a tree structure. Such CTUs are also sometimes commonly referred to as “video blocks” or “blocks”.

[0083]

[0094] In general, the mode selection unit 202 also controls its components (e.g., motion estimation unit 222, motion compensation unit 224, and intra-prediction unit 226) to generate a predicted block for the current block (e.g., the current CU, or, in HEVC, the overlapping portion of PU and TU). For intra-prediction of the current block, the motion estimation unit 222 may perform a motion search to identify one or more perfectly matching reference blocks in one or more reference pictures (e.g., one or more previously coded pictures stored in DPB 218). In particular, the motion estimation unit 222 may calculate a value representing how similar the potential reference blocks are to the current block, for example, according to the sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute difference (MAD), mean squared difference (MSD), etc. The motion estimation unit 222 may generally perform these calculations using sample-by-sample differences between the current block and the reference blocks under consideration. The motion estimation unit 222 may identify the reference block with the lowest value obtained from these calculations, which indicates the reference block that most closely matches the current block. In some examples, the motion estimation unit 222 may include an AMVP unit 223 that can determine at least one of time-based candidates or history-based candidates. The AMVP unit 223 may also determine at least one non-adjacent candidate from at least one non-adjacent candidate unit, where at least one non-adjacent candidate unit is not adjacent to the current PU. The AMVP unit 223 may determine an AMVP candidate list that includes at least one of the time-based candidates or history-based candidates and at least one non-adjacent candidate. The AMVP unit 223 may add at least one non-adjacent candidate to the AMVP candidate list after the time-based candidates and / or before the history-based candidates.

[0084]

[0095] The motion estimation unit 222 may form one or more motion vectors (MVs) that define the position of a reference block in a reference picture relative to the position of the current block in the current picture. The motion estimation unit 222 may then provide the motion vectors to the motion compensation unit 224. For example, in unidirectional interpretation, the motion estimation unit 222 may provide a single motion vector, while in bidirectional interpretation, the motion estimation unit 222 may provide two motion vectors. The motion compensation unit 224 may then use the motion vectors to generate prediction blocks. Prediction blocks can be blocks of prediction samples and are sometimes called PUs. For example, the motion compensation unit 224 may use the motion vectors to extract data for a reference block. As another example, if the motion vectors have fractional sample precision, the motion compensation unit 224 may interpolate values ​​for the prediction block according to one or more interpolation filters. Furthermore, in bidirectional interpretation, the motion compensation unit 224 may extract data for two reference blocks identified by their respective motion vectors and combine the extracted data, for example, through sample-wise averaging or weighted averaging.

[0085]

[0096] As another example, in intra-prediction, or intra-prediction coding, the intra-prediction unit 226 may generate a prediction block from samples adjacent to the current block. For example, in directional mode, the intra-prediction unit 226 may mathematically combine the values ​​of adjacent samples to produce a prediction block and populate these calculated values ​​in a direction defined across the current block. As another example, in DC mode, the intra-prediction unit 226 may calculate the average of adjacent samples relative to the current block and generate a prediction block so that each sample of the prediction block contains this obtained average.

[0086]

[0097] The mode selection unit 202 provides the residual generation unit 204 with a prediction block, for example, a block of prediction samples. The residual generation unit 204 receives a raw, unencoded version of the current block from the video data memory 230 and the prediction block from the mode selection unit 202. The residual generation unit 204 calculates the sample-by-sample difference between the current block and the prediction block. The resulting sample-by-sample difference defines the residual block for the current block. In some examples, the residual generation unit 204 may also determine the difference between sample values ​​in the residual block to generate the residual block using residual difference pulse code modulation (RDPCM). In some examples, the residual generation unit 204 may be formed using one or more subtractor circuits that perform binary subtraction.

[0087]

[0098] In an example where the mode selection unit 202 divides a CU into PUs, each PU may be associated with a lumar prediction unit and a corresponding chroma prediction unit. The video encoder 200 and video decoder 300 may support PUs of various sizes. As shown above, the size of a CU may refer to the size of the lumar coding block of the CU, and the size of a PU may refer to the size of the lumar prediction unit of the PU. Assuming that the size of a particular CU is 2N×2N, the video encoder 200 may support 2N×2N or N×N PU sizes for intra-prediction and symmetric PU sizes of 2N×2N, 2N×N, N×2N, N×N, or similar for inter-prediction. The video encoder 200 and video decoder 300 may also support asymmetric divisions for PU sizes of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter-prediction.

[0088]

[0099] In an example where the mode selection unit 202 does not further subdivide the CUs into PUs, each CU may be associated with a ruma coding block and a corresponding chroma coding block. As described above, the size of a CU may refer to the size of the ruma coding block of the CU. The video encoder 200 and video decoder 300 may support CU sizes of 2N×2N, 2N×N, or N×2N.

[0089]

[0100] In some examples, such as intra-block copy mode coding, affine mode coding, and linear model (LM) mode coding, the mode selection unit 202 generates a predicted block for the current block being coded via the respective unit associated with the coding technique. In some examples, such as palette mode coding, the mode selection unit 202 may not generate a predicted block, but instead may generate syntax elements indicating the manner in which the block should be reconstructed based on the selected palette. In such modes, the mode selection unit 202 may provide these syntax elements to be coded to the entropy coding unit 220.

[0090]

[0101] As described above, the residual generation unit 204 receives video data for the current block and the corresponding predicted block. The residual generation unit 204 then generates a residual block for the current block. To generate the residual block, the residual generation unit 204 calculates the sample-by-sample difference between the predicted block and the current block.

[0091]

[0102] The transformation processing unit 206 applies one or more transformations to the residual block to generate blocks of transformation coefficients (referred to herein as “transformation coefficient blocks”). The transformation processing unit 206 may apply various transformations to the residual block to form the transformation coefficient blocks. For example, the transformation processing unit 206 may apply a discrete cosine transform (DCT), a directional transform, a Carunenlebe transform (KLT), or a conceptually similar transform to the residual block. In some examples, the transformation processing unit 206 may perform multiple transformations on the residual block, such as linear and quadratic transformations, such as a rotation transform. In some examples, the transformation processing unit 206 does not apply any transformations to the residual block.

[0092]

[0103] The quantization unit 208 may quantize the transformation coefficients in a transformation coefficient block in order to generate a quantized transformation coefficient block. The quantization unit 208 may quantize the transformation coefficients of a transformation coefficient block according to the quantization parameter (QP) value associated with the current block. The video encoder 200 may adjust the degree of quantization applied to the transformation coefficient block associated with the current block by adjusting the QP value associated with the CU (for example, via the mode selection unit 202). Quantization may result in a loss of information, and therefore the quantized transformation coefficients may have lower precision than the original transformation coefficients generated by the transformation processing unit 206.

[0093]

[0104] The inverse quantization unit 210 and the inverse transform processing unit 212 may apply inverse quantization and inverse transform, respectively, to the quantization transform coefficient block in order to reconstruct the residual block from the transform coefficient block. The reconstruction unit 214 may generate a reconstructed block corresponding to the current block (potentially with some degree of distortion) based on the reconstructed residual block and the predicted block generated by the mode selection unit 202. For example, the reconstruction unit 214 may add samples from the reconstructed residual block to the corresponding samples from the predicted block generated by the mode selection unit 202 in order to produce the reconstructed block.

[0094]

[0105] The filter unit 216 may perform one or more filtering operations on the reconstructed block. For example, the filter unit 216 may perform a deblocking operation to reduce blocking artifacts along the edges of the CU. In some examples, the operation of the filter unit 216 may be skipped.

[0095]

[0106] The video encoder 200 stores the reconstructed blocks in the DPB 218. For example, in an example where the filter unit 216 does not operate, the reconstruction unit 214 may store the reconstructed blocks in the DPB 218. In an example where the filter unit 216 operates, the filter unit 216 may store the filtered reconstructed blocks in the DPB 218. The motion estimation unit 222 and the motion compensation unit 224 may retrieve a reference picture formed from the reconstructed (and potentially filtered) blocks from the DPB 218 to interpret blocks of the picture to be encoded later. Furthermore, the intraprediction unit 226 may use the reconstructed blocks in the DPB 218 of the current picture to intrapret other blocks in the current picture.

[0096]

[0107] In general, the entropy coding unit 220 can entropy code syntax elements received from other functional components of the video encoder 200. For example, the entropy coding unit 220 can entropy code a block of quantization conversion coefficients from the quantization unit 208. As another example, the entropy coding unit 220 can entropy code prediction syntax elements from the mode selection unit 202 (e.g., motion information for inter-prediction, or intra-mode information for intra-prediction). The entropy coding unit 220 can perform one or more entropy coding operations on syntax elements, which are another example of video data, to generate entropy coded data. For example, the entropy coding unit 220 may perform context-adaptive variable-length coding (CAVLC) operation, CABAC operation, variable-to-variable (V2V) length coding operation, syntax-based context-adaptive binary arithmetic coding (SBAC) operation, probability interval partitioned entropy (PIPE) coding operation, exponential Golomb coding operation, or another type of entropy coding operation on the data. In some examples, the entropy coding unit 220 may operate in a bypass mode in which syntax elements are not entropically coded.

[0097]

[0108] The video encoder 200 may output a bitstream containing entropy coding syntax elements required to reconstruct the slice or block of the picture. In particular, the entropy coding unit 220 may output a bitstream.

[0098]

[0109] The behavior described above is described in relation to blocks. Such descriptions should be understood as behavior for rumacoding blocks and / or chromacoding blocks. As described above, in some examples the rumacoding block and chromacoding block are the ruma and chroma components of the CU. In some examples the rumacoding block and chromacoding block are the ruma and chroma components of the PU.

[0099]

[0110] In some cases, actions performed for a rumacoding block do not need to be repeated for a chromacoding block. For example, actions to identify the motion vector (MV) and reference picture for a rumacoding block do not need to be repeated to identify the MV and reference picture for a chromablock. Rather, the MV for the rumacoding block may be scaled to determine the MV for the chromablock, and the reference picture may be the same. In another example, an intra-prediction process may be the same for both rumacoding and chromacoding blocks.

[0100]

[0111] The video encoder 200 represents an example of a device configured to encode video data, comprising: a memory configured to store video data; and one or more processing units implemented in the circuit and configured to determine at least one of time-based candidates or history-based candidates; determine at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein the at least one non-adjacent candidate unit determines an advanced motion vector predictor (AMVP) candidate list comprising at least one of time-based candidates or history-based candidates and at least one non-adjacent candidate that is not adjacent to the current predictor unit (PU); wherein the at least one non-adjacent candidate codes the current PU based on the AMVP candidate list, which is added to the AMVP candidate list at least one of after the time-based candidates or before the history-based candidates.

[0101]

[0112] The video encoder 200 also represents an example of a device configured to encode video data, which includes a memory configured to store video data, and one or more processing units implemented in the circuit and configured to determine an AMVP candidate list, wherein at least one candidate in the AMVP candidate list is derived from a non-adjacent candidate unit, wherein at least one non-adjacent candidate unit is not adjacent to the current PU, and encode the current PU based on the AMVP candidate list. In some examples, at least one non-adjacent candidate is after any time candidate and / or before any history-based candidate.

[0102]

[0113] Figure 4 is a block diagram showing an exemplary video decoder 300 capable of performing the techniques of this disclosure. Figure 4 is provided for illustrative purposes and is not intended to limit the techniques that are more broadly illustrated and described in this disclosure. For illustrative purposes, this disclosure describes the video decoder 300 in accordance with the VVC (ITU-T H.266) and HEVC (ITU-T H.265) techniques. However, the techniques of this disclosure may be performed by video coding devices configured to operate in accordance with other video coding standards.

[0103]

[0114] In the example in Figure 4, the video decoder 300 includes a coded picture buffer (CPB) memory 320, an entropy decoding unit 302, a prediction processing unit 304, an inverse quantization unit 306, an inverse transformation processing unit 308, a reconstruction unit 310, a filter unit 312, and a decoded picture buffer (DPB) 314. Any or all of the CPB memory 320, the entropy decoding unit 302, the prediction processing unit 304, the inverse quantization unit 306, the inverse transformation processing unit 308, the reconstruction unit 310, the filter unit 312, and the DPB 314 may be implemented in one or more processors or processing circuits. For example, the units of the video decoder 300 may be implemented as one or more circuits or logic elements, as part of a hardware circuit, or as part of a processor, ASIC, or FPGA. Furthermore, the video decoder 300 may include additional or alternative processors or processing circuits to perform these and other functions.

[0104]

[0115] The prediction processing unit 304 includes a motion compensation unit 316 and an intra-prediction unit 318. The prediction processing unit 304 may include additional units for performing predictions according to other prediction modes. For example, the prediction processing unit 304 may include a pallet unit, an intra-block copy unit (which may form part of the motion compensation unit 316), an affine unit, a linear model (LM) unit, and the like. In other examples, the video decoder 300 may include more, fewer, or different functional components.

[0105]

[0116] The CPB memory 320 may store video data, such as an encoded video bitstream, to be decoded by the components of the video decoder 300. The video data stored in the CPB memory 320 may be obtained, for example, from a computer-readable medium 110 (Figure 1). The CPB memory 320 may include a CPB that stores encoded video data (e.g., syntax elements) from the encoded video bitstream. The CPB memory 320 may also store video data other than syntax elements of an encoded picture, such as temporary data representing outputs from various units of the video decoder 300. The DPB 314 generally stores decoded pictures that can be output and / or used as reference video data when the video decoder 300 decodes subsequent data or pictures from the encoded video bitstream. The CPB memory 320 and DPB 314 may be formed by any of various memory devices, such as DRAM, MRAM, RRAM, or other types of memory devices, including SDRAM. The CPB memory 320 and DPB 314 may be provided by the same memory device or separate memory devices. In various examples, the CPB memory 320 may be on-chip along with the other components of the video decoder 300, or it may be off-chip relative to those components.

[0106]

[0117] As an addition or alternative, in some examples, the video decoder 300 may retrieve encoded video data from memory 120 (Figure 1). That is, memory 120 may store data together with CPB memory 320 as described above. Similarly, memory 120 may store instructions to be executed by the video decoder 300 when some or all of the functions of the video decoder 300 are implemented in software to be executed by the processing circuit of the video decoder 300.

[0107]

[0118] The various units shown in Figure 4 are illustrated to help understand the operations performed by the video decoder 300. The units can be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Similar to Figure 3, fixed-function circuits refer to circuits that provide a specific function and are pre-configured with respect to the operations they can perform. Programmable circuits refer to circuits that can be programmed to perform various tasks and to provide flexible functionality in the operations they can perform. For example, a programmable circuit may execute software or firmware that operates the programmable circuit in a manner defined by software or firmware instructions. Fixed-function circuits may execute software instructions (e.g., to receive or output parameters), but the type of operation performed by a fixed-function circuit is generally immutable. In some examples, one or more of the units may be separate circuit blocks (fixed-function or programmable), and in some examples, one or more of the units may be integrated circuits.

[0108]

[0119] The video decoder 300 may include a programmable core formed from an ALU, EFU, digital circuitry, analog circuitry, and / or programmable circuitry. In an example where the operation of the video decoder 300 is performed by software running on the programmable circuitry, on-chip or off-chip memory may store software instructions (e.g., object code) that the video decoder 300 receives and executes.

[0109]

[0120] The entropy decoding unit 302 can receive encoded video data from the CPB and entropy decode the video data to reconstruct the syntax elements. The prediction processing unit 304, the inverse quantization unit 306, the inverse transformation processing unit 308, the reconstruction unit 310, and the filter unit 312 can generate the decoded video data based on the syntax elements extracted from the bitstream.

[0110]

[0121] Generally, the video decoder 300 reconstructs the picture block by block. The video decoder 300 can perform the reconstruction operation individually for each block (where the block currently being reconstructed, i.e., decoded, is sometimes called the "current block").

[0111]

[0122] The entropy decoding unit 302 can entropy decode transformation information such as syntax elements that define the quantized transformation coefficients of a quantized transformation coefficient block, as well as quantization parameters (QP) and / or (one or more) transformation mode indications. The inverse quantization unit 306 may use the QP associated with the quantized transformation coefficient block to determine the degree of quantization and, similarly, the degree of inverse quantization that the inverse quantization unit 306 should apply. The inverse quantization unit 306 may perform, for example, a bitwise left shift operation to inverse quantize the quantized transformation coefficients. The inverse quantization unit 306 may thereby form a transformation coefficient block containing the transformation coefficients.

[0112]

[0123] After the inverse quantization unit 306 has formed a transformation coefficient block, the inverse transformation processing unit 308 may apply one or more inverse transformations to the transformation coefficient block to generate a residual block associated with the current block. For example, the inverse transformation processing unit 308 may apply an inverse DCT, an inverse integer transformation, an inverse Carunenlebe transform (KLT), an inverse rotation transform, an inverse direction transform, or another inverse transformation to the transformation coefficient block.

[0113]

[0124] Furthermore, the prediction processing unit 304 generates prediction blocks according to prediction information syntax elements entropy-decoded by the entropy decoding unit 302. For example, if the prediction information syntax elements indicate that the current block is interpredicted, the motion compensation unit 316 may generate a prediction block. In this case, the prediction information syntax elements may indicate a reference picture in the DPB 314 from which the reference block should be extracted, as well as a motion vector identifying the location of the reference block in the reference picture relative to the location of the current block in the current picture. The motion compensation unit 316 may generally carry out the interprediction process in a manner substantially similar to the manner described with respect to the motion compensation unit 224 (Figure 3). In some examples, the motion compensation unit 316 may include an AMVP unit 317 that can determine at least one of time-based or history-based candidates. The AMVP unit 317 may also determine at least one non-adjacent candidate from at least one non-adjacent candidate unit, where at least one non-adjacent candidate unit is not adjacent to the current PU. The AMVP unit 317 may determine an AMVP candidate list that includes at least one time-based candidate or history-based candidate and at least one non-adjacent candidate. The AMVP unit 317 may add at least one non-adjacent candidate to the AMVP candidate list after the time-based candidate and / or before the history-based candidate.

[0114]

[0125] As another example, if a prediction information syntax element indicates that the current block is intra-predicted, the intra-prediction unit 318 may generate a predicted block according to the intra-prediction mode indicated by the prediction information syntax element. In this case as well, the intra-prediction unit 318 may perform the intra-prediction process in a manner that is generally substantially the same as that described with respect to the intra-prediction unit 226 (Figure 3). The intra-prediction unit 318 may retrieve adjacent sample data for the current block from the DPB 314.

[0115]

[0126] The reconstruction unit 310 may reconstruct the current block using the predicted block and the residual block. For example, the reconstruction unit 310 may add the samples of the residual block to the corresponding samples of the predicted block in order to reconstruct the current block.

[0116]

[0127] The filter unit 312 may perform one or more filtering operations on the reconstructed block. For example, the filter unit 312 may perform a deblocking operation to reduce blocking artifacts along the edges of the reconstructed block. The operations of the filter unit 312 are not necessarily performed in all examples.

[0117]

[0128] The video decoder 300 may store the reconstructed blocks in the DPB 314. For example, in an example where the filter unit 312 is not operating, the reconstruction unit 310 may store the reconstructed blocks in the DPB 314. In an example where the filter unit 312 is operating, the filter unit 312 may store the filtered reconstructed blocks in the DPB 314. As described above, the DPB 314 may provide the prediction processing unit 304 with reference information, such as a sample of the current picture for intra-prediction and a previously decoded picture for subsequent motion compensation. Furthermore, the video decoder 300 may output the decoded picture (e.g., decoded video) from the DPB 314 for subsequent presentation on a display device such as the display device 118 in Figure 1.

[0118]

[0129] In this way, the video decoder 300 represents an example of a video decoding device, comprising a memory configured to store video data, and one or more processing units implemented in the circuit and configured to determine at least one of time-based candidates or history-based candidates, determine at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein the at least one non-adjacent candidate unit determines an advanced motion vector predictor (AMVP) candidate list comprising at least one of time-based candidates or history-based candidates and at least one non-adjacent candidate that is not adjacent to the current predictor unit (PU), wherein the at least one non-adjacent candidate codes the current PU based on the AMVP candidate list, which is added to the AMVP candidate list at least one of after the time-based candidate or before the history-based candidate.

[0119]

[0130] The video decoder 300 also represents an example of a video decoding device, which includes a memory configured to store video data, and one or more processing units implemented in the circuit and configured to determine an AMVP candidate list, wherein at least one candidate in the AMVP candidate list is derived from a non-adjacent candidate unit, wherein a non-adjacent candidate unit is not adjacent to the current PU, and to decode the current PU based on the AMVP candidate list.

[0120]

[0131] This section describes the HEVC, JEM, and VVC Draft10 techniques. Video coding standards include ITU-T H.261, ISO / IEC MPEG-1 Visual, ITU-T H.262 or ISO / IEC MPEG-2 Visual, ITU-T H.263, ISO / IEC MPEG-4 Visual, and ITU-T H.264 (also known as ISO / IEC MPEG-4 AVC), which includes its Scalable Video Coding (SVC) and Multiview Video Coding (MVC) extensions.

[0121]

[0132] Furthermore, the video coding standard HEVC or ITU-T H.265, including its range extensions, multiview extensions (MV-HEVC) and scalable extensions (SHVC), was developed by the Joint Collaboration Team on Video Coding (JCT-VC) and the Joint Collaboration Team on 3D Video Coding Extensions Development (JCT-3V) of the ITU-T Video Coding Expert Group (VCEG) and the ISO / IEC Motion Picture Expert Group (MPEG).

[0122]

[0133] The ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC1 / SC29 / WG11) studied the potential need for standardization of future video coding technologies with compression capabilities significantly exceeding those of the HEVC standard (including its current and short-term extensions for screen content coding and high dynamic range coding). The group collaborated on this exploration activity in a joint collaborative work known as the Joint Video Exploration Team (JVET) to evaluate the compression technology designs proposed by their experts in this area. A reference software version, namely VVC Test Model 10 (VTM10.0), can be downloaded from https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM.

[0123]

[0134] The General-Purpose Video Coding (VVC) draft specification is sometimes referred to as JVET-T2001 or VVC Draft10. The algorithm description for General-Purpose Video Coding and Test Model 10 (VTM10.0) is sometimes referred to as JVET-T2002.

[0124]

[0135] Next, we will discuss the structure of CUs and motion vector prediction in VVC Draft 10. In VVC Draft 10, the largest coding unit in a slice is called a CTB or CTU. A CTB contains a quad tree, binary tree, or terminal tree, and its nodes are CUs.

[0125]

[0136] The size of the CTB can range from 4x4 to 128x128 in the VVC main profile. The CU can range from the same size as the CTB to a smaller size of about 4x4. Each CU can be coded using one mode, namely intermode or intramode. When a CU is interconnected, it can be further divided into two, three, or four PUs, or it can be just one PU if no further division is applied. For example, a video encoder 200 or a video decoder 300 can divide a CU into PUs. When two PUs are present in one CU, they are equally divided into rectangles of half the size. When three PUs are present in one CU, three rectangles of 1 / 4, 1 / 2, and 1 / 4 the size of the CU are used.

[0126]

[0137] When a CU is intercoded, each PU has a set of motion information derived using its own inter-prediction mode.

[0127]

[0138] Next, we will discuss motion vector prediction in VVC Draft 10. In VVC Draft 10, there are two interpretation modes for the PU: merge mode (skip mode is considered a special case of merge mode) and AMVP mode. In AMVP mode, an MV candidate list is maintained for multiple motion vector predictors. The reference index for the current PU in both MV and AMVP modes is generated by taking one candidate from the MV candidate list.

[0128]

[0139] In VVC Draft 10, the MV candidate list contains up to two candidates for AMVP mode. Under AMVP mode, for each potential prediction direction from either List 0 or List 1, the AMVP candidate contains only motion vectors, so the reference index is explicitly signaled along with the MV predictor (MVP) index to the motion vector predictor list. For example, video encoder 200 may explicitly signal the reference index. In AMVP mode, the predicted motion vectors may be further refined.

[0129]

[0140] Candidates for AMVP mode may be derived or determined from spatially adjacent blocks, temporally adjacent blocks, or a history-based motion information table. For example, a video encoder 200 or video decoder 300 may derive or determine candidates for AMVP mode.

[0130]

[0141] Next, we will discuss spatially adjacent motion vector candidates in VVC Draft 10. Figure 5 is a conceptual diagram showing exemplary spatially adjacent motion vector candidates for the AMVP mode. Spatially adjacent MV candidates are derived or determined from the adjacent blocks shown in Figure 5.

[0131]

[0142] In AMVP mode, adjacent blocks are divided into two groups, as shown in Figure 5: the left group consisting of blocks 0 and 1, and the upper group consisting of blocks 2, 3, and 4. For each group, potential candidates in neighboring blocks that reference the same reference picture as the reference picture indicated by the signaled reference index have the highest priority to be selected to form the final candidate for the group. Not all neighboring blocks may contain motion vectors pointing to the same reference picture. Therefore, if no such candidate can be found, the first available candidate will be scaled to form the final candidate. Thus, the time distance difference from the reference picture can be compensated.

[0132]

[0143] Next, we will discuss time-motion vector prediction in VVC. If enabled and available, time-motion vector predictor (TMVP) candidates are added to the MV candidate list after spatial motion vector candidates. For example, a video encoder 200 or video decoder 300 may add TMVPs to the MV candidate list after spatial motion vector candidates.

[0133]

[0144] Figures 6A and 6B are conceptual diagrams showing exemplary time motion vector predictor candidates and motion vector scaling, respectively. The primary block location for TMVP candidate derivation is the lower right block outside the collated PU, shown in Figure 6A as block "T" 600, to compensate for biases to the blocks above and to the left used to generate spatial adjacency candidates. However, if that block is located outside the current CTB row (shown as block 602) or motion information is unavailable, the block is replaced with the central block 604 of PU0 606.

[0134]

[0145] The motion vector for the TMVP candidate is derived or determined from the collated PU of the collated picture shown at the slice level. The motion vector for the collated PU is called the collated motion vector.

[0135]

[0146] Similar to the time-direct mode in AVC, in order to derive or determine the TMVP candidate motion vector, the collated motion vector 610 needs to be scaled to compensate for the time-distance difference, as shown in Figure 6B. For example, the video decoder 300 may scale the collated motion vector 610 to compensate for the time-distance difference.

[0136]

[0147] Next, we will discuss history-based motion vector prediction (HMVP) in VVC Draft 10. An HMVP candidate is defined as motion information of a previously coded block. A history table with multiple HMVP candidates may be maintained during the encoding / decoding process. For example, a video encoder 200 or video decoder 300 may maintain a table with multiple HMVP candidates. Whenever there is an intercoded non-affine block, the relevant motion information may be added as a new HMVP candidate to the last entry in the history table.

[0137]

[0148] Figure 7 is a conceptual diagram showing an exemplary decoding flowchart for history-based motion vector prediction. For example, video decoder 300 may load a history table with HMVP candidates (700). Video decoder 300 may decode a block using one or more of the HMVP candidates (702). Video decoder 300 may then update the table with the decoded motion information (704). For example, video decoder 300 may add motion information for intercoded non-affine blocks to the history table. For example, when the HMVP candidate table is completely occupied, video decoder 300 may apply a FIFO (First-In, First-Out) rule to remove the oldest candidate in the history table and add the most recently decoded interCU motion information to the history table as the last entry in the history table. The motion vectors of the last four HMVP candidates in the table are inserted into the AMVP candidate list after the TMVP candidates. Only HMVP candidates with the same reference picture as the AMVP target reference picture are used to make up the AMVP candidate list. Pruning may be applied to the HMVP candidates. For example, the video decoder 300 can prune HMVP candidates.

[0138]

[0149] Next, we will describe other aspects of motion prediction in HEVC. Several aspects of AMVP mode are described as follows:

[0139]

[0150] Motion Vector Scaling: The value of a motion vector can be proportional to the distance between pictures during presentation / display time. A motion vector relates two pictures, namely a reference picture and a picture containing the motion vector (i.e., a containing picture). When one motion vector is used to predict another, the distance between the containing picture and the reference picture is calculated based on the picture order count (POC) value.

[0140]

[0151] For the motion vectors that will be predicted, in some examples, both the associated containing picture and the reference picture of the motion vector may be different from those for the predictor's motion vector. Therefore, the video encoder 200 or video decoder 300 may calculate a new distance (based on POC). The video encoder 200 or video decoder 300 may scale the motion vectors based on these two POC distances. For spatially adjacent candidates, the containing pictures for the two motion vectors are the same, while the reference pictures may be different. In the VVC draft 10, motion vector scaling is applied to temporally adjacent candidates when constructing the AMVP list.

[0141]

[0152] Pseudo-motion vector candidate generation: If the motion vector candidate list is incomplete, the video encoder 200 or video decoder 300 may generate pseudo-motion vector candidates until the AMVP list contains all candidates (e.g., the list is filled), and may insert a pseudo-motion vector candidate at the end of the AMVP list. In AMVP mode, candidate 0 is used to fill any remaining slots in the AMVP list. For example, if there are remaining slots in the AMVP list, the video encoder 200 or video decoder 300 may use candidate 0 to fill the remaining slots in the AMVP list.

[0142]

[0153] Pruning process for candidate insertion: Candidates from different blocks may coincidentally be the same, which reduces the efficiency of the AMVP candidate list. Therefore, the video encoder 200 or video decoder 300 may apply a pruning process to address this problem. During the pruning process, the video encoder 200 or video decoder 300 may compare one candidate with other candidates in the current candidate list (e.g., the AMVP candidate list) to some extent to avoid inserting the same candidate into the current candidate list.

[0143]

[0154] Next, we will discuss template matching (TM) prediction. TM prediction is a special merge mode based on the frame rate upconversion (FRUC) technique. In this mode, block motion information is not signaled but is derived or determined by the decoder. For example, video encoder 200 may not signal motion information with TM prediction, while video decoder 300 may derive or determine the motion information. TM prediction can be applied to both AMVP mode and normal merge mode. In AMVP mode, the selection of motion vector predictor (MVP) candidates may be determined based on template matching such that the template with the smallest difference between the current block's template and the reference block's template is selected. In normal merge mode, video encoder 200 may signal a TM mode flag to indicate the use of TM mode, and video decoder 300 may apply TM mode to merge candidates indicated by the merge index for MV improvement.

[0144]

[0155] Figure 8 is a conceptual diagram showing a TM performed on a search area around the initial motion vector. As shown in Figure 8, the TM technique may be used to derive or determine the motion information of the current PU. Deriving motion information may involve finding the closest match between template 800 in the current picture 802 (in adjacent blocks above and / or to the left of the current PU) and block 804 in reference picture 806 (for example, the same size as template 800). For AMVP candidates selected based on the initial matching error, the candidate MVP is refined by template matching. For merge candidates indicated by a signaled merge index, the merged MV of the candidate corresponding to reference picture list 0 (L0) and reference picture list 1 (L1) is refined independently by TM, and then the less accurate MV is further refined again using a more accurate MV as the previous reference. For example, video decoder 300 receives and parses a signaled merge index and applies template matching to the merged MV to refine the MV.

[0145]

[0156] Cost function: When the motion vector points to a fractional sample position, the video decoder 300 may use motion compensation interpolation. To reduce complexity, bilinear interpolation is used instead of the usual 8-tap discrete cosine transform-interpolation filter (DCT-IF) interpolation for both generating templates for the TM and the reference picture. The matching cost C for template matching can be calculated as follows:

[0146]

number

[0147] Here, w is a weighting coefficient that is empirically set to 4, and MV and MV sThese represent the MV currently being tested and the initial MV (for example, the MVP candidate in AMVP mode or the merged motion vector in merge mode), respectively. The absolute difference sum (SAD) can be used as the matching cost for template matching.

[0148]

[0157] When motion manipulation (TM) is used, the motion can be improved by using only lumen samples. The derived or determined motion can be used for both lumen and chromen for motion compensation (MC) interpretation. After the motion mass (MV) is determined, the final MC is performed using an 8-tap interpolation filter for the lumen and a 4-tap interpolation filter for the chromen. For example, video decoder 300 may improve the motion using only lumen samples.

[0149]

[0158] Search Method: MV refinement can be a pattern-based MV search using TM cost as a criterion. Two search patterns may be supported for MV refinement: diamond pattern search and cross pattern search. For example, video decoder 300 may use either diamond pattern search or cross pattern search for MV refinement. The MV may be searched directly with 1 / 4 lumasample motion vector difference (MVD) accuracy with a diamond pattern, then searched with 1 / 4 lumasample MVD accuracy with a cross pattern, and then a 1 / 8 lumasample MVD refinement with a cross pattern may follow after the 1 / 4 lumasample MVD accuracy cross pattern search. The search range for MV refinement may be set to equal (-8, +8) lumasamples around the initial MV.

[0150]

[0159] The video encoder 200 or video decoder 300 may apply TM techniques to refine AMVP candidates after the AMVP candidate list has been constructed. In this case, the AMVP candidate with the smallest TM cost will be used as a starting point for TM-based refinement, and the final refined MV will be used as the final MVP, so the video encoder 200 may skip signaling of the AMVP index. However, AMVP candidates are derived or determined from history-based motion information obtained from spatial neighbor blocks, collated time blocks or lower-right time blocks, or recently decoded PUs. Therefore, it is quite possible that all these potential AMVP candidates contain exactly the same motion information, which is detrimental to the diversity of the AMVP candidate list. Furthermore, the AMVP candidate list contains at most two candidates. Therefore, the possibility of containing diverse AMVP candidates is further reduced.

[0151]

[0160] Even when TM-based improvements are applied to AMVP candidates, the signaling overhead of the AMVP index may be preserved, and it may not always be the best choice for the video encoder 200 or video decoder 300 to use the AMVP candidate with the lowest TM cost as a starting point for MV improvements. For example, using the AMVP candidate with the lowest TM cost may result in a minimal TM cost, but this may not necessarily be a good overall choice. In addition, the TM cost does not always accurately represent the difference between the current block and the reference block.

[0152]

[0161] Next, non-adjacent AMVP candidates are described. This disclosure describes a category of motion information that may be added to the AMVP candidate list, which is one or more non-adjacent AMVP candidates, e.g., motion vector candidates derived or determined from blocks or units that are not adjacent to the current PU. All existing categories of candidates for AMVP are located near the current PU (e.g., derived or determined from blocks near the current PU) and are very likely to give similar motion information. The introduction of non-adjacent AMVP candidates may improve the diversity of the AMVP candidate list. This introduction of non-adjacent AMVP candidates may allow a video encoder 200 or video decoder 300 applying a TM-based improvement process to have a higher probability of finding a better final motion vector predictor. In some examples, non-adjacent AMVP candidates may be used in either intra-mode (e.g., in which case they may not be spatially adjacent to the current PU) or inter-mode (e.g., in which case they may not be spatially adjacent to a collated PU in another picture).

[0153]

[0162] Figure 9 is a conceptual diagram showing exemplary non-adjacent AMVP candidate locations, e.g., the relative locations of blocks or units with respect to the current PU from which non-adjacent AMVP candidates are derived or determined. Exemplary patterns and locations of non-adjacent candidate units are shown in Figure 9. In the example in Figure 9, each small square in grid 900 represents one 4x4 unit or block with the width of four lumens and the height of four lumens. The current PU is labeled PU. The upper left lumens of the current PU is shown in a box filled with the pattern within the current PU. The locations of 18 potential non-adjacent candidate units are shown as numbered 1 through 18.For example, the video encoder 200 or video decoder 300 is located two units (blocks) below the current PU (marked as 1) to the left, one unit (block) above the current PU (marked as 2), two units (blocks) above the current PU (marked as 3), three units (blocks) below the current PU (marked as 4), two units (blocks) above the current PU (marked as 5), three units (blocks) above the current PU (marked as 6), three units (blocks) to the left of the current PU (marked as 7), three units (blocks) above the current PU (marked as 8), four units (blocks) below the current PU (marked as 9), and one unit (block) below the current PU (marked as 10). At least one non-adjacent candidate unit can be selected from a group of potential non-adjacent candidate units consisting of units / blocks located 3 units (blocks) to the right of 4 units (blocks) above the current PU (marked as 11), 4 units (blocks) to the left of the current PU (marked as 12), 4 units (blocks) above the current PU (marked as 13), 5 units (blocks) to the left of the current PU (marked as 14), 4 units (blocks) above the current PU (blocks) (marked as 15), 5 units (blocks) above the current PU (blocks) (marked as 16), 5 units (blocks) to the left of the current PU (blocks) (marked as 17), and 5 units (blocks) above the current PU (blocks) (marked as 18). The simulation shows that using units / blocks in these locations as AMVP candidate units can improve coding efficiency for some classes of video with a slight increase in complexity. The simulation results are as follows:

[0154] [Table 1]

[0155] Here, ECM is the Extended Compression Model, Y is the chroma, U is the chroma of the blue projection, V is the chroma of the red projection, EncT is the encoding time, and DecT is the decoding time. Negative percentages for Y, U, and V indicate improved efficiency. Class A1 and A2 videos have a resolution of 3840×2160 (width and height in pixels), Class B videos have a resolution of 1920×1080, Class C videos have a resolution of 832×480, and Class D videos have a resolution of 416×240. Class D videos are not included in the overall results described above. The reason why Class D videos do not perform as well in the simulation as the other classes is that the resolution is low and the image boundaries are relatively close to the current PU's position. Therefore, the number of available non-contiguous units that can be used to derive motion information is relatively small in higher resolution videos. Furthermore, the image content in Class D videos changes relatively quickly due to the low resolution. Therefore, non-adjacent units may be irrelevant to the current PU's image content, and even if motion information is available for non-adjacent units, it may be unlikely to be useful. The performance of the techniques of this disclosure was not simulated using Class E video.

[0156]

[0163] The video encoder 200 or video decoder 300 may derive or determine the location of at least one non-adjacent PU / block based on the size of the current PU. For example, the video encoder 200 or video decoder 300 may determine the size of the current PU and determine at least one non-adjacent candidate based at least partially on the size of the current PU. For example, a distance level may be used to control how far away at least one non-adjacent PU / block is located relative to the current PU. A level index may also be used as a group index for each of the non-adjacent candidate groups. A total of four groups may be defined as follows: Group 1: {1,2,3} (for example, locations numbered 1 through 3) Group 2: {4, 5, 6, 7, 8} (for example, locations numbered 4 through 8) Group 3: {9,10,11,12,13} (for example, locations numbered 9 through 13) Group 4: {14, 15, 16, 17, 18} (for example, locations numbered 14 through 18) For example, potential non-adjacent candidate units include: 1) a first group of potential non-adjacent candidate units located 1 unit (block) below and 2 units (blocks) to the left of the current PU, 2 units (blocks) above and 1 unit (block) to the right of the current PU, and 2 units (blocks) above and to the left of the current PU; 2) a second group of potential non-adjacent candidate units located 2 units (blocks) below and 3 units (blocks) to the left of the current PU, 3 units (blocks) above and 2 units (blocks) to the right of the current PU, 3 units (blocks) above and 3 units (blocks) above and to the left of the current PU; and 3) 3 units ( The blocks can be divided into four groups: a third group of potential non-adjacent candidate units located 4 units (blocks) below and to the left of the current PU, 3 units (blocks) above and to the right of the current PU's 4 units (blocks), above the current PU's 4 units (blocks), to the left of the current PU's 4 units (blocks), and 4 units (blocks) above and to the left of the current PU's 4 units (blocks); and a fourth group of potential non-adjacent candidate units located 5 units (blocks) below and to the left of the current PU's 4 units (blocks), 4 units (blocks) above and to the right of the current PU's 5 units (blocks), above the current PU's 5 units (blocks), to the left of the current PU's 5 units (blocks), and 5 units (blocks) above and to the left of the current PU's 5 units (blocks).

[0157]

[0164] In some examples, an index map array can be defined as follows: idxMap[4][5]={{0,1,2},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}. Two variables can be used to determine the final index, and idxMap[level_index][position_index] can be linked to a formula for calculating the location of at least one non-adjacent candidate. The variable level_index can be in the range of 0 to 3. For example, level_index of 0 may indicate group 1, level_index of 1 may indicate group 2, level_index of 2 may indicate group 3, and level_index of 3 may indicate group 4. The variable position_index can be in the range of 0 to 2 when level_index is equal to 0 (e.g., indicating a location numbered 1, 2, or 3 respectively), and can be in the range of 0 to 4 when level_index is equal to 1, 2, or 3 (e.g., indicating a specific location within group 2, group 3, or group 4). For example, a video encoder 200 or video decoder 300 may signal or parse a level index indicating a group of several groups, and a position index indicating one of the potential non-adjacent candidate units belonging to the group indicated by the level index.

[0158]

[0165] In some examples, for each idxMap[level_index][position_index], the following formulas 0 through 4 may be used to derive or determine the location of at least one non-adjacent candidate unit. 0:offsetX=-iNADistanceHor-1;offsetY=height+iNADistanceVer-1; 1:offsetX=width+iNADistanceHor-1;offsetY=-iNADistanceVer-1; 2:offsetX=width>>1;offsetY=-iNADistanceVer-1; 3:offsetX=-iNADistanceHor-1;offsetY=height>>1; 4:offsetX=-iNADistanceHor-1;offsetY=-iNADistanceVer-1; Here, width is the width of the current PU in some rumor samples, height is the height of the current PU in some rumor samples, and iNA represents the i-th non-adjacent unit / block. iNADistanceHor = width * (level_index + 1) and iNADistanceVer = height * (level_index + 1), where ">>" refers to a bitwise right shift, where n>>k is equivalent to n / (2^k). For example, a video encoder 200 or a video decoder 300 uses a first syntax element indicating the level index (e.g., level_index) and a second syntax element, for example, idxMap[level_index][position_index], to determine the horizontal and vertical offsets offsetX = -iNADistanceHor-1 and offsetY = height + iNADistanceVer-1, offsetX = width + iNADistanceHor-1 and offsetY = -iNADistanceVer-1, offsetX = width>>1 and offsetY = -iNADistanceVer-1, offsetX = width>>1 and offsetY = -iNADistanceVer-1, offset You can choose one of the following: tX = -iNADistanceHor - 1 and offsetY = height >> 1, or offsetX = -iNADistanceHor - 1 and offsetY = -iNADistanceVer - 1, where offsetX is the horizontal offset, offsetY is the vertical offset, width is the width of the current PU in some lumane samples, height is the height of the current PU in some lumane samples, iNA represents the i-th non-adjacent candidate unit / block, iNADistanceHor = width * (first syntax element + 1), and iNADistanceVer = height * (first syntax element + 1). The signs "+" for X and Y in the formulas correspond to the rightward and downward directions, respectively, in Figure 9. Formula 0 may correspond to locations numbered 1, 4, 9, and 14. Formula 1 may correspond to locations numbered 2, 5, 10, and 15. Formula 2 may correspond to locations numbered 6, 11, and 16. Formula 3 may correspond to locations numbered 7, 12, and 17.Official 4 may correspond to locations numbered 3, 8, 13, and 18.

[0159]

[0166] The video encoder 200 or video decoder 300 may calculate the location of the final non-adjacent candidate unit by adding a horizontal offset (offsetX) and a vertical offset (offsetY) to the lumens sample coordinates of the top-left corner of the current PU. The motion information from the 4x4 unit / block containing the derived or determined non-adjacent sample position may be used as a candidate to be added to the AMVP candidate list if the 4x4 unit / block is encoded.

[0160]

[0167] From the derivation process described above, we can see that a total of 18 non-adjacent positions can be used as potential candidate units for non-adjacent candidates to be derived or determined in the AMVP candidate list. However, if the size of the AMVP candidate list is still kept at 2, the likelihood of non-adjacent candidates being added to the AMVP candidate list may remain relatively low. To retain conventional space, time, and historical candidates in the AMVP candidate list while still providing the possibility of non-adjacent candidates being considered, the size of the AMVP candidate list can be increased. In one example, the size of the AMVP candidate list is increased to 5.

[0161]

[0168] Non-adjacent candidates may also be added with different priorities. In one example, non-adjacent candidates are added after time candidates (for example, with a lower priority) and / or before history-based candidates (for example, with a higher priority). For example, the AMVP candidate list may contain at least one of either time candidates or history-based candidates and at least one non-adjacent candidate. The video encoder 200 or video decoder 300 may add at least one non-adjacent candidate to the AMVP candidate list after time candidates or before history-based candidates. In another example, at least one non-adjacent candidate is added after history-based candidates and / or before zero candidates. In a third example, at least one non-adjacent candidate is added after spatial candidates and / or before time candidates.

[0162]

[0169] In one example, the priority of at least one non-adjacent candidate is treated similarly to that of a spatial candidate, where a candidate unit located to the left of the current PU has a higher priority than a candidate unit located above the current PU. One example of the AMVP candidate list construction process is as follows: 1) spatially adjacent candidates to the left, 2) spatially non-adjacent candidates to the left, 3) spatially adjacent candidates above, 4) spatially non-adjacent candidates above, 5) time candidates, 6) history candidates, 7) 0MV candidates. In some examples, only one candidate may be added to the AMVP candidate list at each of the seven steps of the technique described above. For example, a video encoder 200 or a video decoder 300 may add up to one candidate to the AMVP candidate list at each of the seven steps.

[0163]

[0170] Next, we will discuss signaling of the AMVP index using TM AMVP refinement. In VVC Draft 10, the AMVP index is not signaled when TM is used to refine the AMVP candidate list. The AMVP candidate with the lowest TM cost may be used as a starting point for TM AMVP refinement, and the final refined MV may be used as the motion vector predictor. However, as mentioned above, such a design may reduce the diversity in the AMVP candidate list used for the selection of the final AMVP candidate. This may result in less accurate AMVP candidates when the template does not have a strong correlation with the data samples of the current PU. To retain the advantages of refining AMVP candidates with TM while increasing the potential diversity of the AMVP candidate list, a hybrid approach may be introduced into the AMVP candidate list construction process. Compared to the VVC Draft 10 design, instead of always using the TM-refined AMVP candidate as the MVP after the TM AMVP refinement process is complete, the video encoder 200 or video decoder 300 may not exclude other unrefined AMVP candidates from the AMVP candidate list. In this example, the video encoder 200 still has the possibility of selecting an unimproved AMVP candidate as the final motion vector predictor. To achieve this, the video encoder 200 may signal the AMVP index. For example, when an AMVP index equal to 0 is signaled, an AMVP candidate improved in TM may be used as the MVP, and when an AMVP index equal to 1 is signaled (for example, for an AMVP candidate list of size 2), an unimproved AMVP candidate may be used as the MVP instead. For example, the video encoder 200 may signal an AMVP index of 0 or 1, and the video decoder 300 may parse the AMVP index to determine whether to use an improved AMVP candidate as the MVP or (for example, for an AMVP candidate list of size 2) an unimproved AMVP candidate.

[0164]

[0171] Next, we will discuss determining a second AMVP candidate based on the MV distance. As in the previous paragraph, when the MV cand0Mv(hor,ver) of the TM-improved AMVP candidate is derived or determined by applying the TM improvement to the AMVP candidate with the lowest TM cost, the remaining AMVP candidates may be further evaluated to determine the MV cand1Mv(hor,ver) of the second AMVP candidate. In this example, the video encoder 200 or video decoder 300 may determine the MV of the second AMVP candidate based on the MV distance to cand0Mv. The determined MV cand1Mv of the second AMVP candidate may have a longer MV distance to cand0Mv compared to the other AMVP candidates, and therefore the current block may have a completely different predictor compared to the TM-improved AMVP candidate cand0Mv.

[0165]

[0172] Assuming an AMVP candidate MV candXMv(hor,ver), the MV distance to cand0Mv can be defined as candXMvDistance=((|candXMv(hor)-cand0Mv(hor)|+|candXMv(ver)-cand0Mv(ver)|)>>N), where N is a predetermined integer value for deriving or determining the scaled MV distance.

[0166]

[0173] In one example, the second AMVP candidate MV is determined to be candXMv, which has the greatest MV distance to cand0Mv.

[0167]

[0174] In another example, the second AMVP candidate was: smallest TM Cost Ratio of MV distance It is determined that candXMv has the following characteristics. Assuming an AMVP candidate MV candXMv(hor,ver) has a TM cost tmCostX, the TM cost-MV distance ratio can be defined as tmCostX / candXMvDistance.

[0168]

[0175] Figure 10 is a flowchart illustrating exemplary AMVP candidate techniques according to one or more aspects of the present disclosure. A video encoder 200 or video decoder 300 may determine at least one of time-based candidates or history-based candidates (1000). For example, the video encoder 200 or video decoder 300 may determine time-based candidates, history-based candidates, or both time-based and history-based candidates. Such candidates may be candidates for motion vector predictors.

[0169]

[0176] The video encoder 200 or video decoder 300 may determine at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein at least one non-adjacent candidate unit is not adjacent to the current prediction unit (PU) (1002). For example, the video encoder 200 or video decoder 300 may determine at least one non-adjacent candidate from at least one non-adjacent candidate unit located at one of the locations indicated by numbers 1 to 18 in Figure 9. At least one non-adjacent candidate may be a candidate for a motion vector predictor.

[0170]

[0177] The video encoder 200 or video decoder 300 may determine an AMVP candidate list comprising at least one time-based candidate or history-based candidate and at least one non-adjacent candidate, wherein at least one non-adjacent candidate is added to the AMVP candidate list at least one of either a time-based candidate or a history-based candidate (1004). For example, the video encoder 200 or video decoder 300 may construct an AMVP candidate list that includes at least one non-adjacent candidate after a time-based candidate, before a history-based candidate, or after a time-based candidate and before a history-based candidate.

[0171]

[0178] The video encoder 200 or video decoder 300 may code the current PU based on the AMVP candidate list (1006). For example, the video encoder 200 or video decoder 300 may select a motion vector predictor from the AMVP candidate list and use the selected motion vector predictor to code the current PU.

[0172]

[0179] In some examples, determining at least one non-adjacent candidate involves selecting at least one non-adjacent candidate unit from a group of potential non-adjacent candidate units consisting of units located 1 unit below and 2 units to the left of the current PU, 2 units above and 1 unit to the right of the current PU, 2 units above and 2 units to the left of the current PU, 2 units below and 3 units to the left of the current PU, 3 units above and 2 units to the right of the current PU, 3 units above the current PU, 3 units to the left of the current PU, 3 units above and 3 units to the left of the current PU, 4 units above the current PU and 3 units to the right of the current PU, 4 units above the current PU and 4 units to the left of the current PU, 4 units below and 5 units to the left of the current PU, 5 units above and 4 units to the right of the current PU, 5 units above the current PU, 5 units to the left of the current PU, and 5 units above and 5 units to the left of the current PU.

[0173]

[0180] In some examples, multiple potential non-adjacent candidate units are divided into multiple groups. In some examples, the video encoder 200 or video decoder 300 signals or parses a level index indicating one of the multiple groups, and signals or parses a position index indicating one of the potential non-adjacent candidate units belonging to the group indicated by the level index, wherein at least one non-adjacent candidate unit comprises one of the potential non-adjacent candidate units.

[0174]

[0181] In some examples, multiple groups include: 1) a first group of potential non-adjacent candidate units located 1 unit below and 2 units to the left of the current PU, 2 units above and 1 unit to the right of the current PU, and 2 units above and 2 units to the left of the current PU; 2) a second group of potential non-adjacent candidate units located 2 units below and 3 units to the left of the current PU, 3 units above and 2 units to the right of the current PU, 3 units above the current PU, 3 units to the left of the current PU, and 3 units above and 3 units to the left of the current PU; 3) a third group of potential non-adjacent candidate units located 3 units below and 4 units to the left of the current PU, 4 units above and 3 units to the right of the current PU, 4 units above the current PU, 4 units to the left of the current PU, and 4 units above and 4 units to the left of the current PU; and 4) a fourth group of potential non-adjacent candidate units located 4 units below and 5 units to the left of the current PU, 5 units above and 4 units to the right of the current PU, 5 units above the current PU, 5 units to the left of the current PU, and 5 units above and 5 units to the left of the current PU.

[0175]

[0182] In some examples, the video encoder 200 or video decoder 300 may determine the size of the current PU, where determining at least one non-adjacent candidate is at least partially based on the size of the current PU. In some examples, determining at least one non-adjacent candidate comprises determining a horizontal offset and a vertical offset of the current PU relative to the top-left lumen sample. In some examples, determining at least one non-adjacent candidate further comprises adding the horizontal and vertical offsets to the top-left lumen sample coordinates of the current PU. In some examples, the video encoder 200 or video decoder 300 uses offsetX=-iNADistanceHor-1 and offsetY=height+iNADistanceVer-1, offsetX=width+iNADistanceHor-1 and offsetY=-iNADistanceVer-1, offsetX=width>>1 and offsetY=-iNADistanceVer-1, offsetX=-iNADistance You can choose one of the following: Hor-1 and offsetY=height>>1, or offsetX=-iNADistanceHor-1 and offsetY=-iNADistanceVer-1, where offsetX is the horizontal offset, offsetY is the vertical offset, width is the width of the current PU in some rumor samples, height is the height of the current PU in some rumor samples, iNA represents the i-th non-adjacent candidate unit, iNADistanceHor=width*(first syntax element+1), and iNADistanceVer=height*(first syntax element+1).

[0176]

[0183] In some cases, the number of candidates in the AMVP candidate list is greater than 2. In some cases, the number of candidates in the AMVP candidate list is 5.

[0177]

[0184] In some examples, the video encoder 200 or video decoder 300 may determine the template matching cost for each candidate in the AMVP candidate list and determine the motion vector predictor that is the candidate in the AMVP candidate list having the smallest template matching cost among multiple candidates in the AMVP candidate list.

[0178]

[0185] In some examples, the AMVP candidate list includes AMVP candidates that have been improved with template matching and those that have not. In some examples, the video encoder 200 or video decoder 300 may signal or parse an AMVP index, which indicates whether a motion vector predictor is an AMVP candidate that has been improved with template matching or one that has not. In some examples, the AMVP index is equal to 0, and the video encoder 200 or video decoder 300 may determine that the motion vector predictor is an AMVP candidate that has been improved with template matching. In some examples, the AMVP index is equal to 1, and the video encoder 200 or video decoder 300 may determine that it is an AMVP candidate that has not been improved. In some examples, the video encoder 200 or video decoder 300 may determine the respective template matching cost for each candidate in the AMVP candidate list and use template matching to improve the AMVP candidate that has the minimum template matching cost to create an AMVP candidate that has been improved with template matching. In some cases, an unimproved AMVP candidate has the largest motion vector distance to an AMVP candidate improved by template matching among any potential AMVP candidates. , minimum template matching cost vsMotion vector distance ratio To have. In some examples, coding includes encoding. In some examples, coding includes decoding.

[0179]

[0186] Figure 11 is a flowchart illustrating an exemplary method for encoding a current block using the technique of the present disclosure. The current block may comprise a current CU. While the video encoder 200 (Figures 1 and 3) is described, it should be understood that other devices may be configured to perform a similar method to that of Figure 11.

[0180]

[0187] In this example, the video encoder 200 first predicts the current block (1100). For example, the video encoder 200 may form a predicted block for the current block. When predicting the current block, the video encoder 200 may employ the AMVP technique shown in Figure 10. For example, the AMVP unit 223 (Figure 3) may determine at least one of the time-based candidates or the history-based candidates. The AMVP unit 223 may also determine at least one non-adjacent candidate from at least one non-adjacent candidate unit, where at least one non-adjacent candidate unit is not adjacent to the current PU. The AMVP unit 223 may determine an AMVP candidate list containing at least one of the time-based candidates or the history-based candidates and at least one non-adjacent candidate. The AMVP unit 223 may add at least one non-adjacent candidate to the AMVP candidate list after the time-based candidate units and / or before the history-based candidate units. The video encoder 200 may then calculate the residual block for the current block (1102). To compute the residual block, the video encoder 200 may compute the difference between the original unencoded block and the predicted block for the current block. The video encoder 200 may then transform the residual block and quantize the transformation coefficients of the residual block (1104). Next, the video encoder 200 may scan the quantized transformation coefficients of the residual block (1106). During or following the scan, the video encoder 200 may entropy encode the transformation coefficients (1108). For example, the video encoder 200 may encode the transformation coefficients using CAVLC or CABAC. The video encoder 200 may then output the entropy encoded data of the block (1110).

[0181]

[0188] Figure 12 is a flowchart illustrating an exemplary method for decoding the current block of video data using the technique of the present disclosure. The current block may comprise the current CU. While a video decoder 300 (Figures 1 and 4) is described, it should be understood that other devices may be configured to perform a similar method to that shown in Figure 12.

[0182]

[0189] The video decoder 300 may receive entropy-encoded data about the current block, such as entropy-encoded prediction information and entropy-encoded data about the transformation coefficients of the residual block corresponding to the current block (1200). The video decoder 300 may entropy-decode the entropy-encoded data to determine the prediction information for the current block and to reconstruct the transformation coefficients of the residual block (1202). The video decoder 300 may predict the current block, for example, using an intra-prediction or inter-prediction mode indicated by the prediction information for the current block, in order to compute a prediction block for the current block (1204). For example, the video decoder 300 may use the AMVP technique shown in Figure 10 when predicting the current block. For example, the AMVP unit 317 (Figure 4) may determine at least one of time-based candidates or history-based candidates. The AMVP unit 317 may also determine at least one non-adjacent candidate from at least one non-adjacent candidate unit, where at least one non-adjacent candidate unit is not adjacent to the current PU. The AMVP unit 317 may determine an AMVP candidate list that includes at least one time-based candidate or history-based candidate and at least one non-adjacent candidate. The AMVP unit 317 may add at least one non-adjacent candidate to the AMVP candidate list after the time-based candidate unit and / or before the history-based candidate unit. The video decoder 300 may then reverse-scan the replayed transformation coefficients to create a block of quantized transformation coefficients (1206). The video decoder 300 may then de-quantize the transformation coefficients and apply the inverse transformation to the transformation coefficients to create a residual block (1208). The video decoder 300 may finally decode the current block by combining the predicted block and the residual block (1210).

[0183]

[0190] This disclosure includes the following non-limiting clauses:

[0184]

[0191] Clause 1A. A method for coding video data, comprising determining an advanced motion vector predictor (AMVP) candidate list, wherein at least one candidate in the AMVP candidate list is a non-adjacent candidate, wherein the non-adjacent candidate is not adjacent to the current predictor unit (PU), and coding the current PU based on the AMVP candidate list.

[0185]

[0192] The method of Clause 1A, wherein determining the AMVP candidate list comprises determining the size of the current PU and determining non-adjacent candidates based at least in part on the size of the current PU.

[0186]

[0193] The method according to any one of the provisions of 1A to 2A, wherein determining the AMVP candidate list comprises determining distance levels and determining non-adjacent candidates based at least in part on the distance levels.

[0187]

[0194] Clause 4A. Potential non-adjacent candidates are divided into multiple groups as described in any of Clauses 1A-3A.

[0188]

[0195] Clause 5A. Determining non-adjacent candidates is the method of any of Clauses 2A to 4A, comprising determining the x-offset and the y-offset.

[0189]

[0196] Clause 6A. Determining non-adjacent candidates is the method of Clause 5A, comprising adding an x ​​offset and a y offset to the top-left lumen sample coordinates of the current PU.

[0190]

[0197] Clause 7A. The number of AMVP candidates in the AMVP candidate list is greater than 2, as described in any of Clauses 1A-6A.

[0191]

[0198] Article 8A. The number of AMVP candidates in the AMVP candidate list is 5, as described in Article 7A.

[0192]

[0199] Clause 9A. Non-adjacent candidates are added to the AMVP candidate list after any time candidate and before any history-based candidate, as described in any of Clauses 1A-8A.

[0193]

[0200] Clause 10A. Non-adjacent candidates are added to the AMVP candidate list after any history-based candidate and before any zero candidates, as described in any of Clauses 1A-8A.

[0194]

[0201] Clause 11A. Non-adjacent candidates are added to the AMVP candidate list after any spatial candidate and before any temporal candidate, as described in any of Clauses 1A-8A.

[0195]

[0202] The method of any of the provisions 1A to 8A, wherein determining the AMVP candidate list is further comprising determining the AMVP candidate list in an order based on priority.

[0196]

[0203] Clause 13A. The method according to Clause 12A, wherein the priority is, firstly, a spatially left adjacent candidate; secondly, a spatially left non-adjacent candidate; thirdly, a spatially above adjacent candidate; fourthly, a spatially above non-adjacent candidate; fifthly, a time candidate; sixthly, a history candidate; and seventhly, a 0MV candidate.

[0197]

[0204] Clause 14A. AMVP candidate list is a list of unimproved AMVP candidates as described in any of Clauses 1A-13A.

[0198]

[0205] The method of any of the provisions 1A to 14A, further comprising parsing or signaling an AMVP index.

[0199]

[0206] The method of Clause 15A, further comprising the condition that the AMVP index is equal to 0, wherein the method determines that the motion vector predictor is an AMVP candidate improved by template matching.

[0200]

[0207] The method of the

[0201]

[0208] Clause 18A. The AMVP candidate list comprises AMVP candidates improved by template matching and unimproved AMVP candidates, wherein the unimproved AMVP candidates are located further away from the current PU than the AMVP candidates improved by template matching, as described in any of Clauses 1A to 17A.

[0202]

[0209] Clause 19A. An unimproved AMVP candidate has the maximum movement vector distance to the current PU of a potential AMVP candidate, as described in Clause 18A.

[0203]

[0210] Clause 20A. The method of Clause 18A, wherein an unimproved AMVP candidate has the minimum template matching cost minus the motion vector distance of the potential AMVP candidate.

[0204]

[0211] Clause 21A. The method according to any one of Clauses 1A to 20A, comprising coding and decoding.

[0205]

[0212] Clause 22A: The method of any one of Clauses 1A to 21A, wherein coding comprises encoding.

[0206]

[0213] Clause 23A. A device for coding video data, comprising one or more means for carrying out the method described in any of Clauses 1A to 22A.

[0207]

[0214] The device described in Clause 23A, wherein one or more means comprises one or more processors implemented in the circuit.

[0208]

[0215] Clause 25A. A device as described in any of Clauses 23A to 24A, further comprising memory for storing video data.

[0209]

[0216] Clause 26A: A device as described in any of Clauses 23A to 25A, further comprising a display configured to display decoded video data.

[0210]

[0217] Clause 27A: A device as described in any of Clauses 23A to 26A, comprising one or more of the following: a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

[0211]

[0218] Clause 28A. A device as described in any of Clauses 23A to 27A, wherein the device comprises a video decoder.

[0212]

[0219] Clause 29A: A device as described in any of Clauses 23A to 28A, wherein the device comprises a video encoder.

[0213]

[0220] Clause 30A. A computer-readable storage medium storing instructions, wherein, when an instruction is executed, it causes one or more processors to perform the method described in any of Clauses 1A to 29A.

[0214]

[0221] Clause 31A. A device for encoding video data, comprising means for determining an advanced motion vector predictor (AMVP) candidate list, wherein at least one candidate in the AMVP candidate list is a non-adjacent candidate, wherein the non-adjacent candidate is not adjacent to the current predictor unit (PU), and means for coding the current PU based on the AMVP candidate list.

[0215]

[0222] Clause 1B. A method for coding video data, comprising: determining at least one of time-based candidates or history-based candidates; determining at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein the at least one non-adjacent candidate unit determines an advanced motion vector predictor (AMVP) candidate list comprising at least one of time-based candidates or history-based candidates and at least one non-adjacent candidate that is not adjacent to the current predictor unit (PU); wherein the at least one non-adjacent candidate codes the current PU based on the AMVP candidate list, which is added to the AMVP candidate list at least one of after the time-based candidates or before the history-based candidates.

[0216]

[0223] Clause 2B. The method according to Clause 1B, wherein determining at least one non-adjacent candidate unit comprises selecting at least one non-adjacent candidate unit from a group consisting of potential non-adjacent candidate units located 1 unit below and 2 units to the left of the current PU, 2 units above and 1 unit to the right of the current PU, 2 units above and 2 units to the left of the current PU, 2 units below and 3 units to the left of the current PU, 3 units above and 2 units to the right of the current PU, 3 units above the current PU, 3 units to the left of the current PU, 3 units above and 3 units to the left of the current PU, 4 units above the current PU and 3 units to the right of the current PU, 4 units above the current PU, 4 units to the left of the current PU, 4 units above and 4 units to the left of the current PU, 4 units below and 5 units to the left of the current PU, 5 units above and 4 units to the right of the current PU, 5 units above the current PU, 5 units to the left of the current PU, and 5 units above and 5 units to the left of the current PU.

[0217]

[0224] The method according to Clause 1B or Clause 2B, further comprising: Clause 3B. Multiple potential non-adjacent candidate units are divided into multiple groups, wherein the method signals or parses a level index indicating one of the multiple groups; and signals or parses a position index indicating one of the potential non-adjacent candidate units belonging to the group indicated by the level index, wherein at least one non-adjacent candidate unit comprises one of the potential non-adjacent candidate units.

[0218]

[0225] Clause 4B. The method according to Clause 3B, wherein the groups include a first group of potential non-adjacent candidate units located 1 unit below and 2 units to the left of the current PU, 2 units above and 1 unit to the right of the current PU, and 2 units above and 2 units to the left of the current PU; a second group of potential non-adjacent candidate units located 2 units below and 3 units to the left of the current PU, 3 units above and 2 units to the right of the current PU, 3 units above the current PU, 3 units to the left of the current PU, and 3 units above and 3 units to the left of the current PU; a third group of potential non-adjacent candidate units located 3 units below and 4 units to the left of the current PU, 4 units above and 3 units to the right of the current PU, 4 units above the current PU, 4 units to the left of the current PU, and 4 units above and 4 units to the left of the current PU; and a fourth group of potential non-adjacent candidate units located 4 units below and 5 units to the left of the current PU, 5 units above and 4 units to the right of the current PU, 5 units above the current PU, 5 units to the left of the current PU, and 5 units above and 5 units to the left of the current PU.

[0219] The method of Clauses 1B-4B, further comprising: Clause 5B. Determining the size of the current PU, wherein determining at least one non-adjacent candidate is based at least partially on the size of the current PU.

[0220]

[0226] Clause 6B. The method according to Clauses 1B to 5B, wherein determining at least one non-adjacent candidate comprises determining the horizontal offset and the vertical offset of the current PU relative to the upper-left lumens sample.

[0221]

[0227] Clause 7B. The method of Clause 6B, further comprising determining at least one non-adjacent candidate unit by adding a horizontal offset and a vertical offset to the lumens sample coordinates of the top-left of the current PU.

[0222]

[0228] Clause 8B. According to the first syntax element indicating the level index and the second syntax element indicating the horizontal and vertical offsets, use offsetX=-iNADistanceHor-1 and offsetY=height+iNADistanceVer-1, offsetX=width+iNADistanceHor-1 and offsetY=-iNADistanceVer-1, offsetX=width>>1 and offsetY=-iNADistanceVer-1, offsetX=-iNADistanceHor-1 and offsetY=height>>1, The method according to clause 6B or clause 7B, further comprising selecting one of offsetX=-iNADistanceHor-1 and offsetY=-iNADistanceVer-1, where offsetX is the horizontal offset, offsetY is the vertical offset, width is the width of the current PU in some rumor samples, height is the height of the current PU in some rumor samples, iNA represents the i-th non-adjacent candidate unit, iNADistanceHor=width*(level index+1), and iNADistanceVer=height*(level index+1).

[0223]

[0229] Clause 9B. The number of candidates in the AMVP candidate list is greater than 2, as described in any of Clauses 1B-8B.

[0224]

[0230] The method described in Article 9B, where Article 10B states that the number of candidates in the AMVP candidate list is 5.

[0225]

[0231] The method according to any of the clauses 1B to 10B, further comprising determining the respective template matching cost for each candidate in the AMVP candidate list, and determining the motion vector predictor that is a candidate in the AMVP candidate list having the minimum template matching cost among multiple candidates in the AMVP candidate list.

[0226]

[0232] The method according to any of the clauses 1B to 11B, further comprising: Clause 12B. AMVP candidate list comprising AMVP candidates improved by template matching and AMVP candidates not improved by template matching; and method, signaling or parsing an AMVP index, the AMVP index indicating whether a motion vector predictor is an AMVP candidate improved by template matching or not.

[0227]

[0233] The method of Clause 12B, further comprising the condition that the AMVP index is equal to 0, wherein the method determines that the motion vector predictor is an AMVP candidate improved by template matching.

[0228]

[0234] The method of Clause 12B, further comprising the determination that the AMVP index is equal to 1, wherein the method determines that the motion vector predictor is an unimproved AMVP candidate.

[0229]

[0235] The method according to any of the clauses 12B to 14B, further comprising determining the respective template matching cost for each candidate in the AMVP candidate list, and using template matching to refine the AMVP candidate having the minimum template matching cost in order to create an AMVP candidate improved by template matching.

[0230]

[0236] Clause 16B. The method of any of Clauses 12B-15B, wherein an unimproved AMVP candidate unit has the maximum motion vector distance to an improved AMVP candidate among any potential AMVP candidates by template matching.

[0231]

[0237] Clause 17B. Unimproved AMVP candidates are compared with any potential AMVP candidates that have been improved by template matching. , minimum template matching cost vs Motion vector distance ratio The method described in any of clauses 12B to 15B.

[0232]

[0238] Clause 18B. The method of any one of Clauses 1B to 17B, wherein coding comprises encoding.

[0233]

[0239] Clause 19B. The method of any one of Clauses 1B to 17B, comprising coding and decoding.

[0234]

[0240] Clause 20B. A device for coding video data, comprising a memory configured to store video data, and one or more processors implemented in the circuit and communicatively coupled to the memory, wherein one or more processors are configured to determine at least one of time-based candidates or history-based candidates; determine at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein the at least one non-adjacent candidate unit determines an advanced motion vector predictor (AMVP) candidate list comprising at least one time-based candidate or history-based candidate and at least one non-adjacent candidate that is not adjacent to the current predictor unit (PU), wherein the at least one non-adjacent candidate codes the current PU based on the AMVP candidate list, which is added to the AMVP candidate list at least one of after the time-based candidate or before the history-based candidate.

[0235]

[0241] The device described in Clause 20B, wherein one or more processors are configured, as part of determining at least one non-adjacent candidate, to select at least one non-adjacent candidate unit from a group of potential non-adjacent candidate units consisting of units located 1 unit below and 2 units to the left of the current PU, 2 units above and 1 unit to the right of the current PU, 2 units above and 2 units to the left of the current PU, 2 units below and 3 units to the left of the current PU, 3 units above and 2 units to the right of the current PU, 3 units above the current PU, 3 units to the left of the current PU, 3 units above and 3 units to the left of the current PU, 4 units below and 4 units to the left of the current PU, 4 units above and 3 units to the right of the current PU, 4 units above the current PU, 4 units to the left of the current PU, 4 units above and 4 units to the left of the current PU, 5 units below and 5 units to the left of the current PU, 5 units above and 5 units to the left of the current PU.

[0236]

[0242] Clause 22B. A device according to Clause 20B or Clause 21B, wherein a plurality of potential non-adjacent candidate units are divided into a plurality of groups, wherein one or more processors are further configured to signal or parse level indices indicating groups among the plurality of groups, and to signal or parse position indices indicating one of the potential non-adjacent candidate units belonging to the group indicated by the level indices, wherein at least one non-adjacent candidate unit comprises one of the potential non-adjacent candidate units.

[0237]

[0243] Clause 23B. The device according to Clause 22B, wherein the multiple groups include a first group of potential non-adjacent candidate units located 1 unit below and 2 units to the left of the current PU, 2 units above and 1 unit to the right of the current PU, and 2 units above and 2 units to the left of the current PU; a second group of potential non-adjacent candidate units located 2 units below and 3 units to the left of the current PU, 3 units above and 2 units to the right of the current PU, 3 units above the current PU, 3 units to the left of the current PU, and 3 units above and 3 units to the left of the current PU; a third group of potential non-adjacent candidate units located 3 units below and 4 units to the left of the current PU, 4 units above and 3 units to the right of the current PU, 4 units above the current PU, 4 units to the left of the current PU, and 4 units above and 4 units to the left of the current PU; and a fourth group of potential non-adjacent candidate units located 4 units below and 5 units to the left of the current PU, 5 units above and 4 units to the right of the current PU, 5 units above the current PU, 5 units to the left of the current PU, and 5 units above and 5 units to the left of the current PU.

[0238]

[0244] The device according to any of the clauses 20B to 23B, wherein one or more processors are further configured to determine the size of the current PU, wherein one or more processors are configured to determine at least one non-adjacent candidate based at least in part on the size of the current PU.

[0239]

[0245] Clause 25B. A device according to any one of Clauses 20B to 24B, in which one or more processors are configured to determine a horizontal offset to the upper left lumens sample of the current PU and a vertical offset to the upper left lumens sample of the current PU as part of determining at least one non-adjacent candidate.

[0240]

[0246] Clause 26B. As part of determining at least one non-adjacent candidate unit, one or more processors are further configured to add a horizontal offset and a vertical offset to the lumens sample coordinates of the top-left of the current PU, as described in Clause 25B.

[0241]

[0247] Clause 27B. One or more processors use offsetX=-iNADistanceHor-1 and offsetY=height+iNADistanceVer-1, offsetX=width+iNADistanceHor-1 and offsetY=-iNADistanceVer-1, offsetX=width>>1 and offsetY=-iNADistanceVer-1, offsetX=-iNADistanceHor-1 and offsetY=height>>1 to determine the horizontal and vertical offsets according to a first syntax element indicating the level index and a second syntax element indicating the horizontal and vertical offsets. The device described in Clause 25B or Clause 26B is further configured to do the following: select one of offsetX=-iNADistanceHor-1 and offsetY=-iNADistanceVer-1, where offsetX is the horizontal offset, offsetY is the vertical offset, width is the width of the current PU in some lumane samples, height is the height of the current PU in some lumane samples, iNA represents the i-th non-adjacent candidate unit, iNADistanceHor=width*(level index+1), and iNADistanceVer=height*(level index+1).

[0242]

[0248] A device listed in any of the clauses 20B-27B, where the number of candidates in the AMVP candidate list is greater than 2.

[0243]

[0249] The number of candidates in the AMVP candidate list is 5, as described in Clause 29B, for the devices described in Clause 28B.

[0244]

[0250] The device according to any one of the clauses 20B to 29B, wherein one or more processors are further configured to determine the respective template matching cost for each candidate in the AMVP candidate list and to determine the motion vector predictor that is the candidate in the AMVP candidate list having the minimum template matching cost among multiple candidates in the AMVP candidate list.

[0245]

[0251] The AMVP candidate list in any of the devices in any of the clauses 20B to 30B comprises AMVP candidates improved by template matching and unimproved AMVP candidates, wherein one or more processors are further configured to signal or parse AMVP indices, the AMVP indices indicating whether a motion vector predictor is an AMVP candidate improved by template matching or an unimproved AMVP candidate.

[0246]

[0252] Clause 32B. The AMVP index is equal to 0, wherein one or more processors are further configured to determine that the motion vector predictor is an AMVP candidate improved by template matching, as described in Clause 31B.

[0247]

[0253] Clause 33B. The AMVP index is equal to 1, wherein one or more processors are further configured to determine that the motion vector predictor is an unimproved AMVP candidate, as described in Clause 31B.

[0248]

[0254] The device according to any one of the clauses 31B to 33B, wherein one or more processors are further configured to determine the respective template matching cost for each candidate in the AMVP candidate list and to use template matching to refine the AMVP candidate having the minimum template matching cost in order to create an AMVP candidate improved by template matching.

[0249]

[0255] Clause 35B. An unimproved AMVP candidate is a device described in any of Clauses 31B to 34B that has the maximum motion vector distance to an improved AMVP candidate in template matching among any potential AMVP candidates.

[0250]

[0256] Clause 36B. Unimproved AMVP candidates are compared with any potential AMVP candidates that have been improved by template matching. , minimum template matching cost vs Motion vector distance ratio A device possessing any of the devices described in any of clauses 31B to 34B.

[0251]

[0257] Clause 37B. A device as described in any of Clauses 20B to 36B, further comprising a display configured to display decoded video data.

[0252]

[0258] Clause 38B. A device as described in any of Clauses 20B to 37B, further comprising a camera configured to capture video data.

[0253]

[0259] Clause 39B. A computer-readable storage medium storing instructions, the instructions, when executed, cause one or more processors to: determine at least one of time-based candidates or history-based candidates; determine at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein at least one non-adjacent candidate unit determines an advanced motion vector predictor (AMVP) candidate list comprising at least one time-based candidate or history-based candidate and at least one non-adjacent candidate that is not adjacent to the current predictor unit (PU); wherein at least one non-adjacent candidate codes the current PU based on the AMVP candidate list, which is added to the AMVP candidate list at least one of after the time-based candidates or before the history-based candidates.

[0254]

[0260] Clause 40B. A device for coding video data, comprising means for determining at least one of time-based candidates or history-based candidates; means for determining at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein the at least one non-adjacent candidate unit is means for determining an advanced motion vector predictor (AMVP) candidate list comprising at least one of time-based candidates or history-based candidates and at least one non-adjacent candidate, wherein the at least one non-adjacent candidate is added to the AMVP candidate list at least one of after time-based candidates or before history-based candidates, and means for coding the current PU based on the AMVP candidate list.

[0255]

[0261] In accordance with the examples above, it should be noted that some of the actions or events of any of the techniques described herein may be performed in different sequences, added, merged, or completely excluded (for example, not all described actions or events are necessarily required for the practice of this technique). Furthermore, in some examples, actions or events may be performed not sequentially, but simultaneously, for example, through multithreading, interrupt handling, or across multiple processors.

[0256]

[0262] In one or more examples, the described functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on or transmitted through a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may include computer-readable storage media corresponding to tangible media such as data storage media, or it may include communication media, for example, any medium that facilitates the transfer of computer programs from one location to another according to a communication protocol. Thus, the computer-readable medium may generally correspond to (1) non-transient tangible computer-readable storage media, or (2) communication media such as signals or carrier waves. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementation of the techniques described herein. Computer program products may include computer-readable media.

[0257]

[0263] As an example, and not an limitation, such computer-readable storage media may include RAM, ROM, EEPROM®, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and can be accessed by a computer. Any connection is also appropriately referred to as computer-readable media. For example, if instructions are transmitted from a website, server or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media. However, it should be understood that computer-readable storage media and data storage media refer to non-temporary, tangible storage media, rather than connections, carriers, signals, or other temporary media. As used herein, the terms "disk" and "disc" include Compact Disc (CD), LaserDisc® (disc), Optical Disc (disc), Digital Multipurpose Disc (disc) (DVD), Floppy Disk (disk), and Blu-ray Disc (disc), where a disk typically reproduces data magnetically, and a disc reproduces data optically using a laser. Any combination of the above should also be included within the scope of computer-readable media.

[0258]

[0264] Instructions may be executed by one or more processors, such as one or more DSPs, general-purpose microprocessors, ASICs, FPGAs, or other equivalent integrated circuits or discrete logic circuits. Therefore, the terms “processor” and “processing circuit” as used herein may refer to any of the above-described structures or any other structure suitable for implementing the techniques described herein. Furthermore, in some embodiments, the functions described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a composite codec. Moreover, the techniques can be adequately implemented in one or more circuits or logic elements.

[0259]

[0265] The techniques of this disclosure can be implemented in a wide variety of devices or apparatus, including wireless handsets, integrated circuits (ICs), or sets of ICs (e.g., chipsets). Various components, modules, or units are described in this disclosure to highlight the functional aspects of a device configured to perform the disclosed techniques, but do not necessarily require implementation by different hardware units. Rather, as described above, various units, along with suitable software and / or firmware, may be combined in a codec hardware unit, including one or more processors described above, or provided by a set of interoperable hardware units.

[0260]

[0266] Various examples have been described. These and other examples fall within the scope of the attached claims. The invention described in the original claims of this application is listed below. [1] A method for coding video data, Determine at least one of the time-based candidates or historical-based candidates, To determine at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein the at least one non-adjacent candidate unit is not adjacent to the current prediction unit (PU). Determining an advanced motion vector predictor (AMVP) candidate list comprising at least one of the time candidates or the history-based candidates and at least one non-adjacent candidate, wherein the at least one non-adjacent candidate is added to the AMVP candidate list at least one of the time candidates or the history-based candidates. Coding the current PU based on the AMVP candidate list A method for providing this. [2] The method according to [1], wherein determining the at least one non-adjacent candidate is further comprising selecting the at least one non-adjacent candidate unit from a group of potential non-adjacent candidate units located 1 unit below and 2 units to the left of the current PU, 2 units above and 1 unit to the right of the current PU, 2 units above and 2 units to the left of the current PU, 2 units above and 2 units to the right of the current PU, 3 units above the current PU, 3 units to the left of the current PU, 3 units above and 3 units to the left of the current PU, 4 units below and 4 units to the left of the current PU, 4 units above and 3 units to the right of the current PU, 4 units above the current PU, 4 units to the left of the current PU, 4 units above and 4 units to the left of the current PU, 4 units below and 5 units to the left of the current PU, 5 units above and 4 units to the right of the current PU, 5 units above the current PU, 5 units to the left of the current PU, and 5 units above and 5 units to the left of the current PU. [3] Multiple potential non-adjacent candidate units are divided into multiple groups, wherein the method is Signaling or parsing level indices that represent groups among the aforementioned multiple groups, Signaling or parsing a position index indicating one of the potential non-adjacent candidate units belonging to the group indicated by the level index, wherein the at least one non-adjacent candidate unit comprises the one of the potential non-adjacent candidate units. The method described in [1] further comprises the following: [4] The aforementioned groups are A first group of potential non-adjacent candidate units located one unit below and two units to the left of the current PU, two units above and one unit to the right of the current PU, and two units above and two units to the left of the current PU, A second group of potential non-adjacent candidate units located 2 units below and 3 units to the left of the current PU, 3 units above and 2 units to the right of the current PU, 3 units above the current PU, 3 units to the left of the current PU, and 3 units above and 3 units to the left of the current PU, A third group of potential non-adjacent candidate units located 3 units below and 4 units to the left of the current PU, 4 units above and 3 units to the right of the current PU, 4 units above the current PU, 4 units to the left of the current PU, and 4 units above and 4 units to the left of the current PU, A fourth group of potential non-adjacent candidate units located 4 units below and 5 units to the left of the current PU, 5 units above and 4 units to the right of the current PU, 5 units above the current PU, 5 units to the left of the current PU, and 5 units above and 5 units to the left of the current PU. The method according to [3], comprising: [5] Determining the size of the current PU, wherein determining the at least one non-adjacent candidate is at least partially based on the size of the current PU, The method described in [1] further comprises the following: [6] The method according to [1], wherein determining the at least one non-adjacent candidate comprises determining the horizontal offset of the current PU with respect to the upper left lumens sample and the vertical offset of the current PU with respect to the upper left lumens sample. [7] The method according to [6], wherein determining the at least one non-adjacent candidate unit further comprises adding the horizontal offset and the vertical offset to the lumens sample coordinates of the current PU. [8] According to a first syntax element indicating the level index and a second syntax element indicating the horizontal offset and the vertical offset, offsetX=-iNADistanceHor-1 and offsetY=height+iNADistanceVer-1, offsetX=width+iNADistanceHor-1 and offsetY=-iNADistanceVer-1, offsetX=width>>1 and offsetY=-iNADistanceVer-1, offsetX=-iNADistanceHor-1 and offsetY=height>>1, and The method according to [6], further comprising selecting one of offsetX=-iNADistanceHor-1 and offsetY=-iNADistanceVer-1, where offsetX is the horizontal offset, offsetY is the vertical offset, width is the width of the current PU in some lumane samples, height is the height of the current PU in some lumane samples, iNA represents the i-th non-adjacent candidate unit, iNADistanceHor=width*(the level index+1), and iNADistanceVer=height*(the level index+1). [9] The method described in [1], wherein the number of candidates in the AMVP candidate list is greater than 2.

[10] The method according to [9], wherein the number of candidates in the AMVP candidate list is 5.

[11] Determine the template matching cost for each candidate in the AMVP candidate list, The method according to [1], further comprising determining a motion vector predictor which is a candidate in the AMVP candidate list having the smallest template matching cost among multiple candidates in the AMVP candidate list.

[12] The AMVP candidate list comprises AMVP candidates improved by template matching and AMVP candidates not improved by template matching, the method further comprising signaling or parsing an AMVP index, the AMVP index indicating whether the motion vector predictor is an AMVP candidate improved by template matching or an AMVP candidate not improved by template matching, the method according to [1].

[13] The AMVP index is equal to 0, where the method is The motion vector predictor is determined to be an AMVP candidate improved by the template matching. The method described in

[12] , further comprising:

[14] The AMVP index is equal to 1, where the method is The motion vector predictor is determined to be the unimproved AMVP candidate. The method described in

[12] , further comprising:

[15] Determine the template matching cost for each candidate in the AMVP candidate list, Using template matching, improve the AMVP candidate having the minimum template matching cost in order to create an AMVP candidate improved by the template matching. The method described in

[12] , further comprising:

[16] The method according to

[12] , wherein the unimproved AMVP candidate has the greatest motion vector distance from any of the potential AMVP candidates to the AMVP candidate improved by template matching.

[17] The method according to

[12] , wherein the unimproved AMVP candidate has the minimum template matching cost with respect to the motion vector distance ratio of any potential AMVP candidate that has been improved by template matching.

[18] The method described in [1], wherein coding comprises encoding.

[19] The method of [1] comprising coding and decoding.

[20] A device for coding video data, A memory configured to store the aforementioned video data, The circuit comprises one or more processors implemented in the circuit and communicatively coupled to the memory, wherein the one or more processors Determine at least one of the time-based candidates or historical-based candidates, To determine at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein the at least one non-adjacent candidate unit is not adjacent to the current prediction unit (PU). Determining an advanced motion vector predictor (AMVP) candidate list comprising at least one of the time candidates or the history-based candidates and at least one non-adjacent candidate, wherein the at least one non-adjacent candidate is added to the AMVP candidate list at least one of the time candidates or the history-based candidates. Coding the current PU based on the AMVP candidate list A device configured to perform the following actions.

[21] The device according to

[20] , wherein, as part of determining the at least one non-adjacent candidate, the one or more processors are configured to select the at least one non-adjacent candidate unit from a group of potential non-adjacent candidate units consisting of units located 1 unit below and 2 units to the left of the current PU, 2 units above and 1 unit to the right of the current PU, 2 units above and 2 units to the left of the current PU, 2 units above and 2 units to the right of the current PU, 3 units above the current PU, 3 units to the left of the current PU, 3 units above and 3 units to the left of the current PU, 4 units below the current PU, 3 units above and 3 units to the right of the current PU, 4 units above the current PU, 4 units above the current PU, 4 units to the left of the current PU, 4 units above and 4 units to the left of the current PU, 5 units below the current PU, 4 units above and 4 units to the right of the current PU, 5 units above the current PU, 5 units to the left of the current PU, and 5 units above and 5 units to the left of the current PU.

[22] Multiple potential non-adjacent candidate units are divided into multiple groups, wherein one or more processors Signaling or parsing level indices that represent groups among the aforementioned multiple groups, Signaling or parsing a position index indicating one of the potential non-adjacent candidate units belonging to the group indicated by the level index, wherein the at least one non-adjacent candidate unit comprises the one of the potential non-adjacent candidate units. The device described in

[20] is further configured to perform the following:

[23] The aforementioned groups A first group of potential non-adjacent candidate units located one unit below and two units to the left of the current PU, two units above and one unit to the right of the current PU, and two units above and two units to the left of the current PU, A second group of potential non-adjacent candidate units located 2 units below and 3 units to the left of the current PU, 3 units above and 2 units to the right of the current PU, 3 units above the current PU, 3 units to the left of the current PU, and 3 units above and 3 units to the left of the current PU, A third group of potential non-adjacent candidate units located 3 units below and 4 units to the left of the current PU, 4 units above and 3 units to the right of the current PU, 4 units above the current PU, 4 units to the left of the current PU, and 4 units above and 4 units to the left of the current PU, A fourth group of potential non-adjacent candidate units located 4 units below and 5 units to the left of the current PU, 5 units above and 4 units to the right of the current PU, 5 units above the current PU, 5 units to the left of the current PU, and 5 units above and 5 units to the left of the current PU. The device described in

[22] , comprising:

[24] The one or more processors Determining the size of the current PU, wherein one or more processors are configured to determine the at least one non-adjacent candidate based at least partially on the size of the current PU. The device described in

[20] is further configured to perform the following:

[25] As part of determining the at least one non-adjacent candidate, the device according to

[20] is configured to determine a horizontal offset of the current PU with respect to the upper left lumens sample and a vertical offset of the current PU with respect to the upper left lumens sample.

[26] As part of determining the at least one non-adjacent candidate unit, the device according to

[25] is further configured to add the horizontal offset and the vertical offset to the lumens sample coordinates of the current PU.

[27] The one or more processors use offsetX=-iNADistanceHor-1 and offsetY=height+iNADistanceVer-1, offsetX=width+iNADistanceHor-1 and offsetY=-iNADistanceVer-1, offsetX=width>>1 and offsetY=-iNADistanceVer-1, offsetX=-iNADistanceHor-1 and offsetY=height>> The device described in

[25] is further configured to do 1, or select one of offsetX=-iNADistanceHor-1 and offsetY=-iNADistanceVer-1, where offsetX is the horizontal offset, offsetY is the vertical offset, width is the width of the current PU in some lumens samples, height is the height of the current PU in some lumens samples, iNA is the i-th non-adjacent candidate unit, iNADistanceHor=width*(the level index+1), and iNADistanceVer=height*(the level index+1).

[28] The device described in

[20] , wherein the number of candidates in the AMVP candidate list is greater than 2.

[29] The device according to

[28] , wherein the number of candidates in the AMVP candidate list is 5.

[30] The one or more processors Determining the template matching cost for each candidate in the aforementioned AMVP candidate list, The device according to

[20] , further configured to determine a motion vector predictor which is a candidate in the AMVP candidate list having the smallest template matching cost among multiple candidates in the AMVP candidate list.

[31] The AMVP candidate list comprises AMVP candidates improved by template matching and AMVP candidates not improved by template matching, wherein the one or more processors are further configured to signal or parse an AMVP index, the AMVP index indicating whether a motion vector predictor is an AMVP candidate improved by template matching or an AMVP candidate not improved by template matching, as described in

[20] .

[32] The AMVP index is equal to 0, where the one or more processors are The motion vector predictor is determined to be an AMVP candidate improved by the template matching. The device described in

[31] , further configured to perform the following:

[33] The AMVP index is equal to 1, wherein the one or more processors are The motion vector predictor is determined to be the unimproved AMVP candidate. The device described in

[31] , further configured to perform the following:

[34] The one or more processors Determining the template matching cost for each candidate in the aforementioned AMVP candidate list, Using template matching, improve the AMVP candidate having the minimum template matching cost in order to create an AMVP candidate improved by the template matching. The device described in

[31] , further configured to perform the following:

[35] The device according to

[31] , wherein the unimproved AMVP candidate has the greatest motion vector distance from any of the potential AMVP candidates to the AMVP candidate improved by template matching.

[36] The device according to

[31] , wherein the unimproved AMVP candidate has the minimum template matching cost with respect to the motion vector distance ratio among any potential AMVP candidates improved by template matching.

[37] The device described in

[20] further comprises a display configured to show the decoded video data.

[38] The device according to

[20] , further comprising a camera configured to capture the video data.

[39] A computer-readable storage medium storing instructions, wherein, when an instruction is executed, one or more processors Determine at least one of the time-based candidates or historical-based candidates, To determine at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein the at least one non-adjacent candidate unit is not adjacent to the current prediction unit (PU). Determining an advanced motion vector predictor (AMVP) candidate list comprising at least one of the time candidates or the history-based candidates and at least one non-adjacent candidate, wherein the at least one non-adjacent candidate is added to the AMVP candidate list at least one of the time candidates or the history-based candidates. Coding the current PU based on the AMVP candidate list A computer-readable storage medium that enables the following process.

[40] A device for coding video data, Means for determining at least one of time-based candidates or historical-based candidates, and means for determining at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein the at least one non-adjacent candidate unit is not adjacent to the current prediction unit (PU). Means for determining an advanced motion vector predictor (AMVP) candidate list comprising at least one of the time candidates or the history-based candidates and at least one non-adjacent candidate, wherein the at least one non-adjacent candidate is added to the AMVP candidate list at least one of the time candidates or the history-based candidates. A device comprising means for coding the current PU based on the AMVP candidate list.

Claims

1. A method for coding video data, Determine at least one of the time-based candidates or historical-based candidates, To determine at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein the at least one non-adjacent candidate unit is not adjacent to the current prediction unit (PU). Determining an advanced motion vector predictor (AMVP) candidate list comprising at least one of the time candidates or the history-based candidates and at least one non-adjacent candidate, wherein the at least one non-adjacent candidate is added to the AMVP candidate list at least one of the time candidates or the history-based candidates, the AMVP candidate list comprising template-matching-enhanced AMVP candidates and non-enhanced AMVP candidates, the non-enhanced AMVP candidate having the smallest template matching cost to motion vector distance ratio with respect to the template-matching-enhanced AMVP candidate among any potential AMVP candidates. Signaling or parsing an AMVP index, and indicating whether the motion vector predictor is an AMVP candidate improved by the template matching or an unimproved AMVP candidate. Coding the current PU based on the motion vector predictor, A method for providing this.

2. The method according to claim 1, wherein determining the at least one non-adjacent candidate unit comprises selecting the at least one non-adjacent candidate unit from a group of potential non-adjacent candidate units located one unit below and two units to the left of the current PU, two units above and one unit to the right of the current PU, two units above and two units to the left of the current PU, two units below and three units to the left of the current PU, three units above and two units to the right of the current PU, three units above the current PU, three units to the left of the current PU, three units above and three units to the left of the current PU, four units above the current PU, four units above the current PU, four units to the left of the current PU, four units above and four units to the left of the current PU, five units below and five units to the left of the current PU, five units above and four units to the right of the current PU, five units above the current PU, five units to the left of the current PU, and five units above and five units to the left of the current PU.

3. Multiple potential non-adjacent candidate units are divided into multiple groups, where the method is as follows: Signaling or parsing level indices that represent groups among the aforementioned multiple groups, Signaling or parsing a position index indicating one of the potential non-adjacent candidate units belonging to the group indicated by the level index, wherein the at least one non-adjacent candidate unit comprises one of the potential non-adjacent candidate units. Furthermore, Optionally, the aforementioned multiple groups A first group of potential non-adjacent candidate units located one unit below and two units to the left of the current PU, two units above and one unit to the right of the current PU, and two units above and two units to the left of the current PU, A second group of potential non-adjacent candidate units located 2 units below and 3 units to the left of the current PU, 3 units above and 2 units to the right of the current PU, 3 units above the current PU, 3 units to the left of the current PU, and 3 units above and 3 units to the left of the current PU, A third group of potential non-adjacent candidate units located 3 units below and 4 units to the left of the current PU, 4 units above and 3 units to the right of the current PU, 4 units above the current PU, 4 units to the left of the current PU, and 4 units above and 4 units to the left of the current PU, A fourth group of potential non-adjacent candidate units located 4 units below and 5 units to the left of the current PU, 5 units above and 4 units to the right of the current PU, 5 units above the current PU, 5 units to the left of the current PU, and 5 units above and 5 units to the left of the current PU, The method according to claim 1, comprising:

4. Determining the size of the current PU, wherein determining the at least one non-adjacent candidate is at least partially based on the size of the current PU, The method according to claim 1, further comprising:

5. The method according to claim 1, wherein determining the at least one non-adjacent candidate comprises determining the horizontal offset of the current PU with respect to the upper-left lumens sample and the vertical offset of the current PU with respect to the upper-left lumens sample.

6. The method of claim 5, wherein determining the at least one non-adjacent candidate unit further comprises adding the horizontal offset and the vertical offset to the lumens sample coordinates of the current PU.

7. According to a first syntax element indicating the level index and a second syntax element indicating the horizontal and vertical offsets, offsetX = -iNADistanceHor-1 and offsetY = height + iNADistanceVer-1, offsetX = width + iNADistanceHor-1 and offsetY = -iNADistanceVer-1, offsetX = width>>1 and offsetY = -iNADistanceVer-1, offsetX = -iNADistanceHor-1 and offsetY = height>>1, and The method according to claim 5, further comprising selecting one of offsetX = -iNADistanceHor-1 and offsetY = -iNADistanceVer-1, where offsetX is the horizontal offset, offsetY is the vertical offset, width is the width of the current PU in some lumern samples, height is the height of the current PU in some lumern samples, iNA represents the i-th non-adjacent candidate unit, iNADistanceHor = width * (level index + 1), and iNADistanceVer = height * (level index + 1).

8. The method according to claim 1, wherein the number of candidates in the AMVP candidate list is greater than 2.

9. Determining the template matching cost for each candidate in the aforementioned AMVP candidate list, To determine the motion vector predictor that is a candidate in the AMVP candidate list having the smallest template matching cost among multiple candidates in the AMVP candidate list. The method according to claim 1, further comprising:

10. The AMVP index is equal to 0, and herein, the method is The motion vector predictor is determined to be an AMVP candidate improved by the template matching. To further include, or, The AMVP index is equal to 1, and herein, the method is The motion vector predictor is determined to be the unimproved AMVP candidate. The method according to claim 1, further comprising:

11. Determining the template matching cost for each candidate in the aforementioned AMVP candidate list, Using template matching, improve the AMVP candidate having the minimum template matching cost in order to create an AMVP candidate improved by the template matching. The method according to claim 1, further comprising:

12. The method according to claim 1, wherein coding comprises encoding or decoding.

13. The method according to claim 8, wherein the number of candidates in the AMVP candidate list is 5.

14. A device for coding video data, A memory configured to store the aforementioned video data, One or more processors implemented in the circuit and coupled to the memory in a communicative manner. The one or more processors are equipped with Determine at least one of the time-based candidates or historical-based candidates, To determine at least one non-adjacent candidate from at least one non-adjacent candidate unit, wherein the at least one non-adjacent candidate unit is not adjacent to the current prediction unit (PU). Determining an advanced motion vector predictor (AMVP) candidate list comprising at least one of the time candidates or the history-based candidates and at least one non-adjacent candidate, wherein the at least one non-adjacent candidate is added to the AMVP candidate list at least one of the time candidates or the history-based candidates, the AMVP candidate list comprising template-matching-enhanced AMVP candidates and non-enhanced AMVP candidates, the non-enhanced AMVP candidate having the smallest template matching cost to motion vector distance ratio with respect to the template-matching-enhanced AMVP candidate among any potential AMVP candidates. Signaling or parsing an AMVP index, and indicating whether the motion vector predictor is an AMVP candidate improved by the template matching or an unimproved AMVP candidate. Coding the current PU based on the motion vector predictor, A device configured to perform the following actions.

15. The device according to claim 14, wherein the one or more processors are further configured to carry out the method described in any one of claims 2 to 13.

16. A computer-readable storage medium storing instructions, wherein, when executed, the instructions cause one or more processors to carry out the method according to any one of claims 1 to 13.

Images