Vector difference candidate list construction

Abstract

A method for decoding video data includes determining a vector predictor for a current block; determining a first set of vector difference candidates based on a first pel interval defining a first distance between consecutive vector difference candidates in the first set of vector difference candidates starting from the vector predictor; determining a second set of vector difference candidates based on the first set of vector difference candidates and a second pel interval defining a second distance between consecutive vector difference candidates in the second set of vector difference candidates; constructing a vector difference candidate list based on the second set of vector difference candidates; determining a vector for the current block based on the vector predictor and a vector difference from the vector difference candidate list; and reconstructing the current block based on the vector for the current block.

Description

[Technical Field]

[0001] This application claims priority to U.S. Patent Application No. 18 / 540,046, filed December 14, 2023, and U.S. Provisional Patent Application No. 63 / 478,323, filed January 3, 2023, and U.S. Provisional Patent Application No. 63 / 480,280, filed January 17, 2023, the entire contents of which are incorporated herein by reference. U.S. Patent Application No. 18 / 540,046, filed December 14, 2023, claims the benefit of U.S. Provisional Patent Application No. 63 / 478,323, filed January 3, 2022, and U.S. Provisional Patent Application No. 63 / 480,280, filed January 17, 2022.

[0002]

[0002] This disclosure relates to video encoding and decoding. [Background technology]

[0003]

[0003] Digital video capabilities can be incorporated into a wide range of devices, including digital televisions, digital direct broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video gaming devices, video game consoles, cellular or satellite wireless telephones, so-called "smartphones," video teleconferencing devices, video streaming devices, and the like. Digital video devices implement video coding techniques such as those described in standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, Advanced Video Coding (AVC), ITU-T H.265 / High Efficiency Video Coding (HEVC), ITU-T H.266 / Versatile Video Coding (VVC), and extensions to such standards, as well as proprietary video codecs / formats such as AOMedia Video 1 (AV1) developed by the Alliance for Open Media. By implementing such video coding techniques, video devices may transmit, receive, encode, decode, and / or store digital video information more efficiently.

[0004]

[0004] Video coding techniques include spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to reduce or remove redundancy inherent in video sequences. In block-based video coding, a video slice (e.g., a video picture or a portion of a video picture) may be partitioned into video blocks, which are also sometimes referred to as coding tree units (CTUs), coding units (CUs), and / or coding nodes. Video blocks in an intra-coded (I) slice of a picture are encoded using spatial prediction with respect to reference samples in neighboring blocks in the same picture. Video blocks in an inter-coded (P or B) slice of a picture may use spatial prediction with respect to reference samples in neighboring blocks in the same picture or temporal prediction with respect to reference samples in other reference pictures. A picture may be referred to as a frame, and a reference picture may be referred to as a reference frame. Summary of the Invention

[0005] Generally, this disclosure describes techniques for constructing a vector difference candidate list (e.g., a block vector difference (BVD) candidate list or a motion vector difference (MVD) candidate list). As described in more detail, a video coder (e.g., a video encoder or a video decoder) may determine a set of vector difference candidates from which the video coder may construct the vector difference candidate list. For example, the video coder may use template matching or some other metric to sort the set of vector difference candidates and construct the vector difference candidate list.

[0006]

[0006] Coding gains can be realized by increasing the granularity of the distance between vector difference candidates and / or increasing the number of directions for the vector predictor that are evaluated. However, increasing the granularity and directions may result in additional vector difference candidates, which in turn increases the number of calculations for template matching or other such calculations when constructing a vector difference candidate list.

[0007]

[0007] This disclosure describes exemplary techniques for constructing vector difference candidate lists in a manner that allows for an increased number of vector difference candidates while reducing coding delays due to increased computation. For example, the exemplary techniques describe methods for dynamically determining which candidates are evaluated for inclusion in a vector difference candidate list such that more desirable candidates are more likely to be included in the vector difference candidate list while minimizing processing time inefficiencies. In this manner, the exemplary techniques provide practical applications for constructing vector difference candidate lists in a manner that improves the overall operation of video coding techniques.

[0008]

[0008] In one example, the present disclosure describes a method for decoding video data, the method including: determining a vector predictor for a current block of the video data; determining a first set of vector difference candidates based on a first pel interval that defines a first distance between consecutive vector difference candidates in the first set of vector difference candidates starting from the vector predictor; determining a second set of vector difference candidates based on the first set of vector difference candidates and a second pel interval that defines a second distance between consecutive vector difference candidates in the second set of vector difference candidates and that is smaller than the first pel interval; constructing a vector difference candidate list based on the second set of vector difference candidates; determining a vector for the current block based on the vector predictor and the vector difference from the vector difference candidate list; and reconstructing the current block based on the vector for the current block.

[0009]

[0009] In one example, the present disclosure describes a device for decoding video data, the device comprising one or more memories configured to store the video data and a processing circuit configuration coupled to the one or more memories, configured to: determine a vector predictor for a current block; determine a first set of vector difference candidates based on a first pel interval defining a first distance between consecutive vector difference candidates in the first set of vector difference candidates starting from the vector predictor; determine a second set of vector difference candidates based on the first set of vector difference candidates and a second pel interval defining a second distance between consecutive vector difference candidates in the second set of vector difference candidates, the second pel interval being smaller than the first pel interval; construct a vector difference candidate list based on the second set of vector difference candidates; determine a vector for the current block based on the vector predictor and the vector difference from the vector difference candidate list; and reconstruct the current block based on the vector for the current block.

[0010]

[0010] In one example, the present disclosure describes a computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to determine a vector predictor for a current block of video data, determine a first set of vector difference candidates based on a first pel interval that defines a first distance between consecutive vector difference candidates in the first set of vector difference candidates starting from the vector predictor, determine a second set of vector difference candidates based on the first set of vector difference candidates and a second pel interval that defines a second distance between consecutive vector difference candidates in the second set of vector difference candidates and that is smaller than the first pel interval, construct a vector difference candidate list based on the second set of vector difference candidates, determine a vector for the current block based on the vector predictor and a vector difference from the vector difference candidate list, and reconstruct the current block based on the vector for the current block.

[0011]

[0011] In one example, the present disclosure describes a device for encoding video data, the device including one or more memories configured to store the video data; and processing circuitry coupled to the one or more memories configured to: determine a vector predictor for a current block of the video data; determine a first set of vector difference candidates based on a first pel interval that defines a first distance between consecutive vector difference candidates in the first set of vector difference candidates starting from the vector predictor; determine a second set of vector difference candidates based on the first set of vector difference candidates and a second pel interval that defines a second distance between consecutive vector difference candidates in the second set of vector difference candidates, the second pel interval being smaller than the first pel interval; construct a vector difference candidate list based on the second set of vector difference candidates; and signal an index into the vector difference candidate list for decoding the current block.

[0012] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will become apparent from the description, drawings, and claims. [Brief explanation of the drawings]

[0013] [Figure 1]

[0013] FIG. 1 is a block diagram illustrating an example video encoding and decoding system that may implement the techniques of this disclosure. [Figure 2]

[0014] FIG. 2 is a block diagram illustrating an example video encoder that may implement the techniques of this disclosure. [Figure 3]

[0015] FIG. 2 is a block diagram illustrating an example video decoder that may implement the techniques of this disclosure. [Figure 4]

[0016] 10 is a flowchart illustrating an example method for encoding a current block, in accordance with techniques of this disclosure. [Figure 5]

[0017] 10 is a flowchart illustrating an example method for decoding a current block, in accordance with techniques of this disclosure. [Figure 6]

[0018] FIG. 1 is a conceptual diagram illustrating an example of padding candidates for replacement of zero vectors in an intra-block copy (IBC) list. [Figure 7]

[0019] 7A to 7D are conceptual diagrams illustrating examples of IBC reference regions that depend on the position of the current coding unit (CU). [Figure 8]

[0020] FIG. 1 is a conceptual diagram illustrating an example of a reference area for an IBC having a coding tree unit (CTU). [Figure 9]

[0021] FIG. 1 is a conceptual diagram illustrating an example of a template and reference samples for the template in a reference picture. [Figure 10]

[0022] 1 is a conceptual diagram illustrating an example of a template for a block and a reference sample of the template, along with motion of a sub-block using motion information of the sub-block of a current block. [Figure 11]

[0023] A conceptual diagram showing an example of an additional direction along the k*π / 8 diagonal angle. [Figure 12]

[0024] 12A and 12B are conceptual diagrams illustrating block vector (BV) adjustment for horizontal flipping. [Figure 13]

[0025] FIG. 10 is a conceptual diagram illustrating a first derivation step of a vector difference candidate list. [Figure 14]

[0026] FIG. 10 is a conceptual diagram illustrating a second derivation step of a vector difference candidate list. [Figure 15]

[0027] FIG. 15 is a conceptual diagram showing the third step of deriving the vector difference candidate list. [Figure 16]

[0028] FIG. 16 is a conceptual diagram showing the fourth step of deriving the vector difference candidate list. [Figure 17]

[0029] FIG. 1 is a conceptual diagram illustrating an example of an advanced motion vector prediction (AMVP) process. [Figure 18]

[0030] FIG. 10 is a conceptual diagram illustrating another example of the AMVP process using template search starting from a prefix vector. [Figure 19]

[0031] 10 is a flowchart illustrating another example method for decoding a current block, in accordance with techniques of this disclosure. [Figure 20]

[0032] 10 is a flowchart illustrating another example method for encoding a current block, in accordance with techniques of this disclosure. [Figure 21]

[0033] 1 is a flowchart illustrating an exemplary method for constructing a vector difference candidate list. [Figure 22]

[0034] 10 is a flowchart illustrating another exemplary method for constructing a vector difference candidate list. DETAILED DESCRIPTION OF THE INVENTION

[0014]

[0035] In video coding, a video encoder may determine a vector for a current block (e.g., a motion vector for inter prediction or a block vector for intra block copying (IBC)). However, rather than signaling information of the vector for the current block, the video encoder may signal a vector predictor (e.g., a vector of a neighboring block) and information indicating a vector difference between the vector for the current block and the vector predictor.

[0015]

[0036] In one or more examples, the video encoder and video decoder may construct a vector difference candidate list rather than or in addition to signaling information of the vector difference. The video encoder may signal, and the video decoder may receive, an index to the vector difference candidate list. The video decoder may determine a vector difference based on the index and determine a vector for the current block based on the vector difference (e.g., summing the vector difference and the vector predictor). The video decoder may determine a predictive block based on the determined vector. The video decoder may also receive residual information from the video encoder indicating a difference between the predictive block and the current block. The video decoder may reconstruct the current block based on the predictive block (e.g., adding the predictive block and the residual information).

[0016]

[0037] In some example techniques, such as IBC merge mode with block vector differences (IBC-MBVD) mode, the video encoder and video decoder use template matching to sort a set of vector difference candidates (e.g., 80 vector difference candidates) to build a vector difference candidate list (e.g., a block vector difference (BVD) list) of eight candidates for each vector predictor (e.g., a block vector predictor (BVP)).

[0017]

[0038] For example, there may be coding efficiency gains by increasing the number of vector difference candidates (e.g., from 80 to 1024) by increasing the granularity of the distance between vector difference candidates and by increasing the direction relative to the vector predictor that is evaluated. However, such an increase also increases the number of calculations. For example, if template matching is used, there may be 1024 template matching (TM) sum of absolute difference (SAD) calculations for each vector predictor when there are 1024 vector difference candidates, compared to 80 TM SAD calculations when there are 80 vector difference candidates.

[0018]

[0039] This disclosure describes example techniques for adding candidates to a subset of vector difference candidates, reordering the candidates to derive another subset (e.g., based on template matching), and selecting the first N candidates from the other subset. In this way, it may be possible to limit the number of candidates evaluated (e.g., for each step, the number of vector candidates to evaluate is limited), while also allowing for an increase in the granularity and direction of the vector candidates evaluated.

[0019]

[0040] For example, in IBC-MBVD, in some techniques, the video encoder and video decoder may be configured to use a predefined list of positions for the BVPs that the video encoder and video decoder evaluate. As an example, there may be 80 predefined BVDs. The video encoder and video decoder may add each of the 80 BVDs to the BVP, and for each resulting BV candidate (e.g., 80 BV candidates), the video encoder and video decoder may determine an individual cost value (e.g., using a template matching technique). The video encoder and video decoder may determine the eight BVDs that resulted in the lowest cost and construct a BVD candidate list by sorting the eight BVDs from lowest cost to highest cost.

[0020]

[0041] However, such a technique may have certain problems. For example, since 80 BVDs are predefined, there may be BVDs that provide better coding gains that have not been evaluated. One way to address this problem is to increase the number of BVDs that are evaluated. However, increasing the number of BVDs that are evaluated increases computational complexity and delays processing.

[0021]

[0042] For example, the BVD to be evaluated can be defined in terms of "pels," which defines how many samples away the video encoder and video decoder are from the BVP to evaluate. As an example, a distance of 50 pels may mean that the video encoder and video decoder add or subtract 50 from the x-coordinate of the BVP (e.g., if the BVD is 50) to determine the BV candidate. The video encoder and video decoder may determine a cost value of this BVD. The video encoder and video decoder may be able to evaluate at a granularity of 1 pel, 2 pels, 3 pels, 4 pels, etc. However, as mentioned above, increasing the granularity of some evaluated BVDs may be inefficient.

[0022]

[0043] This disclosure describes example techniques for constructing a vector difference candidate list based on cost values ​​associated with a first set of vector difference candidates and determining a second set of vector difference candidates based on the first set of vector difference candidates that have all been evaluated to construct the vector difference candidate list. In this manner, the vector difference candidates (e.g., BVDs) to be evaluated may be dynamically determined, resulting in the evaluation of vector difference candidates that are more likely to provide coding gain compared to examples in which vector difference candidates are predefined.

[0023]

[0044] For example, the video encoder and video decoder may determine a vector predictor (e.g., BVP) for a current block and determine a first set of vector difference candidates based on a first pel interval. For example, the first pel interval may define a first distance between consecutive vector difference candidates in the first set of vector difference candidates starting from the vector predictor. As an example, the video encoder and video decoder may evaluate BVDs with values ​​of 4 pels, 8 pels, 12 pels, 16 pels, etc. until a range threshold is met or a cost threshold is met. In this example, the distance between the vector difference candidates starting from the vector predictor is 4, so the first pel interval may be considered to be 4. Also, the distance between consecutive vector difference candidates is 4 (e.g., 4 pels from 4 pels to 8 pels, 4 pels from 8 pels to 12 pels, etc.).

[0024]

[0045] The video encoder and the video decoder may add candidates to the first set of vector difference candidates, each candidate being a set of positions that surround the position of the vector predictor and are an integer multiple of a first distance from the vector predictor. For example, a first vector difference candidate in the first set of vector difference candidates may be 4 pels away from the vector predictor, and a second vector difference candidate in the first set of vector difference candidates may be 2 pels away from the vector predictor. * The third vector difference candidate in the first set of vector difference candidates is 3 pels away from the vector predictor. * 4 pels (i.e., 12 pels) apart, etc.

[0025]

[0046] The video encoder and video decoder may determine a second set of vector difference candidates based on the first set of vector difference candidates and the second pel interval. The second pel interval may define a second distance between consecutive vector difference candidates in the second set of vector difference candidates. The second pel interval may be smaller than the first pel interval. For example, if the first pel interval is 4 pels, the second pel interval may be 2 pels or less.

[0026]

[0047] As an example, the video encoder and video decoder may include at least a subset of candidates from the first set of vector difference candidates in the second set of vector difference candidates and add candidates between the subset of candidates from the first set of vector difference candidates based on an integer multiple of the second distance. For example, assume that two vector difference candidates from the first set of vector difference candidates are included in the second set of vector difference candidates. If the second pel spacing is 2 pels and the second distance is equal to 2 pels, the video encoder and video decoder may add one more vector difference candidate between the two difference candidates from the first set of vector difference candidates to generate the second set of vector difference candidates. If the second pel spacing is 1 pel and the second distance is equal to 1 pel, the video encoder and video decoder may add three more vector difference candidates between the two difference candidates from the first set of vector difference candidates to generate the second set of vector difference candidates.

[0027]

[0048] The video encoder and video decoder may construct a vector difference candidate list based on the second set of vector difference candidates. As an example, the vector difference candidate list may be equal to the second set of vector difference candidates or a distilled version of the second set of vector difference candidates. As another example, the video encoder and video decoder may iteratively repeat the above steps with smaller and smaller pel spacings (e.g., shorter and shorter distances) up to a pel spacing threshold.

[0028]

[0049] Thus, in one or more examples, such as for IBC-MBVD list derivation (e.g., where the vector difference candidate list is an IBC-MBVD list), the exemplary technique enables adaptive BVD offset along the MBVD direction. For example, the video encoder and video decoder may perform an MBVD candidate search, which is a two-step algorithm, or possibly a more-step algorithm if iterative steps are performed. The video encoder and video decoder may start by checking the BVD cost values ​​added to the BVP at intervals of M pels along each direction. The second step of the search checks the cost values ​​of intermediate candidates around the selected candidate from the first step. The candidate with the lowest cost is included in the final MBVD list (e.g., in the vector difference candidate list).

[0029]

[0050] The video encoder may signal an index into the vector difference candidate list, and the video decoder may determine a vector for the current block based on the vector predictor and the vector difference determined from the index (e.g., add the vector predictor and the vector difference). The video decoder may then reconstruct the current block based on the vector for the current block.

[0030]

[0051] 1 is a block diagram illustrating an example video encoding and decoding system 100 in which techniques of this disclosure may be implemented. The techniques of this disclosure are generally directed to coding (encoding and / or decoding) video data. Generally, video data includes any data for processing video. Thus, video data may include raw uncoded video, coded video, decoded (e.g., reconstructed) video, and video metadata, such as signaling data.

[0031]

[0052] 1 , in this example, system 100 includes a source device 102 that provides encoded video data to be decoded and displayed by a destination device 116. Specifically, source device 102 provides the video data to destination device 116 via a computer-readable medium 110. Source device 102 and destination device 116 may be or include any of a wide range of devices, such as a desktop computer, a notebook (i.e., laptop) computer, a mobile device, a tablet computer, a set-top box, a telephone handset such as a smartphone, a television, a camera, a display device, a digital media player, a video gaming console, a video streaming device, a broadcast receiver device, etc. In some cases, source device 102 and destination device 116 may be capable of wireless communication and thus may be referred to as wireless communication devices.

[0032]

[0053] In the example of FIG. 1 , source device 102 includes video source 104, memory 106, video encoder 200, and output interface 108. Destination device 116 includes input interface 122, video decoder 300, memory 120, and display device 118. According to this disclosure, video encoder 200 of source device 102 and video decoder 300 of destination device 116 may be configured to apply techniques for vector difference candidate list construction (e.g., motion vector difference (MVD) or block vector difference (BVD) candidate list construction). Thus, source device 102 represents an example of a video encoding device, while destination device 116 represents an example of a video decoding device. In other examples, source device and destination device may include other components or configurations. For example, source device 102 may receive video data from an external video source, such as an external camera. Similarly, destination device 116 may interface with an external display device rather than including an integrated display device.

[0033]

[0054] System 100 as shown in FIG. 1 is merely an example. In general, any digital video encoding device and / or digital video decoding device may implement techniques for vector difference candidate list construction (e.g., motion vector difference (MVD) or block vector difference (BVD) candidate list construction). Source device 102 and destination device 116 are merely examples of coding devices, such that source device 102 generates the coded video data that destination device 116 transmits to destination device 116. This disclosure refers to devices that perform coding (encoding and / or decoding) of data as “coding” devices. Accordingly, video encoder 200 and video decoder 300 represent examples of coding devices, specifically, video encoders and video decoders, respectively. In some examples, source device 102 and destination device 116 may operate in a substantially symmetrical manner, such that source device 102 and destination device 116 each include video encoding and decoding components. Thus, system 100 may support one-way or two-way video transmission between source device 102 and destination device 116, for example, video streaming, video playback, video broadcasting, or video telephony.

[0034]

[0055] Generally, video source 104 represents a source of video data (i.e., raw, unencoded video data) and provides a continuous series of pictures (also called “frames”) of the video data to video encoder 200, which encodes the picture data. Video source 104 of source device 102 may include a video capture device such as a video camera, a video archive containing previously captured raw video, and / or a video feed interface that receives video from a video content provider. As a further alternative, video source 104 may generate computer-graphics-based data as source video, or a combination of live video, archived video, and computer-generated video. In each case, video encoder 200 encodes the captured, pre-captured, or computer-generated video data. Video encoder 200 may reorder the pictures from the order in which they were received (sometimes referred to as “display order”) to a coding order for coding. Video encoder 200 may generate a bitstream containing the encoded video data. Source device 102 may then output the encoded video data via output interface 108 to computer-readable medium 110, for receipt and / or retrieval by input interface 122 of destination device 116, for example.

[0035]

[0056] Memory 106 of source device 102 and memory 120 of destination device 116 represent general-purpose memory. In some examples, memory 106, 120 may store raw video data, e.g., raw video from video source 104 and raw decoded video data from video decoder 300. Additionally or alternatively, memory 106, 120 may store software instructions executable by, e.g., video encoder 200 and video decoder 300, respectively. While memory 106 and memory 120 are shown separate from video encoder 200 and video decoder 300 in this example, it should be understood that video encoder 200 and video decoder 300 may also include internal memory for functionally similar or equivalent purposes. Furthermore, memory 106, 120 may store, e.g., encoded video data output from video encoder 200 and input to video decoder 300. In some examples, a portion of the memory 106, 120 may be allocated as one or more video buffers, for example, to store raw decoded video data and / or raw encoded video data.

[0036]

[0057] The computer-readable medium 110 may represent any type of medium or device capable of transferring encoded video data from the source device 102 to the destination device 116. In one example, the computer-readable medium 110 represents a communication medium that enables the source device 102 to transmit encoded video data directly to the destination device 116 in real time, for example, via a radio frequency network or a computer-based network. The output interface 108 may modulate a transmission signal containing the encoded video data, and the input interface 122 may demodulate a received transmission signal in accordance with a communication standard such as a wireless communication protocol. The communication medium may comprise any wireless or wired communication medium, such as the radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful for facilitating communication from the source device 102 to the destination device 116.

[0037]

[0058] In some examples, source device 102 may output encoded data from output interface 108 to storage device 112. Similarly, destination device 116 may access encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray Disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium that stores encoded video data.

[0038]

[0059] In some examples, source device 102 may output encoded video data to file server 114 or another intermediate storage device that may store the encoded video data generated by source device 102. Destination device 116 may access the stored video data from file server 114 via streaming or download.

[0039]

[0060] File server 114 may be any type of server device capable of storing encoded video data and transmitting the encoded video data to destination device 116. File server 114 may represent a web server (e.g., for a website), a server configured to provide file transfer protocol services (such as File Transfer Protocol (FTP) or File Delivery over Unidirectional Transport (FLUTE) protocol), a content delivery network (CDN) device, a hypertext transfer protocol (HTTP) server, a Multimedia Broadcast Multicast Service (MBMS) or Enhanced MBMS (eMBMS) server, and / or a network attached storage (NAS) device. The file server 114 may additionally or alternatively implement one or more HTTP streaming protocols, such as Dynamic Adaptive Streaming over HTTP (DASH), HTTP Live Streaming (HLS), Real Time Streaming Protocol (RTSP), HTTP Dynamic Streaming, etc.

[0040]

[0061] Destination device 116 may access the encoded video data from file server 114 through any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both suitable for accessing encoded video data stored on file server 114. Input interface 122 may be configured to operate according to any one or more of the various protocols mentioned above to retrieve or receive media data from file server 114, or other such protocols to retrieve media data.

[0041]

[0062] Output interface 108 and input interface 122 may represent a wireless transmitter / receiver, a modem, a wired network component (e.g., an Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples in which output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 may be configured to transfer data, such as encoded video data, according to a cellular communication standard, such as 4G, 4G-LTE (Long Term Evolution), LTE-Advanced, 5G, etc. In some examples in which output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured to transfer data, such as encoded video data, according to other wireless standards, such as the IEEE 802.11 standard, the IEEE 802.15 standard (e.g., ZigBee™), the Bluetooth™ standard, etc. In some examples, source device 102 and / or destination device 116 may include respective system-on-chip (SoC) devices. For example, source device 102 may include an SoC device that performs the functionality attributed to video encoder 200 and / or output interface 108, and destination device 116 may include an SoC device that performs the functionality attributed to video decoder 300 and / or input interface 122.

[0042]

[0063] The techniques of this disclosure may be applied to video coding supporting any of a variety of multimedia applications, such as over-the-air television broadcast, cable television transmission, satellite television transmission, Internet streaming video transmission such as Dynamic Adaptive Streaming over HTTP (DASH), digital video encoded on a data storage medium, decoding of digital video stored on a data storage medium, or other applications.

[0043]

[0064] The input interface 122 of the destination device 116 receives an encoded video bitstream from a computer-readable medium 110 (e.g., a communications medium, a storage device 112, a file server 114, etc.). The encoded video bitstream may include signaling information defined by the video encoder 200 that is also used by the video decoder 300, such as syntax elements having values ​​that describe characteristics and / or processing of video blocks or other coded units (e.g., slices, pictures, groups of pictures, sequences, etc.). The display device 118 displays decoded pictures of the decoded video data to a user. The display device 118 may represent any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0044]

[0065] Although not shown in FIG. 1, in some examples, the video encoder 200 and the video decoder 300 may each be integrated with an audio encoder and / or audio decoder and may include appropriate MUX-DEMUX units or other hardware and / or software to handle multiplexed streams containing both audio and video in a common data stream.

[0045]

[0066] Video encoder 200 and video decoder 300 may each be implemented as any of a variety of suitable encoder and / or decoder circuit configurations, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the techniques are implemented partially in software, a device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to implement the techniques of this disclosure. Each of video encoder 200 and video decoder 300 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (CODEC) in a separate device. A device including the video encoder 200 and / or the video decoder 300 may implement the video encoder 200 and / or the video decoder 300 in processing circuitry such as an integrated circuit and / or a microprocessor. Such a device may be a wireless communication device, such as a cellular telephone, or any other type of device described herein.

[0046]

[0067] The video encoder 200 and the video decoder 300 may operate according to a video coding standard such as ITU-T H.265, also known as High Efficiency Video Coding (HEVC), or an extension thereof, such as multiview and / or scalable video coding extensions. Alternatively, the video encoder 200 and the video decoder 300 may operate according to other proprietary or industry standards, such as ITU-T H.266, also known as Generic Video Coding (VVC). In other examples, the video encoder 200 and the video decoder 300 may operate according to a proprietary video codec / format, such as AOMedia Video 1 (AV1), an extension of AV1, and / or a successor version of AV1 (e.g., AV2). In other examples, the video encoder 200 and the video decoder 300 may operate according to other proprietary formats or industry standards. However, the techniques of this disclosure are not limited to any particular coding standard or format. In general, video encoder 200 and video decoder 300 may be configured to implement the techniques of this disclosure in conjunction with any video coding technique that uses vector difference candidate list construction (e.g., motion vector difference (MVD) or block vector difference (BVD) candidate list construction).

[0047]

[0068] Generally, the video encoder 200 and the video decoder 300 may perform block-based coding of pictures. The term “block” generally refers to a structure containing data to be processed (e.g., encoded, decoded, or otherwise used in an encoding and / or decoding process). For example, a block may include a two-dimensional matrix of luminance and / or chrominance data samples. Generally, the video encoder 200 and the video decoder 300 may code video data represented in YUV (e.g., Y, Cb, Cr) format. That is, rather than coding red, green, and blue (RGB) data for picture samples, the video encoder 200 and the video decoder 300 may code luminance and chrominance components, which may include both red and blue chrominance components. In some examples, the video encoder 200 converts received RGB-format data to a YUV representation before encoding, and the video decoder 300 converts the YUV representation to RGB format. Alternatively, pre-processing and post-processing units (not shown) may perform these transformations.

[0048]

[0069] This disclosure may generally refer to coding (e.g., encoding and decoding) a picture as including the process of encoding or decoding data for a picture. Similarly, this disclosure may refer to coding a block of a picture as including the process of encoding or decoding data for the block, e.g., predictive and / or residual coding. A coded video bitstream generally includes a series of values ​​of syntax elements that represent coding decisions (e.g., coding modes) and the partitioning of a picture into blocks. Thus, references to coding a picture or a block should generally be understood as coding values ​​of the syntax elements that form the picture or block.

[0049]

[0070] HEVC defines various blocks, including coding units (CUs), prediction units (PUs), and transform units (TUs). According to HEVC, a video coder (such as video encoder 200) partitions coding tree units (CTUs) into CUs according to a quadtree structure. That is, the video coder partitions CTUs and CUs into four equal, non-overlapping squares, and each node of the quadtree has either zero or four child nodes. A node with no child nodes may be called a "leaf node," and a CU of such a leaf node may include one or more PUs and / or one or more TUs. The video coder may further partition PUs and TUs. For example, in HEVC, a residual quadtree (RQT) represents the partitioning of TUs. In HEVC, a PU represents inter-predicted data, and a TU represents residual data. An intra-predicted CU includes intra-prediction information, such as an intra-mode indication.

[0050]

[0071] As another example, video encoder 200 and video decoder 300 may be configured to operate according to VVC. According to VVC, a video coder (such as video encoder 200) partitions a picture into multiple CTUs. Video encoder 200 may partition the CTUs according to a tree structure, such as a quadtree-binary tree (QTBT) structure or a multi-type tree (MTT) structure. The QTBT structure eliminates the concept of multiple partition types, such as the separation between CUs, PUs, and TUs in HEVC. The QTBT structure includes two levels: a first level partitioned according to quadtree partitioning and a second level partitioned according to binary tree partitioning. The root node of the QTBT structure corresponds to a CTU. The leaf nodes of the binary tree correspond to CUs.

[0051]

[0072] In the MTT partitioning structure, blocks may be partitioned using quadtree (QT) partitioning, binary tree (BT) partitioning, and one or more types of triple tree (TT) (also called ternary tree (TT)) partitioning. A triple tree partitioning or triple tree partitioning is a partitioning in which a block is divided into three sub-blocks. In some examples, a triple tree partitioning or triple tree partitioning divides a block into three sub-blocks without splitting the original block through the center. The partition types in MTT (e.g., QT, BT, and TT) can be symmetric or asymmetric.

[0052]

[0073] When operating according to the AV1 codec, the video encoder 200 and the video decoder 300 may be configured to code video data in blocks. In AV1, the largest coding block that can be processed is called a superblock. In AV1, a superblock can be either 128x128 luma samples or 64x64 luma samples. However, in successor video coding formats (e.g., AV2), a superblock can be defined by a different (e.g., larger) luma sample size. In some examples, a superblock is the top level of a block quadtree. The video encoder 200 may further partition the superblock into smaller coding blocks. The video encoder 200 may partition the superblock and other coding blocks into smaller blocks using square or non-square partitioning. Non-square blocks may include N / 2xN, NxN / 2, N / 4xN, and NxN / 4 blocks. The video encoder 200 and the video decoder 300 may perform separate prediction and transform processes for each of the coding blocks.

[0053]

[0074] AV1 also defines tiles of video data. A tile is a rectangular array of superblocks that may be coded independently of other tiles. That is, video encoder 200 and video decoder 300 may encode and decode coding blocks within a tile, respectively, without using video data from other tiles. However, video encoder 200 and video decoder 300 may perform filtering across tile boundaries. Tiles may be uniform or non-uniform in size. Tile-based coding may enable parallel processing and / or multithreading for encoder and decoder implementations.

[0054]

[0075] In some examples, the video encoder 200 and the video decoder 300 may use a single QTBT or MTT structure to represent each of the luminance and chrominance components, and in other examples, the video encoder 200 and the video decoder 300 may use two or more QTBT or MTT structures, such as one QTBT / MTT structure for the luminance component and another QTBT / MTT structure for both chrominance components (or two QTBT / MTT structures for each chrominance component).

[0055]

[0076] Video encoder 200 and video decoder 300 may be configured to use quadtree partitioning, QTBT partitioning, MTT partitioning, superblock partitioning, or other partitioning structures.

[0056]

[0077] In some examples, a CTU includes a coding tree block (CTB) of luma samples, two corresponding CTBs of chroma samples for a picture having three sample arrays, or a CTB of samples for a picture coded using three separate color planes and syntax structures used to code a monochrome picture or samples. A CTB may be an N×N block of samples for some value of N, partitioned to divide the components into CTBs. A component is a single sample from one of the three arrays (luma and two chroma) that make up a picture in 4:2:0, 4:2:2, or 4:4:4 color format, or a single sample from an array or arrays that make up a picture in monochrome format. In some examples, a coding block is an M×N block of samples for some values ​​of M and N, partitioned to divide the CTB into coding blocks.

[0057]

[0078] Blocks (e.g., CTUs or CUs) may be grouped in various ways within a picture. As an example, a brick may refer to a rectangular region of a CTU row within a particular tile within a picture. A tile may be a rectangular region of CTUs within a particular tile column and a particular tile row within a picture. A tile column refers to a rectangular region of CTUs with a height equal to the height of the picture and a width specified by a syntax element (e.g., in a picture parameter set). A tile row refers to a rectangular region of CTUs with a height specified by a syntax element (e.g., in a picture parameter set) and a width equal to the width of the picture.

[0058]

[0079] In some examples, a tile may be partitioned into multiple bricks, each of which may include one or more CTU rows within the tile. A tile that is not partitioned into multiple bricks may also be referred to as a brick. However, a brick that is a true subset of a tile may not be referred to as a tile. Bricks within a picture may also be arranged as slices. A slice may be an integer number of bricks of a picture that may be contained exclusively within a single network abstraction layer (NAL) unit. In some examples, a slice includes either several complete tiles or only a continuous sequence of complete bricks of one tile.

[0059]

[0080] This disclosure may use "N x N" and "N by N," e.g., 16 x 16 samples or 16 by 16 samples, interchangeably to refer to the sample dimensions of a block (such as a CU or other video block) in the vertical and horizontal dimensions. Generally, a 16 x 16 CU has 16 samples in the vertical direction (y = 16) and 16 samples in the horizontal direction (x = 16). Similarly, an N x N CU generally has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer value. Samples within a CU may be arranged in rows and columns. Furthermore, a CU does not necessarily have to have the same number of samples in the horizontal direction as in the vertical direction. For example, a CU may include N x M samples, where M is not necessarily equal to N.

[0060]

[0081] Video encoder 200 encodes video data for a CU that represents prediction and / or residual information and other information. The prediction information indicates how the CU will be predicted to form a predictive block for the CU. The residual information generally represents sample-by-sample differences between the samples of the CU before encoding and the predictive block.

[0061]

[0082] To predict a CU, video encoder 200 may generally form a predictive block for the CU through inter prediction or intra prediction. Inter prediction generally refers to predicting a CU from data of a previously coded picture, and intra prediction generally refers to predicting a CU from previously coded data of the same picture. To perform inter prediction, video encoder 200 may generate a predictive block using one or more motion vectors. Video encoder 200 may generally perform a motion search to identify a reference block that closely matches the CU with respect to the difference between the CU and the reference block, for example. Video encoder 200 may calculate a difference metric using sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute difference (MAD), mean squared differences (MSD), or other such difference calculations to determine whether the reference block closely matches the current CU. In some examples, video encoder 200 may predict the current CU using unidirectional prediction or bidirectional prediction.

[0062]

[0083] Some examples of VVC also provide an affine motion compensation mode, which may be considered an inter-prediction mode. In affine motion compensation mode, video encoder 200 may determine two or more motion vectors that represent non-translational motion, such as zooming in or out, rotation, perspective movement, or other irregular motion types.

[0063]

[0084] To perform intra prediction, video encoder 200 may select an intra prediction mode to generate a predicted block. Some examples of VVC provide 67 intra prediction modes, including various directional modes, as well as a planar mode and a DC mode. Generally, video encoder 200 selects an intra prediction mode that describes neighboring samples for a current block (e.g., a block of a CU) and predicts samples of the current block therefrom. Assuming that video encoder 200 codes CTUs and CUs in raster scan order (left to right, top to bottom), such samples may generally be above, above and to the left, or to the left of the current block in the same picture as the current block.

[0064]

[0085] Video encoder 200 encodes data representing a prediction mode for the current block. For example, in the case of an inter-prediction mode, video encoder 200 may encode data representing which of various available inter-prediction modes is used as well as motion information for the corresponding mode. In the case of unidirectional inter-prediction or bidirectional inter-prediction, for example, video encoder 200 may use an advanced motion vector prediction (AMVP) mode or a merge mode to encode the motion vector. Video encoder 200 may use a similar mode to encode the motion vector for an affine motion compensation mode.

[0065]

[0086] AV1 includes two general techniques for encoding and decoding coding blocks of video data. The two general techniques are intra-prediction (e.g., intra-frame prediction or spatial prediction) and inter-prediction (e.g., inter-frame prediction or temporal prediction). In the context of AV1, when predicting a block of a current frame of video data using an intra-prediction mode, the video encoder 200 and the video decoder 300 do not use video data from other frames of the video data. In most intra-prediction modes, the video encoder 200 encodes the block of the current frame based on the difference between sample values ​​in the current block and predicted values ​​generated from reference samples in the same frame. The video encoder 200 determines the predicted values ​​generated from the reference samples based on the intra-prediction mode.

[0066]

[0087] Following prediction, such as intra- or inter-prediction, of a block, the video encoder 200 may calculate residual data for the block. The residual data, such as a residual block, represents sample-by-sample differences between the block and a prediction block for that block formed using a corresponding prediction mode. The video encoder 200 may apply one or more transforms to the residual block to produce transform data in the transform domain rather than the sample domain. For example, the video encoder 200 may apply a discrete cosine transform (DCT), an integer transform, a wavelet transform, or a conceptually similar transform to the residual video data. In addition, the video encoder 200 may apply a secondary transform, such as a mode-dependent non-separable secondary transform (MDNSST), a signal-dependent transform, or a Karhunen-Loeve transform (KLT), following the initial transform. The video encoder 200 produces transform coefficients following application of the one or more transforms.

[0067]

[0088] As described above, following any transformation that produces transform coefficients, the video encoder 200 may perform quantization of the transform coefficients. Quantization generally refers to a process in which transform coefficients are quantized to possibly reduce the amount of data used to represent the transform coefficients, thereby providing further compression. By performing the quantization process, the video encoder 200 may reduce the bit depth associated with some or all of the transform coefficients. For example, the video encoder 200 may truncate an n-bit value to an m-bit value during quantization, where n is greater than m. In some examples, to perform quantization, the video encoder 200 may perform a bitwise right shift of the value to be quantized.

[0068]

[0089] Following quantization, the video encoder 200 may scan the transform coefficients, creating a one-dimensional vector from a two-dimensional matrix containing the quantized transform coefficients. The scan may be designed to place transform coefficients with higher energy (and therefore lower frequency) at the front of the vector and transform coefficients with lower energy (and therefore higher frequency) at the back of the vector. In some examples, the video encoder 200 may use a predefined scan order to scan the quantized transform coefficients to create a serialized vector and then entropy code the quantized transform coefficients of the vector. In other examples, the video encoder 200 may perform adaptive scanning. After scanning the quantized transform coefficients to form the one-dimensional vector, the video encoder 200 may entropy code the one-dimensional vector, for example, according to context-adaptive binary arithmetic coding (CABAC). The video encoder 200 may also entropy code values ​​for syntax elements that describe metadata associated with the encoded video data for use by the video decoder 300 in decoding the video data.

[0069]

[0090] To implement CABAC, video encoder 200 may assign a context in a context model to a symbol to be transmitted. The context may relate, for example, to whether neighboring values ​​of the symbol are zeroed. A probability determination may be based on the context assigned to the symbol.

[0070]

[0091] Video encoder 200 may further generate syntax data, such as block-based syntax data, picture-based syntax data, and sequence-based syntax data, for example, within a picture header, a block header, a slice header, or other syntax data, such as a sequence parameter set (SPS), a picture parameter set (PPS), or a video parameter set (VPS), to video decoder 300. Video decoder 300 may similarly decode such syntax data to determine how to decode the corresponding video data.

[0071]

[0092] In this manner, video encoder 200 may generate a bitstream including syntax elements that describe coded video data, e.g., partitions of a picture into blocks (e.g., CUs) and prediction and / or residual information for the blocks. Finally, video decoder 300 may receive the bitstream and decode the coded video data.

[0072]

[0093] Generally, video decoder 300 performs a reciprocal process to that performed by video encoder 200 to decode encoded video data of a bitstream. For example, video decoder 300 may decode values ​​for syntax elements of a bitstream using CABAC in a manner that is reciprocal but substantially similar to the CABAC encoding process of video encoder 200. The syntax elements may define partition information for the partition of a picture into CTUs and the partition of each CTU according to a corresponding partition structure, such as a QTBT structure, to define the CUs of the CTU. The syntax elements may further define prediction information and residual information for blocks of video data (e.g., CUs).

[0073]

[0094] The residual information may be represented, for example, by quantized transform coefficients. The video decoder 300 may dequantize and inverse transform the quantized transform coefficients of the block to reconstruct a residual block for the block. The video decoder 300 uses the signaled prediction mode (intra-prediction or inter-prediction) and associated prediction information (e.g., motion information for inter-prediction) to form a predictive block for the block. The video decoder 300 can then combine the predictive block and the residual block (sample by sample) to reconstruct the original block. The video decoder 300 may also perform additional processing, such as performing a deblocking process to reduce visual artifacts along block boundaries.

[0074]

[0095] This disclosure may generally refer to "signaling" certain information, such as a syntax element. The term "signaling" may generally refer to communication of a value for a syntax element and / or other data used to decode the encoded video data. That is, video encoder 200 may signal a value for a syntax element in a bitstream. Generally, signaling refers to generating a value within the bitstream. As mentioned above, source device 102 may forward the bitstream to destination device 116 in substantially real time or non-real time, which may occur, for example, when storing syntax elements in storage device 112 for later retrieval by destination device 116.

[0075]

[0096] According to one or more examples, the present disclosure relates to intra block copy and inter prediction in video codecs. For example, exemplary techniques may relate to deriving a block vector difference candidate list for an IBC MBVD (merge mode with block vector difference) mode, an IBC AMVP (advanced motion vector prediction) mode, and deriving a motion vector difference candidate list for MMVD for an affine MMVD (merge mode with motion vector difference) mode, a GEO (geometric partitioning mode) MMVD mode, or a normal merge mode.

[0076]

[0097] The following describes intra block copy (IBC) merging and advanced motion vector prediction (AMVP) list construction. IBC merge / AMVP list construction may be performed by video encoder 200 and video decoder 300 as follows: An IBC merge / AMVP candidate may be inserted into the IBC merge / AMVP candidate list only if it is valid. The top-right, bottom-left, and top-left spatial candidates, as well as one pairwise average candidate, may be added to the IBC merge / AMVP candidate list. Template-based adaptive sorting (ARMC-TM) is applied to the IBC merge list.

[0077]

[0098] The history-based motion vector predictor (HMVP) table size for IBC can be 25. After a maximum of 20 IBC merge candidates are derived using full pruning, the IBC merge candidates are sorted together. After sorting, the first six candidates with the lowest template matching costs are selected as final candidates in the IBC merge list.

[0078]

[0099] The zero vector candidates for padding the IBC merge / AMVP list are replaced with the set of BVP candidates located in the IBC reference region. In IBC merge mode, zero vectors are invalid as block vectors and are therefore discarded as BVPs in the IBC candidate list.

[0079]

[0100] Three candidates are located at the nearest corners of the reference region, and three additional candidates are determined at the centers of three sub-regions (A, B, and C), whose coordinates are determined by the width and height of the current block and the ΔX and ΔY parameters, as shown in Figure 6. As an example, Figure 6 shows a current block 600 and an IBC buffer 602, as well as various parameters used to construct the IBC merge / AMVP list for the current block 600.

[0080]

[0101] The following describes IBC using template matching. Template matching (TM) is used in IBC for both IBC merge mode and IBC AMVP mode. The IBC-TM merge list is modified compared to that used by the normal IBC merge mode so that candidates are selected according to a pruning method that uses the motion distance between candidates as in the normal TM merge mode. The ending zero motion fill is replaced by motion vectors to the left (-W, 0), up (0, -H), and up-left (-W, -H), where W is the width of the current CU and H is the height of the current CU.

[0081]

[0102] In IBC-TM merge mode, the selected candidate is refined using template matching before the RDO (rate distortion optimization) process or the decoding process. The IBC-TM merge mode is in competition with the regular IBC merge mode, and the TM merge flag is signaled.

[0082]

[0103] In IBC-TM AMVP mode, up to three candidates are selected from the IBC-TM merge list. Each of these three selected candidates is refined using a template matching method and sorted according to their resulting template matching cost. Then, only the first two candidates are considered in the motion estimation process as usual.

[0083]

[0104] Template matching refinement for both IBC-TM merge mode and AMVP mode may be based on the constraint that the IBC motion vector (i) be an integer and (ii) be within a reference region, as shown in FIGS. 7A-7D for current blocks 700A-700D, respectively. For example, FIGS. 7A-7D show IBC reference regions that depend on the position of the current CU for current blocks 700A-700D, respectively. In IBC-TM merge mode, refinement may be performed with integer precision, while in IBC-TM AMVP mode, refinement may be performed with either integer precision or 4-pel precision depending on the AMVR (adaptive motion vector range) value. Such refinement may access only samples without interpolation. In both cases, the refined motion vector and template used at each refinement step may be based on the reference region constraint.

[0084]

[0105] The following describes the IBC reference area. The reference area of ​​the IBC extends to the top two CTU rows. Figure 8 shows the reference area for coding a CTU(m,n) 800. In Figure 8, the hashed blocks indicate the reference area, and the hollow blocks indicate the invalid reference area.

[0085]

[0106] For a CTU(m,n) 800 to be coded, the reference area includes CTUs with indices (m-2,n-2)...(W,n-2), (0,n-1)...(W,n-1), (0,n)...(m,n), where W indicates the maximum horizontal index in the current tile, slice, or picture. If the CTU size is 256, the reference area is limited to one CTU row. This setting may ensure that IBC does not require additional memory when the CTU size is 128 or 256. The per-sample block vector search (also called local search) range may be limited to [-(C<<1),C>>2] horizontally and [-C,C>>2] vertically to accommodate the reference area extension, where C indicates the CTU size.

[0086]

[0107] The following describes adaptive reordering of merge candidates with template matching (ARMC-TM). Merge candidates are adaptively reordered using template matching (TM). The reordering method applies to the regular merge mode, the TM merge mode, and the affine merge mode (potentially excluding SbTMVP candidates). In the TM merge mode, merge candidates are reordered before the refinement process.

[0087]

[0108] The initial merge candidate list is first constructed according to a given check order, such as spatial candidates, TMVP candidates (temporal motion vector predictors), non-adjacent candidates, HMVP candidates, pairwise candidates, and virtual merge candidates. The candidates in the initial list are then divided into several subgroups. For template matching (TM) merge mode and adaptive DMVR (decoder-side motion vector refinement) mode, each merge candidate in the initial list is first refined using TM / multi-pass DMVR. The merge candidates in each subgroup are sorted to generate a sorted merge candidate list, where the sorting is based on a cost value based on template matching. The video encoder 200 signals the index of the selected merge candidate in the sorted merge candidate list to the video decoder 300. For simplicity, merge candidates in the last subgroup (rather than the first) may not be sorted. All zero candidates from the ARMC sorting process are excluded when constructing the merge motion vector candidate list. The subgroup size is set to 5 for normal merge mode and TM merge mode. The subgroup size is set to 3 for affine merge mode.

[0088]

[0109] For cost calculation, the template matching cost of a merge candidate during the reordering process is measured by the SAD between the template samples of the current block and their corresponding reference samples. The template comprises a set of reconstructed samples neighboring the current block. The template reference samples are positioned according to the motion information of the merge candidate. If the merge candidate utilizes bi-prediction, the template reference samples of the merge candidate are also generated by bi-prediction, as shown in FIG. 9.

[0089]

[0110] In Figure 9, current picture 900 includes current block 902. Template 903 includes samples in current picture 900 that neighbor current block 902. Reference block 906 may be from a reference picture in list 0. Template 910 may be a sample that neighbors reference block 906. Reference block 904 may be from a reference picture in list 1. Template 908 may be a sample that neighbors reference block 904.

[0090]

[0111] For the refinement of the initial merge candidate list, if a multi-pass DMVR is used to derive refined motion for the initial merge candidate list, only the first pass (i.e., PU level) of the multi-pass DMVR is applied in the reordering. If template matching is used to derive refined motion, the template size is set equal to 1. If a block is flat with a block width greater than twice its height or narrow with a height greater than twice its width, only the top or left template can be used during TM motion refinement. TM is extended to implement 1 / 16-pel MVD accuracy. In TM merge mode, the first four merge candidates are reordered using the refined motion.

[0091]

[0112] For subblock-based merging candidates with subblock size equal to W×H, the top template comprises several subtemplates with size W×1, and the left template comprises several subtemplates with size 1×H. As shown in Figure 10, motion information of the subblocks in the first row and first column of the current block is used to derive reference samples for each subtemplate. Figure 10 uses the same reference numbers as Figure 9 to indicate the same blocks.

[0092]

[0113] In the reordering process, a candidate is considered redundant if the cost difference between it and its predecessor is less than the lambda value, for example, |D1-D2|<λ, where D1 and D2 are the costs obtained during the first ARMC reordering, and λ is the Lagrangian parameter used in the RD (rate-distortion) criterion at the encoder side.

[0093]

[0114] The algorithm for reordering may be as follows: The video coder (e.g., video encoder 200 or video decoder 300) may determine the minimum cost difference between a candidate and a previous candidate among all candidates in the list. If the minimum cost difference is greater than or equal to λ, the list is deemed sufficiently diverse, and reordering stops. If this minimum cost difference is less than λ (e.g., less than λ), the candidate is deemed redundant, and the candidate is moved to a further position in the list. This further position is the first position where the candidate is sufficiently diverse compared to its previous candidate.

[0094]

[0115] The algorithm may stop after a finite number of iterations (if the minimum cost difference is no worse than λ). This algorithm may be applied to normal, TM, BM, and affine merge modes. A similar algorithm may be applied to merge MMVD and code MVD prediction methods that also use ARMC for reordering.

[0095]

[0116] The value of λ is set to be equal to the λ of the rate-distortion criterion used to select the best merging candidate at the encoder side for the low-delay configuration, and to a value λ corresponding to another QP (Quantization Parameter) for the random access configuration. A set of λ values ​​corresponding to each of the signaled QP offsets is provided in the SPS (Sequence Parameter Set) or in the slice header for QP offsets not present in the SPS.

[0096]

[0117] The ARMC design is also applicable to AMVP mode, in which case AMVP candidates are sorted according to TM cost. For template matching for advanced motion vector prediction (TM-AMVP) mode, an initial AMVP candidate list is constructed, followed by refinement from TM to construct a refined AMVP candidate list. In addition, MVP candidates with TM costs greater than a threshold equal to five times the cost of the first MVP candidate are skipped. In some examples, if wraparound motion compensation is enabled, MV (motion vector) candidates may be clipped to take into account the wraparound offset.

[0097]

[0118] The following describes TM-based reordering for MMVD and affine MMVD. The MMVD offset is extended for both MMVD and affine MMVD modes. As shown in Figure 11, an additional refinement position along a k × π / 8 diagonal angle is added, thus increasing the number of directions from 4 to 16. Figure 11 shows a current sample 1100 and additional refinement positions with anchor positions 1102A-1102D. All possible MMVD refinement positions (16 × 6) for each base candidate are reordered based on the SAD cost between the template (one row above and one column to the left of the current block) for each refinement position and its reference. The top 1 / 8 refinement positions with the smallest template SAD cost are kept as available positions and therefore for MMVD index coding. The MMVD index is binarized by a Rice code with a parameter equal to 2. The affine MMVD reordering is extended, in which an additional refinement position along a k × π / 4 diagonal angle is added. After sorting, the top 1 / 2 refinement positions with the smallest template SAD cost are kept.

[0098]

[0119] The first N motion candidates in the candidate list before reordering are used as base candidates for MMVD and affine MMVD. N is equal to 3 for MMVD, and equal to [1,3] for affine MMVD, depending on the affine flags of neighboring blocks. Depending on whether the offsets of other reference picture lists are mirrored or directly set to zero, two ways of adding MMVD offsets are possible, including "double-sided" and "single-sided". The way in which reordering is applied to a block may depend on the TM cost.

[0099]

[0120] The following describes the geometric partition mode (GPM) (also called GEO with merge motion vector differences (MMVD)). GPM in VVC is extended by applying motion vector refinement to the existing GPM unidirectional MV. A flag is first signaled to the GPM CU to specify whether this mode is used. If this mode is used, each geometric partition of the GPM CU can further decide whether to signal MVD. After GPM merge candidates are selected, if MVD is signaled for the geometric partition, the partition motion is further refined by the signaled MVD information. All other procedures are kept the same as GPM.

[0100]

[0121] MVD is signaled as a distance and direction pair, similar to MMVD. There are nine candidate distances (1 / 4 pel, 1 / 2 pel, 1 pel, 2 pels, 3 pels, 4 pels, 6 pels, 8 pels, 16 pels) and eight candidate directions (4 horizontal / vertical and 4 diagonal) involved in GPM with MMVD (GPM-MMVD). In addition, if pic_fpel_mmvd_enabled_flag is equal to 1, MVD is shifted left by 2, similar to MMVD.

[0101]

[0122] The following describes reconstruction-reordered IBC (RR-IBC). RR-IBC mode is enabled for IBC-coded blocks. When RR-IBC is applied, samples in a reconstructed block are inverted according to the inversion type of the current block (e.g., current blocks 1200 and 1204 in Figures 12A and 12B, respectively). On the encoder side, the video encoder 200 may invert the original block before motion search and residual calculation, while the predictive block is derived without inversion. On the decoder side, the video decoder 300 may invert the reconstructed block to restore the original block.

[0102]

[0123] Two inversion methods, horizontal and vertical inversion, are supported for RR-IBC coded blocks. First, a syntax flag is signaled indicating whether the reconstruction is inverted for an IBC AMVP coded block, and if so, another flag specifying the inversion type is further signaled. In the case of IBC merging, the inversion type is inherited from the neighboring block without syntax signaling. Considering horizontal or vertical symmetry, the current block (e.g., current blocks 1200 and 1204 in Figures 12A and 12B) and the reference block (e.g., reference blocks 1202 and 1206 in Figures 12A and 12B) are usually aligned horizontally or vertically. Therefore, if horizontal inversion is applied, the vertical component of BV (block vector) is not signaled and is inferred to be equal to 0. Similarly, if vertical inversion is applied, the horizontal component of BV is not signaled and is inferred to be equal to 0.

[0103]

[0124] To take advantage of the symmetry property, an inversion-aware BV adjustment technique is applied to refine the block vector candidates. For example, as shown in Figures 12A and 12B, (xnbr, ynbr) and (xcur, ycur) represent the coordinates of the center samples of the neighboring and current blocks 1200 and 1204, respectively, and BVnbr and BVcur represent the BVs of the neighboring and current blocks, respectively. If the neighboring block is coded with horizontal inversion, i.e., BVcurh = 2(xnbr - xcur) + BVnbrh, the horizontal component of BVcur is calculated by adding a motion shift to the horizontal component of BVnbr (denoted as BVnbrh) instead of directly inheriting the BV from the neighboring block. Similarly, if the neighboring block is coded with vertical inversion, i.e., BVcurv = 2(ynbr - ycur) + BVnbrv, the vertical component of BVcur is calculated by adding a motion shift to the vertical component of BVnbr (denoted as BVnbrv).

[0104]

[0125] The following describes the merge mode using block vector differences (IBC-MBVD). Affine-MMVD and GPM-MMVD are adopted in the Extended Compression Model (ECM) as extensions of the regular MMVD mode. It may be possible to extend the MMVD mode to the IBC merge mode.

[0105]

[0126] In IBC-MBVD, the distance sets are {1 pel, 2 pels, 4 pels, 8 pels, 12 pels, 16 pels, 24 pels, 32 pels, 40 pels, 48 ​​pels, 56 pels, 64 pels, 72 pels, 80 pels, 88 pels, 96 pels, 104 pels, 112 pels, 120 pels, 128 pels} and the BVD directions are two horizontal and two vertical directions.

[0106]

[0127] The base candidate is selected from the first five candidates in the sorted IBC merge list. All possible MBVD refinement positions (20 × 4) for each base candidate are sorted based on the SAD cost between the template (one row above and one column to the left of the current block) for each refinement position and its reference. The top eight refinement positions with the lowest template SAD cost are kept as available positions and therefore reserved for MBVD index coding. The MBVD index is binarized by a Rice code with parameter equal to 1. An IBC-MBVD coded block may not inherit the inversion type from an RR-IBC coded neighboring block.

[0107]

[0128] 17 is a conceptual diagram illustrating an example of an Advanced Motion Vector Prediction (AMVP) process. Table 1 contains some example values ​​for coding as part of AMVP.

[0108] [Table 1]

[0109]

[0129] The AMVP process can be as follows: Block vectors (BV) are found by exhaustive search; a BVP (block vector predictor) is selected from the BVP candidate list, and this list is sorted as described above for IBC merge / AMVP list construction; and a block vector difference (BVD) is calculated in the following way: BVD = BV - BVP.

[0110]

[0130] BVD signaling may be as follows: For each horizontal and vertical component, BVD is split into a prefix and a suffix (BVDx=pBVDx+sBVDx, BVDy=pBVDy+sBVDy) (BVDx=pBVDx+sBVDx, BVDy=pBVDy+sBVDy). The first five bins of the prefix are context coded. The suffix is ​​coded using a fixed length code, where the length depends on the prefix value.

[0111]

[0131] Constructing a vector difference candidate list can pose certain challenges. For example, in the current ECM, the IBC merge mode with block vector difference (IBC-MBVD) mode derives a BVD candidate list of eight candidates for each block vector predictor (BVP) by sorting 80 candidates using template matching. The 80 candidates may have distances to the BVPs from a distance set of {1 pel, 2 pels, 4 pels, 8 pels, 12 pels, 16 pels, 24 pels, 32 pels, 40 pels, 48 ​​pels, 56 pels, 64 pels, 72 pels, 80 pels, 88 pels, 96 pels, 104 pels, 112 pels, 120 pels, 128 pels} in two horizontal and two vertical directions. The distance sets may be predefined and prestored. For example, if there are 20 BVPs in the distance set, multiplying the two horizontal and two vertical directions results in 80 candidates (e.g., 20 * 2 * 2) occurs.

[0112]

[0132] For example, video encoder 200 and video decoder 300 may be configured to evaluate a cost value (e.g., a TM cost value) for each of the 80 candidates, but not for the other candidates, i.e., in some current techniques, there may be no dynamic determination of which candidates are evaluated for building the vector difference candidate list.

[0113]

[0133] Increasing the granularity of the distance set and the number of directions can increase coding performance in terms of BD rate. However, sorting all candidates to derive a final BVD candidate list of 8 candidates can result in a significant increase in coding time on both the encoder and decoder sides. As an example, setting a distance set of 128 values ​​{1 pel, 2 pel, 3 pel, 4 pel, ... 127 pel, 128 pel} in 4 directions requires 512 TM SAD calculations (e.g., 128 pels) for each BVP compared to 80 TM SAD calculations for the current ECM. * 4) is necessary.

[0114]

[0134] In other words, the 80 candidates identified above do not include 5-pel, 6-pel, 7-pel, 9-pel, 10-pel, and many others in either direction, although such candidates may be able to provide relatively good coding gain. To include such candidates, the video encoder 200 and the video decoder 300 may be able to evaluate every possible candidate. Again, the candidate may be considered a distance away from the vector predictor (e.g., a 1-pel BVD defines a distance of, for example, 1 pel away from the vector predictor). However, including every candidate may significantly increase processing time.

[0115]

[0135] The current ECM supports an IBC search range that is twice the CTU size. Therefore, the maximum offset of the BVD set can be increased to 256 pels. The above example requires 1024 (e.g., 256) TM SAD calculations for each BVP, compared to the current ECM's 80 TM SAD calculations. * 4) TM SAD calculation is required.

[0116]

[0136] This disclosure describes example techniques for constructing a vector difference candidate list (e.g., a BVD or MVD candidate list) to achieve a good trade-off in terms of coding performance and coding time. For example, video encoder 200 and video decoder 300 may determine a set of vector difference candidates (e.g., a set of BVD or MVD candidates).

[0117]

[0137] As an example, the following is described with respect to block vectors, block vector predictors, and block vector difference candidates. However, the exemplary techniques may be applicable to block vectors or motion vectors. That is, the exemplary techniques with respect to BV (block vector) or BVP (block vector predictor) may be used with respect to MV (motion vector) or MVP (motion vector predictor).

[0118]

[0138] Given a block vector predictor (e.g., BVP) at position (horOrg,verOrg), video encoder 200 and video decoder 300 may construct a BVD candidate list (bvdCandListN) with up to N candidates from a set of BVD candidates (bvdCandsSet) by using TM to select N candidates with the minimum TM cost. Video encoder 200 may signal an index in the bitstream to indicate which candidate is used together with the BVP by video decoder 300 to derive the final BV for the current IBC block.

[0119]

[0139] In one example, bvdCandsSet is predefined to be a set of positions surrounding a BVP position. The predefined set is determined by defining a direction, a maximum offset along the direction, and a spacing between two candidates along the direction. For example, one predefined set could be four directions, two horizontal and two vertical, with a maximum offset of 256 pels, with a spacing of 1 pel along each direction between each of the two BVD candidates. There are a total of 256 x 4 = 1024 BVD candidates in bvdCandsSet.

[0120]

[0140] In one example, bvdCandsSet is all possible positions surrounding the BVP position. In this example, the BVD candidates have an offset of (horBvd,verBvd), where the value of horBvd is in the value range of {-maxRange,+maxRange} and the value of verBvd is in the value range of {-maxRange,+maxRange}. The value of maxRange is either a predefined value, e.g., 256 (as twice the CTU width), or a value signaled in the bitstream as a high-level syntax value.

[0121]

[0141] Thus, in one or more examples, the video encoder 200 and the video decoder 300 may determine a set of vector difference candidates based on a set of positions surrounding the position of the vector predictor. The set of positions may include all positions surrounding the position of the vector predictor or fewer than all positions surrounding the position of the vector predictor. In some examples, the video encoder 200 and the video decoder 300 may utilize this set of vector difference candidates to construct a vector difference candidate list.

[0122]

[0142] In one example, video encoder 200 and video decoder 300 may select a subset of bvdCandsSet(bvdCandsSubSet) and reorder the subset by using TM, where the first N candidates of bvdCandsSubSet with the smallest TM cost are used to derive bvdCandListN. In this example, the total number of candidates in bvdCandsSet is P, the total number of candidates in bvdCandsSubSet is Q, where N is less than Q and Q is less than P. Thus, in some examples, video encoder 200 and video decoder 300 may determine a subset of vector difference candidates from the set of vector difference candidates. In some examples, video encoder 200 and video decoder 300 may utilize this subset of vector difference candidates to construct a vector difference candidate list.

[0123]

[0143] The video encoder 200 and the video decoder 300 may be configured to construct a vector difference candidate list based on a set of vector difference candidates. For example, the video encoder 200 and the video decoder 300 may derive the candidate list from a subset of a predefined set of BVD candidates.

[0124]

[0144] As an example, bvdCandsSet (e.g., a set of vector difference candidates) is a predefined set of T1 directions with a maximum offset of T2 pels along each direction and a spacing of T3 pels between each of two candidates along each direction. In this example, the total number of candidates is P = T1 × T2 ÷ T3.

[0125]

[0145] The derivation (e.g., construction) of bvdCandListN (e.g., vector difference candidate list) may be defined as follows: In step 1, video encoder 200 and video decoder 300 may construct a subset bvdCandsStep1Input by selecting candidates along each direction starting with S1 pel and ending with T2 pel, with a spacing of S1 pel between each two candidates, e.g., {S1 pel, 2×S1 pel, 3×S1 pel, 4×S1 pel, ..., T2 pel}.

[0126]

[0146] For example, the video encoder 200 and the video decoder 300 may determine a first set of vector difference candidates based on a first pel interval (e.g., S1 pel). In this example, the first pel interval (e.g., S1 pel) defines a first distance between consecutive vector difference candidates in the first set of vector difference candidates starting from the vector predictor. For example, if the candidates in the first set of vector difference candidates are S1 pel, 2 * S1 Pel, and 3 * If it is S1 Pell, S1 Pell and 2 * The distance between S1 and S2 is S1. * S1 Pell and 3 *The distance to the S1 pels is also the S1 pels, and the same applies hereinafter. In one or more examples, to determine the first set of vector difference candidates, the video encoder 200 and the video decoder 300 may each add to the first set of vector difference candidates a candidate that is a set of positions surrounding the position of the vector predictor and separated from the vector predictor by an integer multiple of the first distance. For example, the integer multiple can be 1, 2, 3, 4, etc.

[0127]

[0147] The video encoder 200 and the video decoder 300 may add candidates to the first set of vector difference candidates until the range threshold is satisfied. For example, the video encoder 200 and the video decoder 300 may continue to add candidates to the first set of vector difference candidates until the distance from the vector predictor is equal to the T2 pels. That is, the candidates can be from the S1 pels, 2 * S1 pels, 3 * to the T2 pels from the S1 pels.

[0128]

[0148] The video encoder 200 and the video decoder 300 may derive the subset bvdCandsStep1Output by using the TM to sort all the candidates in bvdCandsStep1Input and selecting the first N candidates having the minimum TM cost. In one or more examples, T3 < S1 ≤ T2. The total number of candidates in bvdCandsStep1Input may be equal to T1 × T2 ÷ S1. In some examples, some of the candidates in bvdCandsStep1Input are not valid BVDs, and the total number of TMs is less than or equal to T1 × T2 ÷ S1. An example for determining a valid BVD is described below.

[0129]

[0149] In step 2, the video encoder 200 and the video decoder 300 may determine a second set of vector difference candidates based on the first set of vector difference candidates and the second pel interval. The second pel interval may define a second distance between consecutive vector difference candidates in the second set of vector difference candidates, and the second pel interval may be smaller than the first pel interval. For example, in the above example, if the first pel interval (e.g., S1 pels) is 4 pels, the second pel interval (e.g., S2 pels) may be smaller than the first pel interval (e.g., 1 pel, 2 pels, or 3 pels).

[0130]

[0150] For example, in step 2, video encoder 200 and video decoder 300 may construct a subset bvdCandsSubSetStep2Input by inserting candidates into bvdCandsStep1Output(candStep1Output) and selecting candidates whose distance to candStep1Output is S2 pels, e.g., {candStep1Output-S2, candStep1Output, candStep1Output+S2}, along the direction of candStep1Output. That is, video encoder 200 and video decoder 300 may include at least a subset of the candidates in the first set of vector difference candidates in the second set of vector difference candidates (e.g., one or more candidates in bvdCandsStep1Output may be in the second set of vector difference candidates). Video encoder 200 and video decoder 300 may add candidates among a subset of the candidates in the first set of vector difference candidates based on an integer multiple of the second distance. For example, based on the above example, if candStep1Output is one of the candidates in the first set of vector difference candidates and is also in the second set of vector difference candidates, then candStep1Output-S2 and canStep1Output+S2 will be included in the second set of vector difference candidates and will be between the two candidates in the first set of vector difference candidates.

[0131]

[0151] The video encoder 200 and the video decoder 300 may derive the subset bvdCandsStep2Output by using TM to sort all the candidates in bvdCandsStep2Input and selecting the first N candidates having the minimum TM cost. In one or more examples, T3 ≤ S2 < S1. The total number of candidates in bvdCandsStep2Input may be equal to N × 3. Similar to the above, some candidates in bvdCandsStep2Input may not be valid BVDs, and the total number of TMs is N × 2 or less.

[0132]

[0152] In step X, the video encoder 200 and the video decoder 300 may repeat step 2 by reducing the distance by S(x) pels, for example, when T3 ≤ S(x) < S < S(x - 1). The total number of candidates in bvdCandsStepXInput may be equal to N × 3. The video encoder 200 and the video decoder 300 may derive the subset bvdCandsStepXOutput by using TM to sort all the candidates in bvdCandsStepXInput and selecting the first N candidates having the minimum TM cost. Similar to the above, some candidates in bvdCandsStepXInput may not be valid BVDs, and the total number of TMs is N × 2 or less.

[0133]

[0153] The video encoder 200 and the video decoder 300 may end the process when S(x) is equal to T3 or when an end criterion is reached, for example, when the TM cost of the last candidate among the N candidates is below a threshold (e.g., a cost threshold). bvdCandListN is derived as the final step output subset bvdCandsStepXOutput. That is, the video encoder 200 and the video decoder 300 may add candidates until a range threshold is satisfied (e.g., the distance from the vector predictor becomes equal to T2 pels) or until the cost value associated with the candidate satisfies the cost threshold (e.g., the cost of the candidate is below the cost threshold).

[0134]

[0154] In this way, to construct the vector difference candidate list, video encoder 200 and video decoder 300 may be configured to determine a first subset of the set of vector difference candidates based on one of selecting candidates along each direction using different pels or selecting candidates along each direction using different pels and permutations (e.g., as in exemplary step 1 above). Video encoder 200 and video decoder 300 may iteratively determine one or more additional subsets of vector difference candidates by adding candidates at different pel distances based on a previous subset of the set of vector difference candidates (e.g., as in step 2 and the subsequent iterative process up to step X above). In this example, the final subset of vector difference candidates from the additional subsets is the vector difference candidate list.

[0135]

[0155] As described above, there may be cases where a vector difference candidate is not valid. In one example, a valid BVD is determined by adding the BVD to the BVP to derive the BV, and the BV should be within a search range of the current block, for example, inside a picture boundary and inside a predefined search range of the block vector. In one example, if the BVP is derived from a neighboring block and the neighboring block is coded with a horizontal flip type, the valid BVD has a vertical component value equal to 0. In one example, if the BVP is derived from a neighboring block and the neighboring block is coded with a vertical flip type, the valid BVD has a horizontal component value equal to 0. In one example, if it is determined that the current block is coded with a horizontal flip type, the valid BVD has a vertical component value equal to 0. In one example, if it is determined that the current block is coded with a vertical flip type, the valid BVD has a horizontal component value equal to 0.

[0136]

[0156] Therefore, the video encoder 200 and the video decoder 300 can be configured to determine that a vector difference candidate in the vector difference candidate list is valid based on one or more of the following. The video encoder 200 and the video decoder 300 can determine that when a vector difference candidate is summed with a vector predictor, a vector of a hypothesis within a search range can be obtained.

[0137]

[0157] When the vector predictor is derived from an adjacent block and the adjacent block is coded in a horizontal flip type, the video encoder 200 and the video decoder 300 can determine that the vector difference candidate has a vertical component value equal to 0. When the vector predictor is derived from an adjacent block and the adjacent block is coded in a vertical flip type, the video encoder 200 and the video decoder 300 can determine that the vector difference candidate has a horizontal component value equal to 0.

[0138]

[0158] When the current block is coded in a horizontal flip type, the video encoder 200 and the video decoder 300 can determine that the vector difference candidate has a vertical component value equal to 0. When the current block is coded in a vertical flip type, the video encoder 200 and the video decoder 300 can determine that the vector difference candidate has a horizontal component value equal to 0.

[0139]

[0159] To determine the value of S1, the video encoder 200 and the video decoder 300 can determine the value of S1 based on at least one of the following conditions, that is, T3 < S1 ≤ T2 and the total number of candidates in bvdCandsStep1Input is N or more, and / or S1 = T3 and the total number of candidates in bvdCandsSet is N or less.

[0140]

[0160] In one example, the value of S1 (e.g., the first pel interval) is a pre - defined value. For example, S1 is equal to 32. In one example, the value of S1 is a value signaled in the bitstream as a high - level syntax parameter. In one example, the value of S1 is first determined to be a value K as a pre - defined value or a value signaled in the bitstream, the total number of valid BVD candidates in bvdCandsStep1Input is less than N, and then the value of S1 is set to a value L, where L is smaller than K. The total number of candidates in bvdCandsStep1Input is N or more. In one example, when the value of S1 is set equal to 32 (K) and the total number of candidates in bvdCandsStep1Input is less than N, the video encoder 200 and the video decoder 300 may attempt to set the value of S1 to 16, the total number of candidates in bvdCandsStep1Input is less than N, the video encoder 200 and the video decoder 300 may attempt to set the value of S1 to 8 (L), and the total number of candidates in bvdCandsStep1Input is greater than N.

[0141]

[0161] In one example, the total number of valid BVD candidates in bvdCandsSet is N or less, the value of S1 is determined to be equal to T3, and all of bvdCandsStep1Input are valid candidates in bvdCandsSet.

[0142]

[0162] In one example, the video encoder 200 and the video decoder 300 may determine the value of S(x) based on the values of S1 and T3. For example, the value of S(x) may be within the range of T3 ≤ S(x) < the value of S1.

[0143]

[0163] In one example, the value of S(x) is a function of F(T3, S1). In one example, the value of S(x) is a set of values {S1 / 2, S1 / 4, S1 / 8,...T3}. In one example, the value of S1 is equal to 32, the value of T3 is equal to 1, and the values of S(x) are S2 = 16, S3 = 8, S4 = 4, S5 = 2, S6 = 1. That is, the first pel interval and the second pel interval may define an integer distance or a fractional distance.

[0144]

[0164] In one or more examples, video encoder 200 and video decoder 300 may determine a threshold TH as a function of the width and height of the current block. If the Nth selected candidate has a TM cost that is less than or equal to TH, the bvdCandListN derivation process is terminated. In one example, the value of TH is equal to 1.

[0145]

[0165] In one example, if both the top and left templates are available, the value of TH is F. * In one example, if the top template is available, the value of TH is F * In one example, if the left template is available, the value of TH is F. * where F is a factor, width is the block width in pels, and height is the block height in pels. At any step of the bvdCandListN derivation (e.g., construction) process, if an early termination criterion is met, video encoder 200 and video decoder 300 may terminate the bvdCandListN derivation process.

[0146]

[0166] Thus, in one or more examples, the video encoder 200 and the video decoder 300 may determine a threshold size (e.g., TH above) of the set of vector difference candidates based on the block size of the current block. To determine the set of vector difference candidates, the video encoder 200 and the video decoder 300 may determine the set of vector difference candidates until the number of vector difference candidates is equal to the threshold size.

[0147]

[0167] The following is an example technique that the video encoder 200 and the video decoder 300 may implement to construct a vector difference candidate list based on a set of vector difference candidates.

[0148]

[0168] In this example, bvdCandsSet is a predefined set of two horizontal and two vertical (T1=4) pixels with a maximum offset equal to 256 pels (T2=256) and a spacing of 1 pel (T3=1). The total number of candidates in bvdCandsSet is equal to P=T1×T2÷T3=4×256÷1=1024.

[0149]

[0169] The construction of bvdCandListN (e.g., a vector difference candidate list) is as follows. In step 1, the video encoder 200 and the video decoder 300 may construct the subset bvdCandsStep1Input by selecting candidates along each direction, starting at 8 pels and ending at 256 pels, with an interval of 8 pels (S1=8) between each two candidates. That is, in this example, the video encoder 200 and the video decoder 300 may determine a first set of vector difference candidates based on a first pel interval (e.g., an 8-pel interval). The first pel interval defines a first distance between consecutive vector difference candidates in the first set of vector difference candidates starting from the vector predictor. For example, along each direction, the candidate distances to the BVP (i.e., the vector predictor) are {8 pels, 16 pels, 24 pels, 32 pels, ..., 248 pels, 256 pels}.

[0150]

[0170] For example, video encoder 200 and video decoder 300 may add candidates that are a set of positions that each surround the position of the vector predictor and are an integer multiple of the first distance away from the vector predictor to the first set of vector difference candidates (e.g., bvdCandsStep1Input). * 8 Pell, 2 * 8 per (16 per), 3 *In some examples, the video encoder 200 and the video decoder 300 may add candidates to the first set of vector difference candidates until a range threshold is met (e.g., from a BVP to a BVD that is 256 pels away) and / or until one candidate in the first set of vector difference candidates meets a cost threshold (e.g., the TM cost of the candidate is less than or equal to the cost threshold).

[0151]

[0171] The total number of TMs is equal to or less than T1 × T2 ÷ S1 = 4 × 256 ÷ 32 = 32. Video encoder 200 and video decoder 300 may derive subset bvdCandsStep1Output by sorting all candidates in bvdCandsStep1Input and selecting the first N = 8 candidates with the smallest TM costs. In Figure 13, the candidates for bvdCandsStep1Output are (-56, 0), (32, 0), (40, 0), (48, 0), (0, -48), (0, -8), (0, 8), and (0, 16). For example, video encoder 200 and video decoder 300 may determine each cost value associated with the candidates in the first set of vector difference candidates and construct a first subset of vector difference candidates based on the respective cost values.

[0152]

[0172] For example, in this example, to determine each cost value associated with a candidate in the first set of vector difference candidates, video encoder 200 and video decoder 300 may determine a first cost value based on template matching between the reference template for the vector and the template of the current block equal to the BVP plus or minus 8 pels (e.g., based on samples above and to the left of the block pointed to by the vector), determine a second cost value based on template matching between the reference template for the vector and the template of the current block equal to the BVP plus or minus 16 pels (e.g., based on samples above and to the left of the current block), and so on. Video encoder 200 and video decoder 300 may construct a first subset of vector difference candidates based on their respective cost values ​​(e.g., determine the eight BVDs with the lowest cost values). An example of the first subset of vector difference candidates is bvdCandsStep1Output.

[0153]

[0173] In step 2, video encoder 200 and video decoder 300 may construct subset bvdCandsSubSetStep2Input by inserting candidates into bvdCandsStep1Output(candStep1Output) and selecting candidates along the direction of candStep1Output that are 4 pels (S2=4) away from candStep1Output. The total number of TMs is less than or equal to N×2=8×2=16. Video encoder 200 and video decoder 300 may derive subset bvdCandsStep2Output by using the TM to sort all candidates in bvdCandsStep2Input and select the first N=8 candidates with the smallest TM cost. In Figure 14, the candidates for bvdCandsStep2Input are (-60,0), (-56,0), (-52,0), (28,0), (32,0), (36,0), (40,0), (44,0), (48,0), (52,0), (0,-52), (0,-48), (0,-44), (0,-12), (0,-8), (0,-4), (0,4), (0,8), (0,12), (0,16), and (0,20). The candidates for bvdCandsStep2Output are (28,0), (32,0), (36,0), (0,-12), (0,-8), (0,-4), (0,4), and (0,8).

[0154]

[0174] For example, the video encoder 200 and the video decoder 300 may determine a second set of vector difference candidates (e.g., bvdCandsStep2Input) based on the first set of vector difference candidates and the second pel spacing. The second pel spacing may define a second distance between consecutive vector difference candidates in the second set of vector difference candidates, and the second pel spacing may be smaller than the first pel spacing. For example, in this example, the second pel spacing is 4 pels. Thus, there is a 4-pel distance between candidates in the second set of vector difference candidates (e.g., there is a 4-pel distance between (28,0) and (32,0), there is a 4-pel distance between (32,0) and (36,0), etc.).

[0155]

[0175] The second set of vector difference candidates (e.g., bvdCandsStep2Input) may include at least a subset of the candidates in the first set of vector difference candidates (e.g., bvdCandsStep1Input or bvdCandsStep1Output). As an example, (0, -48) is present in both bvdCandStep1Input or bvdCandStep1Output (e.g., the first set of vector difference candidates) and bvdCandStep2Input (e.g., the second set of vector difference candidates).

[0156]

[0176] In some examples, the video encoder 200 and the video decoder 300 may determine respective cost values ​​associated with the candidates in the second set of vector difference candidates (e.g., the cost of the candidates in bvdCandStep2Input). The video encoder 200 and the video decoder 300 may construct a vector difference candidate list based on the respective cost values ​​(e.g., starting from the lowest cost to the highest cost).

[0157]

[0177] In an example of a two-step algorithm for determining the vector difference candidate list, video encoder 200 and video decoder 300 may stop after two steps. That is, the vector difference candidate list may be equal to bvdCandStep2Output. This two-step algorithm is described in more detail below, such as where the second pel interval is smaller, allowing for more granularity. However, in some examples, video encoder 200 and video decoder 300 may proceed with more than two steps, iteratively reducing the size of the pel interval until a pel interval threshold is reached.

[0158]

[0178] The following describes an example of iteratively reducing the size of the pel interval to determine the vector difference candidate list using steps 3 and 4. In step 3, video encoder 200 and video decoder 300 may construct the subset bvdCandsSubSetStep3Input by inserting candidates into bvdCandsStep2Output(candStep2Output) and selecting candidates along the direction of candStep2Output that are two pels (S3=2) away from candStep2Output. The total number of TMs is less than or equal to N×2=8×2=16. Video encoder 200 and video decoder 300 may derive the subset bvdCandsStep3Output by using the TM to sort all candidates in bvdCandsStep3Input and select the first N=8 candidates with the smallest TM cost. In Figure 15, the candidates for bvdCandsStep3Input are (26,0), (28,10), (30,0), (32,0), (34,0), (36,0), (38,0), (0,-14), (0,-12), (0,-10), (0,-8), (0,-6), (0,-4), (0,-2), (0,2), (0,4), (0,6), (0,8), and (0,10). The candidates for bvdCandsStep2Output are (26,0), (28,0), (0,-4), (0,-2), (0,2), (0,4), (0,6), and (0,8).

[0159]

[0179] In step 4, video encoder 200 and video decoder 300 may construct subset bvdCandsSubSetStep4Input by inserting candidates into bvdCandsStep3Output (candStep3Output) and selecting candidates whose distance to candStep1Output is 1 pel (S4=1) along the direction of candStep1Output. The total number of TMs is less than or equal to N×2=8×2=16. Video encoder 200 and video decoder 300 may derive subset bvdCandsStep4Output by using the TM to sort all candidates in bvdCandsStep4Input and select the first N=8 candidates with the smallest TM cost. The candidates in bvdCandListN are set as candidates in bvdCandsStep4Output. In Figure 16, the candidates for bvdCandsStep4Input are (25,0), (26,0), (27,0), (28,0), (29,0), (0,-5), (0,-4), (0,-3), (0,-2), (0,-1), (0,1), (0,2), (0,3), (0,4), (0,5), (0,6), (0,7), (0,8), and (0,9). The candidates for bvdCandsStep4Output(bvdCandListN) are (25,0), (26,0), (27,0), (0,-4), (0,-3), (0,1), (0,5), and (0,9). That is, the vector difference candidate list may include (25,0), (26,0), (27,0), (0,-4), (0,-3), (0,1), (0,5), and (0,9).

[0160]

[0180] The following describes an example of deriving a candidate list from a subset of a predefined set of BVD candidates. An exemplary technique may be a two-step derivation process (e.g., a two-step algorithm).

[0161]

[0181] In some examples, the candidate list derivation process is the same as the method described above. For example, the first step is the same as the method described above with respect to deriving the candidate list, and bvdCandsSet is a predefined set in the T1 direction, having a maximum offset of T2 pels along each direction and having an interval of T3 pels between each of the two candidates along each direction, forming a subset of the predefined set of BVD candidates. However, to reduce process latency, the derivation can be performed in two steps as follows.

[0162]

[0182] As step 1, the video encoder 200 and the video decoder 300 select candidates along each direction such that it starts at S1 pels and ends at T2 pels, with an interval of S1 pels between each of the two candidates, for example, like {S1 pels, 2×S1 pels, 3×S1 pels, 4×S1 pels,..., T2 pels}, to constitute the subset bvdCandsStep1Input. The video encoder 200 and the video decoder 300 may derive the subset bvdCandsStep1Output by sorting all the candidates in bvdCandsStep1Input and selecting the first N candidates having the minimum TM cost, where T3 < S1 ≤ T2. The total number of candidates in bvdCandsStep1Input may be equal to T1×T2÷S1. In some cases, some candidates in bvdCandsStep1Input may not be valid BVDs, and the total number of TMs is less than or equal to T1×T2÷S1.

[0163]

[0183] That is, similar to the above, video encoder 200 and video decoder 300 may determine a first set of vector difference candidates (e.g., bvdCandSetup1Input) based on a first pel interval (e.g., S1). The first pel interval may define a first distance between consecutive vector difference candidates in the first set of vector difference candidates starting from the vector predictor. For example, video encoder 200 and video decoder 300 may determine a first set of vector difference candidates each surrounding the position of the vector predictor (e.g., BVP) and separated from the vector predictor by an integer multiple of the first distance (e.g., S1 pel, 2 * S1 Pell, 3 * Candidates that are sets of positions (e.g., S1 pels) may be added to a first set of vector difference candidates (e.g., bvdCandSetup1Input). Video encoder 200 and video decoder 300 may add candidates until a range threshold is met (e.g., until T2 pels are reached) or until a cost value associated with one of the candidates in the first set of vector difference candidates meets a cost threshold (e.g., the cost value is less than or equal to the cost threshold).

[0164]

[0184] The video encoder 200 and the video decoder 300 may determine a second set of vector difference candidates based on the first set of vector difference candidates and the second pel interval. The second pel interval may define a second distance between consecutive vector difference candidates in the second set of vector difference candidates, and the second pel interval may be smaller than the first pel interval. For example, the video encoder 200 and the video decoder 300 may determine respective cost values ​​associated with the candidates in the first set of vector difference candidates and construct a first subset of vector difference candidates (e.g., bvdCandsStep1Output) based on the respective cost values. In this example, the video encoder 200 and the video decoder 300 may determine the second set of vector difference candidates based on the first subset of vector difference candidates.

[0165]

[0185] As step 2, the video encoder 200 and the video decoder 300 may insert candidates into bvdCandsStep1Output (candStep1Output), and select candidates whose distance to candStep1Output is within S2 pels, for example, {candStep1Output - S2, candStep1Output - (S2 - 1), candStep1Output - (S2 - 2), ···, candStep1Output, candStep1Output + (S2 + 1), candStep1Output + (S2 + 2), ···, candStep1Output + S2}, along the direction of candStep1Output, to constitute the subset bvdCandsSubSetStep2Input. The video encoder 200 and the video decoder 300 may use TM to sort all candidates in bvdCandsStep2Input and select the first N candidates with the minimum TM cost to derive the subset bvdCandsStep2Output, where T3 ≤ S2 < S1. The total number of candidates in bvdCandsStep2Input is equal to N×(2×S2 + 1). In some cases, some candidates in bvdCandsStep2Input may not be valid BVDs, and the total number of TMs is no more than N×(2×S2). bvdCandListN is derived as the final step output subset bvdCandsStep2Output.

[0166]

[0186] For example, the video encoder 200 and the video decoder 300 may include at least a subset of candidates of the first set of vector difference candidates in the second set of vector difference candidates to determine the second set of vector difference candidates (for example, may include candidates from bvdCandsStep2Output). The video encoder 200 and the video decoder 300 may add candidates between subsets of candidates of the first set of vector difference candidates based on an integer multiple of the second distance. For example, the second distance may be S2, S2 - 1, S2 - 2, S2 - 3.

[0167]

[0187] In this example, the second pel interval defining the second distance between consecutive vector difference candidates in the second set of vector difference candidates may be considered to be a 1-pel interval defining a 1-pel distance. In an alternative approach, the second pel interval may be considered to be an S2 (e.g., S2-1, S2-2, etc.) defining the second distance. In some examples, the video encoder 200 and the video decoder 300 may add candidates between the subset of candidates until the cost values ​​associated with the candidates in the second set of vector difference candidates meet a cost threshold. For example, in these examples, the second distance is less than or equal to the second pel interval (e.g., less than or equal to S2).

[0168]

[0188] The following is an example of list derivation in two steps. In this example, bvdCandsSet is a predefined set of two horizontal and two vertical directions (T1=4), with a maximum offset equal to 256 pels (T2=256) and a spacing of 1 pel (T3=1). The total number of candidates in bvdCandsSet is equal to P=T1×T2÷T3=4×256÷1=1024.

[0169]

[0189] The derivation of bvdCandListN may be as follows. As step 1, video encoder 200 and video decoder 300 may construct subset bvdCandsStep1Input by selecting candidates along each direction, starting at 8 pels and ending at 256 pels, with an interval of 8 pels (S1=8) between each two candidates. Along each direction, the candidate distance to the BVP is {8 pels, 16 pels, 24 pels, 32 pels, ..., 248 pels, 256 pels}. The total number of TMs is less than or equal to T1×T2÷S1=4×256÷32=32. Video encoder 200 and video decoder 300 may derive subset bvdCandsStep1Output by using the TM to sort all candidates in bvdCandsStep1Input and select the first N=8 candidates with the smallest TM cost. The candidates for bvdCandsStep1Output are (-56,0), (32,0), (40,0), (48,0), (0,-48), (0,-8), (0,8), and (0,16).

[0170]

[0190] For example, the video encoder 200 and the video decoder 300 may determine a first set of vector difference candidates (e.g., bvdCandsStep1Input) based on a first pel interval (e.g., 8 pels). The first pel interval defines a first distance between consecutive vector difference candidates in the first set of vector difference candidates starting from the vector predictor. That is, the BVD of the vector predictor may be considered to be (0,0), and the bvdCandStep1Input starts from the vector predictor (e.g., (0,8), (0,16), etc., including (8,0) and (16,0)) until a range threshold is met (e.g., 256 pels) or a cost threshold is met. For example, the video encoder 200 and the video decoder 300 may determine a first set of vector difference candidates (e.g., bvdCandsStep1Input) that each surround the position of the vector predictor and are separated from the vector predictor by an integer multiple of the first distance (e.g., 8 pels, 256 pels, etc.). * 8 Pell, 3 * A candidate set of positions (e.g., 8 pels) may be added to the first set of vector difference candidates.

[0171]

[0191] Video encoder 200 and video decoder 300 may determine respective cost values ​​associated with candidates in the first set of vector difference candidates (e.g., TM cost values ​​based on the SAD of a reference template around the reference block pointed to by the vector and a current template around the current block). Video encoder 200 and video decoder 300 may construct a first subset of vector difference candidates (e.g., bvdCandsStep1Output) based on the respective cost values.

[0172]

[0192] As step 2, video encoder 200 and video decoder 300 may construct subset bvdCandsSubSetStep2Input by inserting candidates into bvdCandsStep1Output(candStep1Output) and selecting candidates along the direction of candStep1Output whose distance to candStep1Output is 4 pels or less (S2=4). The total number of TMs is less than or equal to N×(4×2)=8×(4×2)=64. Video encoder 200 and video decoder 300 may derive subset bvdCandsStep2Output by using the TM to sort all candidates in bvdCandsStep2Input and select the first N=8 candidates with the smallest TM cost. The candidates for bvdCandsStep2Input are (-60,0), (-59,0), (-58,0), (-57,0), (-56,0), (-55,0), (-54,0), (-53,0), (-52,0), (28,0), (29,0), (30,0), (31,0), (32,0), ( 33,0), (34,0), (35,0), (36,0), (37,0), (38,0), (39,0), (40,0), (41,0), (42,0), (43,0), (44,0), (45,0), (46,0), (47,0), (48,0), (49,0), (50,0), (51,0) , (52,0), (0,-52), (0,-51), (0,-50), (0,-49), (0,-48), (0,-47), (0,-46), (0,-45), (0,-44), (0,-12), (0,-11), (0,-10), (0,-9), (0,-8), (0,-7), (0,-6) , (0,-5), (0,-4), (0,4), (0,5), (0,6), (0,7), (0,8), (0,9), (0,10), (0,11), (0,12), (0,13), (0,14), (0,15), (0,16), (0,17), (0,18), (0,19), and (0,20). The candidates for bvdCandsStep2Output are (28,0), (32,0), (36,0), (0,-12), (0,-8), (0,-4), (0,4), and (0,8).The candidates in bvdCandsStep2Output are used as the final candidate list bvdCandListN.

[0173]

[0193] For example, the video encoder 200 and the video decoder 300 may determine a second set of vector difference candidates (e.g., bvdCandStep1Input or bvdCandsStep1Output, where bvdCandsStep1Output is based on bvdCandStep1Input) based on the second pel spacing. As an example, the video encoder 200 and the video decoder 300 may determine respective cost values ​​associated with the candidates in the first set of vector difference candidates and construct a first subset of vector difference candidates (e.g., bvdCandsStep1Output) based on the respective cost values. The video encoder 200 and the video decoder 300 may determine a second set of vector difference candidates (e.g., bvdCandsStep2Output) based on the first subset of vector difference candidates (e.g., bvdCandsStep1Output).

[0174]

[0194] The second pel interval may define a second distance between consecutive vector difference candidates in the second set of vector difference candidates, and the second pel interval may be smaller than the first pel interval. For example, in the above example, the second pel interval may be considered to be 1 pel defining a 1-pel distance, or may be considered to be 4 pels, and the second distance (e.g., 1 pel) is less than or equal to the second pel interval (e.g., 4 pels).

[0175]

[0195] In either example, video encoder 200 and video decoder 300 may include at least a subset of the candidates in the first set of vector difference candidates (e.g., bvdCandsStep1Output) in the second set of vector difference candidates, and may include an integer multiple of the second distance (e.g., 1 pel, 2 * 1 Pell, 3 *The video encoder 200 and the video decoder 300 may determine a second set of vector difference candidates, such as by adding candidates between a subset of candidates in the first set of vector difference candidates based on a distance (e.g., a fraction of a pel). In some examples, the distance may be a fraction of a pel. The video encoder 200 and the video decoder 300 may add candidates to the second set of vector difference candidates until a cost value associated with a candidate in the second set of vector difference candidates meets a cost threshold.

[0176]

[0196] The above exemplary techniques may also be used for motion vectors. For example, video encoder 200 and video decoder 300 may utilize the above exemplary techniques for inter MMVD candidate lists. Depending on the determined Adaptive Motion Vector Resolution (AMVR) parameters of the current block, the maximum offset T2, the minimum spacing T3, and S(x) are determined accordingly.

[0177]

[0197] In one example, when the AMVR is set to ¼ pel resolution, T3 is set to a fractional pel, e.g., ¼ pel, T2 is set to 8 pels, and S(x) starts at 1 pel and is reduced to ¼ pel. In one example, when the AMVR is set to ½ pel resolution, T3 is set to ½ pel, T2 is set to 16 pels, and S(x) starts at 2 pels and is reduced to ½ pel. In one example, when the AMVR is set to 1 pel resolution, T3 is set to 1 pel, T2 is set to 32 pels, and S(x) starts at 4 pels and is reduced to 1 pel. In one example, when the AMVR is set to 4 pel resolution, T3 is set to 4 pels, T2 is set to 128 pels, and S(x) starts at 8 pels and is reduced to 4 pels.

[0178]

[0198] The following describes examples of IBC AMVP BVD signaling improvements. Figure 18 is a conceptual diagram illustrating another example of the AMVP process using a template search starting from a prefix vector. Figure 18 and below describe one or more examples for deriving a suffix BVD for IBC AMVP mode, e.g., using a template search around a prefix vector with one validation flag. Table 2 includes some example values ​​for coding as part of AMVP.

[0179] [Table 2]

[0180]

[0199] In one or more examples, BvdTplValidationFlag may indicate whether suffixBVD is signaled or derived using a template-based search. Video encoder 200 may signal, and video decoder 300 may parse, prefixBvd from the bitstream. In one or more examples, video encoder 200 and video decoder 300 may derive the search area by finding nextPrefixBVD=prefixBVD+maxPossibleSuffix(prefixBVD)+1. The search area is horizontally from prefixBVDx to nextPrefixBVDx and vertically from prefixBVDy to nextPrefixBVDy.

[0181]

[0200] In some examples, video encoder 200 and video decoder 300 may perform the search in a way that all candidates within a search area are tested. *There may be (nextPrefixBVDy-prefixBVDy). The list of all tested candidates is sorted, and one candidate with the best cost is selected. Video encoder 200 and video decoder 300 may derive BVD=prefixBVD+bvdTplSearchCand0 from the best candidate position.

[0182]

[0201] If BvdTplValidationFlag is equal to 0, video encoder 200 can signal, and video decoder 300 can parse, suffixBVD from the bitstream, and BVD=prefixBVD+suffixBVD.

[0183]

[0202] The following describes a template search using a subset of the search area around the prefix vector with one validation flag and candidate index signaling. In this example, the search area is Nh * There may be limited to Nv possible positions, the number of which (e.g., the values ​​of Nh and Nv) may depend on the values ​​of prefixBVDx and prefixBVDy, respectively: the distance between horizontal positions d(x, x+1) = (nextPrefixBVDx - prefixBVDx) / Nh, and the distance between vertical positions d(y, y+1) = (nextPrefixBVDy - prefixBVDy) / Nv.

[0184]

[0203] Video encoder 200 and video decoder 300 may select the N lowest TM cost suffixBVD candidates and list them ordered by TM cost. Video encoder 200 may signal, and video decoder 300 may receive, a candidate index in the bitstream that indicates which suffixBVD to use.

[0185]

[0204] If N is equal to 1, the suffixBVD is derived as the suffixBVD candidate with the lowest TM cost within the determined search area.

[0186]

[0205] The following describes the template search around the prefix vector with one validation flag and candidate index signaling: For coding, Table 3 contains some example values.

[0187] [Table 3]

[0188]

[0206] In this example, the process of deriving the search area and candidates to be tested is the same as that described above for the template search around the prefix vector with one validation flag, with one difference being that video encoder 200 can signal, and video decoder 300 can parse, bvdTplSearchCandIdx from the bitstream.

[0189]

[0207] Video encoder 200 and video decoder 300 may select the N lowest TM cost suffixBVD candidates and list them ordered by TM cost. Video encoder 200 may signal, and video decoder 300 may parse, a candidate index in the bitstream that indicates which suffixBVD to use.

[0190]

[0208] If N is equal to 1, the suffixBVD is derived as the suffixBVD candidate with the lowest TM cost within the determined search area.

[0191]

[0209] In some examples, after sorting the candidate list, video encoder 200 and video decoder 300 may use candidates with bvdTplSearchCandIdx for BVD derivation: BVD=prefixBVD+bvdTplSearchCand[bvdTplSearchCandIdx]. The maximum value of bvdTplSearchCandIdx is predefined and controlled by maxNumberOfTplCands.

[0192]

[0210] An example of BVD suffix search is described below. In step 1, PrefixBVD is parsed from the bitstream as PrefixBVD={-14,62}. In step 2, the validation flag BvdTplValidationFlag=True is parsed. In step 3, nextPrefixBVD is derived as ={-30,126}.

[0193]

[0211] As step 4, a possible 1-pel search is as follows:

[0194] [Table 4]

[0195]

[0212] In step 5, the video encoder 200 and the video decoder 300 may sort the list tplCandList by cost value and take the best candidate bvdTplSearchCand0={-4,3}. In step 6, BVD is decoded to =prefixBVD+bvdTplSearchCand0={-18,65}.

[0196]

[0213] The following is an example of a BVD suffix search using N=1 candidates with the lowest cost: As step 1, video encoder 200 can signal, and video decoder 300 can parse, that PrefixBVD={2,6} from the bitstream PrefixBVD. As step 2, video encoder 200 can signal, and video decoder 300 can parse, that validation flag BvdTplValidationFlag=True.

[0197]

[0214] In step 3, maxNumberOfTplCands is set to 1, and parsing of bvdTplSearchCandIdx may not be necessary, and bvdTplSearchCandIdx = 0. In step 4, nextPrefixBVD is derived as = {6, 14}.

[0198]

[0215] Step 5 could be a 1-pel search like this:

[0199] [Table 5]

[0200]

[0216] In step 6, the video encoder 200 and the video decoder 300 may sort the list tplCandList by cost value and take the best candidate bvdTplSearchCand[bvdTplSearchCandIdx=0]={1,0}. In step 7, BVD is decoded to =prefixBVD+bvdTplSearchCand0={3,6}.

[0201]

[0217] The following is an example of a BVD suffix search using N=5 candidates with the lowest cost. As step 1, video encoder 200 may determine, and video decoder 300 may parse, that PrefixBVD={2,2} from the bitstream PrefixBVD. As step 2, video encoder 200 may signal, and video decoder 300 may parse, that validation flag BvdTplValidationFlag=True. As step 3, maxNumberOfTplCands is set to 5. As step 4, video encoder 200 may signal, and video decoder 300 may parse, that bvdTplSearchCandIdx=2. As step 5, nextPrefixBVD is derived as ={6,6}.

[0202]

[0218] Step 6 could be a 1-pel search like this:

[0203] [Table 6]

[0204]

[0219] As step 7, video encoder 200 and video decoder 300 may sort list tplCandList by cost value sortedTplCandList=[{0,1},{1,0},{2,2},{1,1},{0,0},{1,2},{2,1},{3,1},{3,2},{2,3},{3,0},{0,3},{0,2},{2,0},{3,3},{1,3}]. As step 8, bvdTplSearchCand=sortedTplCandList[bvdTplSearchCandIdx]=sortedTplCandList[2]={2,2}. As step 9: BVD is decoded=prefixBVD+bvdTplSearchCand={4,4}.

[0205]

[0220] 2 is a block diagram illustrating an example video encoder 200 that may implement the techniques of this disclosure. Figure 2 is provided for purposes of explanation and should not be considered limiting of the techniques broadly illustrated and described in this disclosure. For purposes of explanation, this disclosure describes video encoder 200 in accordance with VVC and HEVC techniques. However, the techniques of this disclosure may be implemented by video encoding devices configured for other video coding standards and video coding formats, such as AV1 and successors to the AV1 video coding format.

[0206]

[0221] 2, video encoder 200 includes video data memory 230, mode select unit 202, residual generation unit 204, transform processing unit 206, quantization unit 208, inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, filter unit 216, decoded picture buffer (DPB) 218, and entropy coding unit 220. Any or all of video data memory 230, mode select unit 202, residual generation unit 204, transform processing unit 206, quantization unit 208, inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, filter unit 216, DPB 218, and entropy coding unit 220 may be implemented in one or more processors or processing circuitry. For example, the units of video encoder 200 may be implemented as one or more circuits or logic elements as part of hardware circuitry, or as part of a processor, ASIC, or FPGA. Moreover, video encoder 200 may include additional or alternative processors or processing circuitry that perform these and other functions.

[0207]

[0222] Video data memory 230 may store video data to be encoded by components of video encoder 200. Video encoder 200 may receive video data stored in video data memory 230, for example, from video source 104 (FIG. 1). DPB 218 may function as a reference picture memory that stores reference video data for use in predicting subsequent video data by video encoder 200. Video data memory 230 and DPB 218 may be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous dynamic random access memory (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. Video data memory 230 and DPB 218 may be provided by the same memory device or separate memory devices. In various examples, video data memory 230 may be on-chip with other components of video encoder 200, as shown, or may be off-chip relative to those components.

[0208]

[0223] In this disclosure, references to video data memory 230 should not be construed as limited to memory internal to video encoder 200 unless specifically stated so, or to memory external to video encoder 200 unless specifically stated so. Rather, references to video data memory 230 should be understood as a reference memory that stores video data that video encoder 200 receives for encoding (e.g., video data for a current block to be encoded). Memory 106 of FIG. 1 may also provide temporary storage of outputs from various units of video encoder 200.

[0209]

[0224] The various units in FIG. 2 are shown to aid in understanding the operations performed by video encoder 200. The units may be implemented as fixed-function circuits, programmable circuits, or a combination thereof. A fixed-function circuit refers to a circuit that provides a specific function, and the operations that may be performed are predefined. A programmable circuit refers to a circuit that may be programmed to perform various tasks, and provides flexibility in the operations that may be performed. For example, a programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by the software or firmware instructions. While a fixed-function circuit may execute software instructions (e.g., receive parameters or output parameters), the types of operations that the fixed-function circuit performs are generally invariant. In some examples, one or more of the units may be different circuit blocks (fixed function or programmable), and in some examples, one or more of the units may be an integrated circuit.

[0210]

[0225] Video encoder 200 may include arithmetic logic units (ALUs), elementary function units (EFUs), digital circuits, analog circuits, and / or a programmable core formed from programmable circuits. In examples in which the operations of video encoder 200 are implemented using software executed by programmable circuits, memory 106 (FIG. 1) may store software instructions (e.g., object code) that video encoder 200 receives and executes, or a separate memory (not shown) within video encoder 200 may store such instructions.

[0211]

[0226] The video data memory 230 is configured to store the received video data. The video encoder 200 may retrieve pictures of the video data from the video data memory 230 and provide the video data to the residual generation unit 204 and the mode selection unit 202. The video data in the video data memory 230 may be raw video data to be encoded.

[0212]

[0227] The mode select unit 202 includes a motion estimation unit 222, a motion compensation unit 224, and an intra prediction unit 226. The mode select unit 202 may include additional functional units that perform video prediction according to other prediction modes. By way of example, the mode select unit 202 may include a palette unit, an intra block copy unit (which may be part of the motion estimation unit 222 and / or the motion compensation unit 224), an affine unit, a linear model (LM) unit, etc.

[0213]

[0228] The mode selection unit 202 generally coordinates multiple coding passes to test combinations of coding parameters and the resulting rate-distortion values ​​for such combinations. The coding parameters may include partitioning of CTUs into CUs, prediction modes for CUs, transform types for residual data of CUs, quantization parameters for residual data of CUs, etc. The mode selection unit 202 may ultimately select a combination of coding parameters that has a better rate-distortion value than the other tested combinations.

[0214]

[0229] Video encoder 200 may partition a picture retrieved from video data memory 230 into a series of CTUs and encapsulate one or more CTUs within a slice. Mode select unit 202 may partition the CTUs of a picture according to a tree structure, such as the MTT structure, QTBT structure, superblock structure, or quadtree structure described above. As described above, video encoder 200 may form one or more CUs from partitioning the CTUs according to the tree structure. Such CUs may also be generally referred to as "video blocks" or "blocks."

[0215]

[0230] In general, the mode select unit 202 also controls its components (e.g., the motion estimation unit 222, the motion compensation unit 224, and the intra prediction unit 226) to generate a prediction block for a current block (e.g., the current CU, or in HEVC, the overlapping portion of the PU and TU). In the case of inter prediction of the current block, the motion estimation unit 222 may perform a motion search to identify one or more closely matching reference blocks among one or more reference pictures (e.g., one or more previously coded pictures stored in the DPB 218). Specifically, the motion estimation unit 222 may calculate a value representing how similar a potential reference block is to the current block according to, for example, the sum of absolute differences (SAD), the sum of squared differences (SSD), the mean absolute difference (MAD), the mean squared difference (MSD), etc. The motion estimation unit 222 may generally perform these calculations using the sample-by-sample difference between the current block and the reference block under consideration. Motion estimation unit 222 may identify the reference block with the lowest value resulting from these calculations, indicating the reference block that most closely matches the current block.

[0216]

[0231] The motion estimation unit 222 may form one or more motion vectors (MVs) that define the position of a reference block in a reference picture relative to the position of the current block in the current picture. The motion estimation unit 222 may then provide the motion vectors to the motion compensation unit 224. For example, in the case of unidirectional inter prediction, the motion estimation unit 222 may provide a single motion vector, while in the case of bidirectional inter prediction, the motion estimation unit 222 may provide two motion vectors. The motion compensation unit 224 may then generate a predictive block using the motion vectors. For example, the motion compensation unit 224 may use the motion vectors to retrieve data of the reference block. As another example, if the motion vectors have fractional sample precision, the motion compensation unit 224 may interpolate values ​​for the predictive block according to one or more interpolation filters. Moreover, in the case of bidirectional inter prediction, the motion compensation unit 224 may retrieve data for the two reference blocks identified by the respective motion vectors and combine the retrieved data, for example, through sample-wise averaging or weighted averaging.

[0217]

[0232] When operating according to the AV1 video coding format, the motion estimation unit 222 and the motion compensation unit 224 may be configured to encode coding blocks of video data (e.g., both luma coding blocks and chroma coding blocks) using translational motion compensation, affine motion compensation, overlapped block motion compensation (OBMC), and / or synthetic inter-intra prediction.

[0218]

[0233] As another example, in the case of intra prediction or intra predictive coding, intra prediction unit 226 may generate a predictive block from samples neighboring the current block. For example, in the case of a directional mode, intra prediction unit 226 may generally mathematically combine values ​​of neighboring samples and populate these calculated values ​​in a defined direction across the current block to create the predictive block. As another example, in the case of a DC mode, intra prediction unit 226 may calculate an average of neighboring samples for the current block and generate a predictive block to include this resulting average for each sample of the predictive block.

[0219]

[0234] When operating according to the AV1 video coding format, intra prediction unit 226 may be configured to encode coding blocks of video data (e.g., both luma coding blocks and chroma coding blocks) using directional intra prediction, non-directional intra prediction, recursive filter intra prediction, chroma-from-luma (CFL) prediction, intra block copy (IBC), and / or color palette mode. Mode select unit 202 may include additional functional units that perform video prediction according to other prediction modes.

[0220]

[0235] The mode select unit 202 provides the prediction block to the residual generation unit 204. The residual generation unit 204 receives a raw, uncoded version of the current block from the video data memory 230 and the prediction block from the mode select unit 202. The residual generation unit 204 calculates sample-by-sample differences between the current block and the prediction block. The resulting sample-by-sample differences define a residual block for the current block. In some examples, the residual generation unit 204 may also determine differences between sample values ​​in the residual block to generate the residual block using residual differential pulse code modulation (RDPCM). In some examples, the residual generation unit 204 may be formed using one or more subtractor circuits that perform binary subtraction.

[0221]

[0236] In examples in which mode select unit 202 partitions CUs into PUs, each PU may be associated with a luma prediction unit and a corresponding chroma prediction unit. Video encoder 200 and video decoder 300 may support PUs having various sizes. As mentioned above, the size of a CU may refer to the size of the luma coding block of the CU, and the size of a PU may refer to the size of the luma prediction unit of the PU. Assuming that the size of a particular CU is 2N×2N, video encoder 200 may support a PU size of 2N×2N or N×N for intra prediction, and a symmetric PU size of 2N×2N, 2N×N, N×2N, N×N, or similar for inter prediction. Video encoder 200 and video decoder 300 may also support asymmetric partitioning of PU sizes of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter prediction.

[0222]

[0237] In examples where mode select unit 202 does not further partition CUs into PUs, each CU may be associated with a luma coding block and a corresponding chroma coding block. As described above, the size of a CU may refer to the size of the luma coding block of the CU. Video encoder 200 and video decoder 300 may support CU sizes of 2N×2N, 2N×N, or N×2N.

[0223]

[0238] For other video coding techniques, such as intra block copy mode coding, affine mode coding, and linear model (LM) mode coding, as some examples, mode select unit 202 generates a predictive block for the current block being coded via a respective unit associated with the coding technique. In some examples, such as palette mode coding, mode select unit 202 may not generate a predictive block, but instead may generate syntax elements that indicate how to reconstruct the block based on a selected palette. In such modes, mode select unit 202 may provide these syntax elements to entropy coding unit 220 to be coded.

[0224]

[0239] As described above, the residual generation unit 204 receives video data for a current block and a corresponding predictive block. The residual generation unit 204 then generates a residual block for the current block. To generate the residual block, the residual generation unit 204 calculates sample-by-sample differences between the predictive block and the current block.

[0225]

[0240] Transform processing unit 206 applies one or more transforms to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). Transform processing unit 206 may apply various transforms to the residual block to form the transform coefficient block. For example, transform processing unit 206 may apply a discrete cosine transform (DCT), a directional transform, a Karhunen-Loeve transform (KLT), or a conceptually similar transform to the residual block. In some examples, transform processing unit 206 may perform multiple transforms on the residual block, e.g., a linear transform and a secondary transform such as a rotational transform. In some examples, transform processing unit 206 does not apply a transform to the residual block.

[0226]

[0241] When operating according to AV1, transform processing unit 206 may apply one or more transforms to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). Transform processing unit 206 may apply various transforms to the residual block to form the transform coefficient block. For example, transform processing unit 206 may apply a horizontal / vertical transform combination, which may include a discrete cosine transform (DCT), an asymmetric discrete sine transform (ADST), an inverse ADST (e.g., ADST in reverse order), and an identity transform (IDTX). When using an identity transform, the transform is skipped in one of the vertical or horizontal directions. In some examples, the transform process may be skipped.

[0227]

[0242] The quantization unit 208 may quantize the transform coefficients in the transform coefficient block to produce a quantized transform coefficient block. The quantization unit 208 may quantize the transform coefficients of the transform coefficient block according to a quantization parameter (QP) value associated with the current block. The video encoder 200 (e.g., via the mode select unit 202) may adjust the degree of quantization applied to the transform coefficient block associated with the current block by adjusting the QP value associated with the CU. Quantization may result in loss of information, and therefore, the quantized transform coefficients may be less accurate than the original transform coefficients produced by the transform processing unit 206.

[0228]

[0243] Inverse quantization unit 210 and inverse transform processing unit 212 may apply inverse quantization and inverse transform, respectively, to the quantized transform coefficient block to reconstruct a residual block from the transform coefficient block. Reconstruction unit 214 may produce a reconstructed block that corresponds to the current block (possibly with some distortion) based on the reconstructed residual block and the predictive block generated by mode select unit 202. For example, reconstruction unit 214 may add samples of the reconstructed residual block to corresponding samples from the predictive block generated by mode select unit 202 to produce the reconstructed block.

[0229]

[0244] Filter unit 216 may perform one or more filter operations on the reconstructed blocks. For example, filter unit 216 may perform a deblocking operation to reduce blockiness artifacts along the edges of a CU. The operations of filter unit 216 may be skipped in some examples.

[0230]

[0245] When operating according to AV1, filter unit 216 may perform one or more filter operations on the reconstructed blocks. For example, filter unit 216 may perform a deblocking operation to reduce blockiness artifacts along the edges of a CU. In other examples, filter unit 216 may apply a constrained directional enhancement filter (CDEF), which may be applied after deblocking and may include application of a non-separable, nonlinear, low-pass directional filter based on estimated edge directions. Filter unit 216 may also include a loop restoration filter, which may be applied after the CDEF and may include a separable symmetric normalized Wiener filter or a dual autoinduction filter.

[0231]

[0246] Video encoder 200 stores the reconstructed blocks in DPB 218. For example, in examples where the operations of filter unit 216 are not performed, reconstruction unit 214 may store the reconstructed blocks in DPB 218. In examples where the operations of filter unit 216 are performed, filter unit 216 may store the filtered reconstructed blocks in DPB 218. Motion estimation unit 222 and motion compensation unit 224 may retrieve reference pictures formed from the reconstructed (and possibly filtered) blocks from DPB 218 to inter-predict blocks of a later-encoded picture. Additionally, intra-prediction unit 226 may use the reconstructed blocks of the current picture in DPB 218 to intra-predict other blocks in the current picture.

[0232]

[0247] Generally, entropy encoding unit 220 may entropy encode syntax elements received from other functional components of video encoder 200. For example, entropy encoding unit 220 may entropy encode quantized transform coefficient blocks from quantization unit 208. As another example, entropy encoding unit 220 may entropy encode predictive syntax elements (e.g., motion information for inter-prediction or intra-mode information for intra-prediction) from mode select unit 202. Entropy encoding unit 220 may perform one or more entropy encoding operations on syntax elements, which are other examples of video data, to generate entropy-encoded data. For example, entropy encoding unit 220 may perform a context-adaptive variable length coding (CAVLC) operation, a CABAC operation, a variable-to-variable (V2V) coding operation, a syntax-based context-adaptive binary arithmetic coding (SBAC) operation, a Probability Interval Partitioning Entropy (PIPE) coding operation, an Exponential-Golomb coding operation, or another type of entropy coding operation on the data. In some examples, entropy encoding unit 220 may operate in a bypass mode in which syntax elements are not entropy coded.

[0233]

[0248] Video encoder 200 may output a bitstream that includes entropy-encoded syntax elements needed to reconstruct blocks of a slice or picture. Specifically, entropy encoding unit 220 may output the bitstream.

[0234]

[0249] The entropy coding unit 220 may be configured as a symbol-to-symbol adaptive multi-symbol arithmetic coder according to AV1. A syntax element in AV1 includes an alphabet of N elements, and a context (e.g., a probability model) includes a set of N probabilities. The entropy coding unit 220 may store the probabilities as n-bit (e.g., 15-bit) cumulative distribution functions (CDFs). The entropy coding unit 220 may perform recursive scaling to update the context, using an update factor based on the alphabet size.

[0235]

[0250] The operations described above are described with respect to blocks. Such descriptions should be understood as operations on luma coding blocks and / or chroma coding blocks. As described above, in some examples, the luma coding blocks and chroma coding blocks are luma and chroma components of a CU. In some examples, the luma coding blocks and chroma coding blocks are luma and chroma components of a PU.

[0236]

[0251] In some examples, operations performed with respect to luma coding blocks need not be repeated for chroma coding blocks. As one example, operations identifying motion vectors (MVs) and reference pictures for luma coding blocks need not be repeated to identify MVs and reference pictures for chroma blocks. Rather, the MVs of luma coding blocks may be scaled to determine the MVs of chroma blocks, and the reference pictures may be the same. As another example, the intra prediction process may be the same for luma coding blocks and chroma coding blocks.

[0237]

[0252] Video encoder 200 illustrates an example of a device configured to encode video data, the device including: a memory configured to store the video data; and one or more processing units implemented in circuitry configured to: determine a vector predictor for a current block; determine a set of vector difference candidates; construct a vector difference candidate list based on the set of vector difference candidates; determine a vector for the current block; determine a vector difference based on the vector predictor and the vector for the current block; and signal an index into the vector difference candidate list that identifies the vector difference.

[0238]

[0253] 3 is a block diagram illustrating an example video decoder 300 that may implement the techniques of this disclosure. Figure 3 is provided for purposes of explanation and does not limit the techniques broadly illustrated and described in this disclosure. For purposes of explanation, this disclosure describes a video decoder 300 in accordance with VVC and HEVC techniques. However, the techniques of this disclosure may be implemented by video coding devices configured for other video coding standards.

[0239]

[0254] In the example of FIG. 3 , the video decoder 300 includes a coded picture buffer (CPB) memory 320, an entropy decoding unit 302, a prediction processing unit 304, an inverse quantization unit 306, an inverse transform processing unit 308, a reconstruction unit 310, a filter unit 312, and a DPB 314. Any or all of the CPB memory 320, the entropy decoding unit 302, the prediction processing unit 304, the inverse quantization unit 306, the inverse transform processing unit 308, the reconstruction unit 310, the filter unit 312, and the DPB 314 may be implemented in one or more processors or processing circuitry. For example, the units of the video decoder 300 may be implemented as one or more circuits or logic elements as part of hardware circuitry, or as part of a processor, ASIC, or FPGA. Moreover, the video decoder 300 may include additional or alternative processors or processing circuitry that perform these and other functions.

[0240]

[0255] Prediction processing unit 304 includes a motion compensation unit 316 and an intra prediction unit 318. Prediction processing unit 304 may include additional units that perform prediction according to other prediction modes. By way of example, prediction processing unit 304 may include a palette unit, an intra block copy unit (which may form part of motion compensation unit 316), an affine unit, a linear model (LM) unit, etc. In other examples, video decoder 300 may include more, fewer, or different functional components.

[0241]

[0256] When operating in accordance with AV1, the motion compensation unit 316 may be configured to decode coding blocks of video data (e.g., both luma coding blocks and chroma coding blocks) using translational motion compensation, affine motion compensation, OBMC, and / or synthetic inter-intra prediction, as described above. The intra prediction unit 318 may be configured to decode coding blocks of video data (e.g., both luma coding blocks and chroma coding blocks) using directional intra prediction, non-directional intra prediction, recursive filter intra prediction, CFL, IBC, and / or color palette mode, as described above.

[0242]

[0257] CPB memory 320 may store video data, such as an encoded video bitstream, to be decoded by components of video decoder 300. The video data stored in CPB memory 320 may be retrieved, for example, from computer-readable medium 110 (FIG. 1). CPB memory 320 may include a CPB that stores encoded video data (e.g., syntax elements) from the encoded video bitstream. CPB memory 320 may also store video data other than syntax elements of coded pictures, such as temporary data representing output from various units of video decoder 300. DPB 314 generally stores decoded pictures that video decoder 300 may output and / or use as reference video data when decoding subsequent data or pictures of the encoded video bitstream. CPB memory 320 and DPB 314 may be formed by any of a variety of memory devices, such as DRAM, including SDRAM, MRAM, RRAM, or other types of memory devices. CPB memory 320 and DPB 314 may be provided by the same memory device or separate memory devices. In various examples, CPB memory 320 may be on-chip with other components of video decoder 300 or off-chip relative to those components.

[0243]

[0258] Additionally or alternatively, in some examples, video decoder 300 may retrieve coded video data from memory 120 (FIG. 1). That is, memory 120 may store data such as those discussed above for CPB memory 320. Similarly, memory 120 may store instructions to be executed by video decoder 300 when some or all of the functionality of video decoder 300 is implemented in software to be executed by processing circuitry of video decoder 300.

[0244]

[0259] The various units shown in FIG. 3 are presented to aid in understanding the operations performed by video decoder 300. The units may be implemented as fixed-function circuits, programmable circuits, or a combination thereof. As with FIG. 2, fixed-function circuits refer to circuits that provide a specific function and have predefined operations that they can perform. Programmable circuits refer to circuits that can be programmed to perform various tasks and offer flexibility in the operations that they can perform. For example, a programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by the software or firmware instructions. While a fixed-function circuit may execute software instructions (e.g., receive parameters or output parameters), the types of operations that the fixed-function circuit performs are generally invariant. In some examples, one or more of the units may be different circuit blocks (fixed function or programmable), and in some examples, one or more of the units may be integrated circuits.

[0245]

[0260] Video decoder 300 may include a programmable core formed from ALUs, EFUs, digital circuits, analog circuits, and / or programmable circuits. In examples where the operations of video decoder 300 are performed by software executing on programmable circuits, on-chip or off-chip memory may store software instructions (e.g., object code) that video decoder 300 receives and executes.

[0246]

[0261] The entropy decoding unit 302 may receive the encoded video data from the CPB and entropy decode the video data to recover the syntax elements. The prediction processing unit 304, the inverse quantization unit 306, the inverse transform processing unit 308, the reconstruction unit 310, and the filter unit 312 may generate decoded video data based on the syntax elements extracted from the bitstream.

[0247]

[0262] Generally, video decoder 300 reconstructs a picture on a block-by-block basis. Video decoder 300 may perform a reconstruction operation on each block individually (the block currently being reconstructed, i.e., decoded, may be referred to as the “current block”).

[0248]

[0263] The entropy decoding unit 302 may entropy decode syntax elements that define the quantized transform coefficients of the quantized transform coefficient block as well as transform information, such as a quantization parameter (QP) and / or a transform mode indication(s). The inverse quantization unit 306 may use the QP associated with the quantized transform coefficient block to determine the degree of quantization, and similarly the degree of inverse quantization, that the inverse quantization unit 306 should apply. The inverse quantization unit 306 may, for example, perform a bitwise left-shift operation to inverse quantize the quantized transform coefficients. The inverse quantization unit 306 may thereby form a transform coefficient block including the transform coefficients.

[0249]

[0264] After the inverse quantization unit 306 forms the transform coefficient blocks, the inverse transform processing unit 308 may apply one or more inverse transforms to the transform coefficient blocks to generate residual blocks associated with the current block. For example, the inverse transform processing unit 308 may apply an inverse DCT, an inverse integer transform, an inverse Karhunen-Loeve transform (KLT), an inverse rotational transform, an inverse transform, or another inverse transform to the transform coefficient blocks.

[0250]

[0265] Further, prediction processing unit 304 generates a predictive block according to the prediction information syntax element entropy decoded by entropy decoding unit 302. For example, if the prediction information syntax element indicates that the current block is inter-predicted, motion compensation unit 316 may generate a predictive block. In this case, the prediction information syntax element may indicate a reference picture in DPB 314 from which to retrieve a reference block, as well as a motion vector that identifies the location of the reference block in the reference picture relative to the location of the current block in the current picture. Motion compensation unit 316 may generally perform the inter-prediction process in a manner substantially similar to that described with respect to motion compensation unit 224 (FIG. 2).

[0251]

[0266] As another example, if the prediction information syntax element indicates that the current block is intra-predicted, intra prediction unit 318 may generate a predictive block according to the intra-prediction mode indicated by the prediction information syntax element. Again, intra prediction unit 318 may generally perform the intra-prediction process in a manner substantially similar to that described with respect to intra prediction unit 226 (FIG. 2). Intra prediction unit 318 may retrieve data of neighboring samples for the current block from DPB 314.

[0252]

[0267] The reconstruction unit 310 may reconstruct the current block using the predictive block and the residual block. For example, the reconstruction unit 310 may add samples of the residual block to corresponding samples of the predictive block to reconstruct the current block.

[0253]

[0268] Filter unit 312 may perform one or more filter operations on the reconstructed blocks. For example, filter unit 312 may perform a deblocking operation to reduce blockiness artifacts along the edges of the reconstructed blocks. The operations of filter unit 312 are not necessarily performed in all instances.

[0254]

[0269] The video decoder 300 may store the reconstructed blocks in the DPB 314. For example, in examples where the operations of the filter unit 312 are not performed, the reconstruction unit 310 may store the reconstructed blocks in the DPB 314. In examples where the operations of the filter unit 312 are performed, the filter unit 312 may store the filtered reconstructed blocks in the DPB 314. As described above, the DPB 314 may provide reference information to the prediction processing unit 304, such as samples of the current picture for intra prediction and previously decoded pictures for subsequent motion compensation. Furthermore, the video decoder 300 may output the decoded pictures (e.g., decoded video) from the DPB 314 for later display on a display device, such as the display device 118 of FIG. 1 .

[0255]

[0270] Thus, video decoder 300 illustrates an example of a video decoding device that includes a memory configured to store video data and one or more processing units implemented in circuitry configured to: determine a vector predictor for a current block; determine a set of vector difference candidates; construct a vector difference candidate list based on the set of vector difference candidates; determine a vector for the current block based on the vector predictor and a vector difference from the vector difference candidate list; and reconstruct the current block based on the vector for the current block.

[0256]

[0271] 4 is a flowchart illustrating an example method for encoding a current block in accordance with the techniques of this disclosure. The current block may be or include a current CU. Although described with respect to video encoder 200 (FIGS. 1 and 2), it should be understood that other devices may be configured to implement methods similar to the method of FIG.

[0257]

[0272] In this example, video encoder 200 first predicts the current block (350). For example, video encoder 200 may form a predictive block for the current block. Video encoder 200 may then calculate a residual block for the current block (352). To calculate the residual block, video encoder 200 may calculate the difference between the original uncoded block and the predictive block for the current block. Video encoder 200 may then transform the residual block and quantize the transform coefficients of the residual block (354). Next, video encoder 200 may scan the quantized transform coefficients of the residual block (356). During or following the scan, video encoder 200 may entropy code the transform coefficients (358). For example, video encoder 200 may code the transform coefficients using CAVLC or CABAC. Video encoder 200 may then output entropy-coded data for the block (360).

[0258]

[0273] 5 is a flowchart illustrating an example method for decoding a current block of video data in accordance with the techniques of this disclosure. The current block may be or include a current CU. Although described with respect to video decoder 300 (FIGS. 1 and 3), it should be understood that other devices may be configured to implement methods similar to the method of FIG.

[0259]

[0274] The video decoder 300 may receive entropy-coded data for the current block, such as entropy-coded prediction information and entropy-coded data for the transform coefficients of the residual block corresponding to the current block (370). The video decoder 300 may entropy decode the entropy-coded data to determine prediction information for the current block and reconstruct the transform coefficients of the residual block (372). The video decoder 300 may predict the current block, e.g., using the intra-prediction mode or inter-prediction mode indicated by the prediction information for the current block, to calculate a predictive block for the current block (374). The video decoder 300 may then inverse-scan the reconstructed transform coefficients to create a block of quantized transform coefficients (376). The video decoder 300 may then dequantize the transform coefficients and apply an inverse transform to the transform coefficients to produce a residual block (378). The video decoder 300 may finally decode the current block by combining the predictive block and the residual block (380).

[0260]

[0275] 19 is a flowchart illustrating another exemplary method for decoding a current block in accordance with the techniques of this disclosure. The example of FIG. 19 is described with reference to processing circuitry of video decoder 300. For example, one or more memories (e.g., memory 120, CPB memory 320, DPB 314, or some other memory) may be configured to store video data. The processing circuitry of video decoder 300 may be coupled to one or more memories.

[0261]

[0276] Processing circuitry of the video decoder 300 may be configured to determine a vector predictor for a current block of video data (1900). For example, the video encoder 200 may signal information (e.g., vectors of neighboring blocks, etc.) that the video decoder 300 uses to determine the vector predictor.

[0262]

[0277] The processing circuitry of the video decoder 300 may be configured to determine a first set of vector difference candidates based on a first pel interval (1902). The first pel interval may define a first distance between consecutive vector difference candidates in the first set of vector difference candidates starting from the vector predictor. For example, the first set of vector difference candidates may be the bvdCandsStep1Input described above, and the first pel interval may be 8 pels or a different interval, including a fractional interval. As an example, the processing circuitry of the video decoder 300 may add candidates to the first set of vector difference candidates that are a set of positions, each of which surrounds the position of the vector predictor and is an integer multiple of the first distance away from the vector predictor. The processing circuitry of the video decoder 300 may add candidates until a range threshold is met and / or until a cost value associated with one candidate in the first set of vector difference candidates meets a cost threshold.

[0263]

[0278] The processing circuitry of the video decoder 300 may determine a second set of vector difference candidates based on the first set of vector difference candidates and the second pel interval (1904). The second pel interval may define a second distance between consecutive vector difference candidates in the second set of vector difference candidates, and the second pel interval may be smaller than the first pel interval.

[0264]

[0279] For example, the second set of vector difference candidates may be bvdCandsStep2Input or bvdCandsSubSetStep2Input described above, and the second pel interval may be a different interval, including 4-pel, 1-pel, or fractional interval. The processing circuitry of the video decoder 300 may determine respective cost values ​​(e.g., SAD-based TM costs) associated with the candidates in the first set of vector difference candidates and construct a first subset of vector difference candidates (e.g., bvdCandsStep1Output) based on the respective cost values. The processing circuitry of the video decoder 300 may determine a second set of vector difference candidates (e.g., bvdCandsStep2Input or bvdCandsSubSetStep2Input) based on the first subset of vector difference candidates (e.g., bvdCandsStep1Output).

[0265]

[0280] As an example, the processing circuitry of the video decoder 300 may include at least a subset of candidates in the first set of vector difference candidates in the second set of vector difference candidates. For example, bvdCandsStep2Input or bvdCandsSubSetStep2Input includes one or more candidates from bvdCandsStep1Output or bvdCandsStep1Input. The processing circuitry may add candidates between the subset of candidates in the first set of vector difference candidates based on an integer multiple of the second distance. In some examples, the second distance is less than or equal to the second pel spacing. As an example, the processing circuitry of the video decoder 300 may add candidates to the second set of vector difference candidates until cost values ​​associated with candidates in the second set of vector difference candidates satisfy a cost threshold.

[0266]

[0281] The processing circuitry of the video decoder 300 may construct a vector difference candidate list (e.g., bvdCandListN) based on the second set of vector difference candidates (1906). For example, the processing circuitry of the video decoder 300 may determine respective cost values ​​associated with the candidates in the second set of vector difference candidates (e.g., the cost values ​​of the candidates in bvdCandsStep2Output). The processing circuitry of the video decoder 300 may construct the vector difference candidate list (e.g., bvdCandListN) based on the respective cost values. For example, the processing circuitry of the video decoder 300 may select the N candidates with the lowest cost values, and optionally, may arrange the N candidates from lowest cost to highest cost.

[0267]

[0282] Processing circuitry in the video decoder 300 may determine a vector for the current block based on the vector predictor and a vector difference from the vector difference candidate list (1908). For example, the video decoder 300 may receive an index into the vector difference candidate list that identifies the vector difference. The video decoder 300 may add the vector difference to the vector predictor to determine a motion vector for the current block.

[0268]

[0283] Processing circuitry in video decoder 300 may reconstruct 1910 the current block based on the vector for the current block. For example, processing circuitry in video decoder 300 may generate a prediction signal based on the vector for the current block and receive residual information from video encoder 200. Processing circuitry in video decoder 300 may sum the prediction signal and the residual information to reconstruct the current block.

[0269]

[0284] 19, the current block may be coded in intra block copy (IBC) merge mode using block vector difference (IBC-MBVD), where the vector predictor may be a block vector predictor (BVP) and the vector difference candidate list may be a block vector difference (BVD) candidate list.

[0270]

[0285] 20 is a flowchart illustrating another exemplary method for encoding a current block in accordance with the techniques of this disclosure. The example of FIG. 20 is described with reference to processing circuitry of video encoder 200. For example, one or more memories (e.g., memory 106, video data memory 230, decoded picture buffer 218, or some other memory) may be configured to store video data. The processing circuitry of video encoder 200 may be coupled to one or more memories.

[0271]

[0286] Using techniques similar to those described above for the video decoder 300, the processing circuitry of the video encoder 200 may determine a vector predictor for a current block of video data (2000), determine a first set of vector difference candidates based on a first pel interval that defines a first distance between consecutive vector difference candidates in the first set of vector difference candidates starting from the vector predictor (2002), determine a second distance between the first set of vector difference candidates and a second pel interval that defines a second distance between consecutive vector difference candidates in a second set of vector difference candidates, the second pel interval being smaller than the first pel interval (2004), and construct a vector difference candidate list based on the second set of vector difference candidates (2006). In Figure 20, the processing circuitry of the video encoder 200 may be configured to signal an index into the vector difference candidate list for decoding the current block (2008). The processing circuitry of the video encoder 200 may also signal residual information indicating a difference between the current block and the prediction signal.

[0272]

[0287] 21 is a flowchart illustrating an example method for constructing a vector difference candidate list. The example of FIG. 21 shows a two-step algorithm (e.g., a derivation process) for constructing a vector difference candidate list. The video encoder 200 and the video decoder 300 may be configured to implement the same technique. The example of FIG. 21 is described with reference to processing circuitry that may be that of the video encoder 200 or the video decoder 300.

[0273]

[0288] The processing circuitry may add candidates that are multiples of the first distance from the vector predictor to a first set of vector difference candidates (e.g., bvdCandsStep1Input) (2100). For example, if the first distance is defined by a first pel spacing (e.g., S1), then candidates may be multiples of S1,2 * S1, 3 * It can be S1, etc.

[0274]

[0289] The processing circuitry may determine individual cost values ​​for the candidates (2102). For example, the processing circuitry may determine the TM cost based on the SAD value between the top-left reference template of the reference block and the top-left current template of the current block, as indicated by the vector predictor added to the candidate. In some examples, the processing circuitry may first add all candidates to a first set of vector difference candidates and then determine the respective cost values.

[0275]

[0290] The processing circuitry may determine whether a range threshold or a cost threshold is met (2104). For example, the processing circuitry may determine whether the last candidate has been evaluated or whether the cost of the candidate is below a threshold. If the range threshold or the cost threshold is not met (2104 is no), the processing circuitry may add another candidate to the first set of vector difference candidates (2100).

[0276]

[0291] If the range threshold or cost threshold is met (YES at 2104), the processing circuitry may construct a first subset of vector difference candidates based on the cost values ​​2106. An example of a first subset of vector difference candidates is bvdCandsStep1Output.

[0277]

[0292] The processing circuitry may add candidates that are between candidates in the first set of vector difference candidates and that are multiples of the second distance to a second set of vector difference candidates (e.g., bvdCandsSubSetStep2Input) 2108. For example, if the second distance is defined by a second pel spacing (e.g., S2), the candidates may be {candStep1Output-S2, candStep1Output-(S2-1), candStep1Output-(S2-2),..., candStep1Output, candStep1Output+(S2+1), candStep1Output+(S2+2),..., candStep1Output+S2}, where candStep1Output is a candidate in the first set of vector difference candidates.

[0278]

[0293] The processing circuitry may determine individual cost values ​​for the candidates (2110). For example, the processing circuitry may determine the TM cost based on the SAD value between the top-left reference template of the reference block and the top-left current template of the current block, as indicated by the vector predictor added to the candidate. In some examples, the processing circuitry may first add all candidates to a second set of vector difference candidates and then determine the respective cost values.

[0279]

[0294] The processing circuitry may determine whether the last candidate has been evaluated or whether a cost threshold has been met 2112. If the last candidate has not been evaluated or the cost threshold has not been met (2112 = no), the processing circuitry may add another candidate to the second set of vector difference candidates 2108.

[0280]

[0295] If the last candidate has been evaluated or the cost threshold is met (2112: yes), the processing circuitry may construct a vector difference candidate list (e.g., bvdCandListN) based on the cost values ​​of the candidates in the second set of vector difference candidates (e.g., bvdCandsSubSetStep2Input) (2114). For example, the processing circuitry may determine the N candidates in bvdCandsSubSetStep2Input that have the lowest cost and arrange the N candidates from lowest cost to highest cost, and the result may be the vector difference candidate list.

[0281]

[0296] 22 is a flowchart illustrating another exemplary method for constructing a vector difference candidate list. The example of FIG. 22 shows an iterative algorithm (e.g., a derivation process) for constructing a vector difference candidate list. The video encoder 200 and the video decoder 300 may be configured to implement the same technique. The example of FIG. 22 is described with reference to processing circuitry that may be that of the video encoder 200 or the video decoder 300.

[0282]

[0297] The processing circuitry may add candidates to the current set of vector difference candidates based on the current pel spacing 2200. For example, the processing circuitry may add candidates that are 8 pels, 12 pels, 16 pels, etc., and the current pel spacing is 8.

[0283]

[0298] The processing circuitry may determine the current subset based on the cost values ​​of the candidates in the current set 2202. For example, the processing circuitry may select the X candidates with the lowest cost.

[0284]

[0299] The processing circuitry may determine whether the pel threshold has been reached (2204). For example, in this example, it is assumed that the pel threshold is 2 pels. Because the current pel interval is 8 pels, the processing circuitry may determine that the pel threshold has not been reached (2204 if no).

[0285]

[0300] The processing circuitry may set the current set of vector difference candidates to the current set of vector difference candidates and set the current pel interval to a smaller pel interval (e.g., 4 pels) (2208). The processing circuitry may then repeat the operations shown in FIG. 22 (if 2204 is Yes) until the pel threshold is reached. The processing circuitry may build a vector difference candidate list based on the current subset (e.g., the X candidates with the lowest cost).

[0286]

[0301] The following numbered clauses illustrate one or more aspects of the devices and techniques described in this disclosure.

[0287]

[0302] Clause 1. A method for decoding video data, comprising: determining a vector predictor for a current block; determining a set of vector difference candidates; constructing a vector difference candidate list based on the set of vector difference candidates; determining a vector for the current block based on the vector predictor and a vector difference from the vector difference candidate list; and reconstructing the current block based on the vector for the current block.

[0288]

[0303] Clause 2. A method for encoding video data, comprising: determining a vector predictor for a current block; determining a set of vector difference candidates; constructing a vector difference candidate list based on the set of vector difference candidates; determining a vector for the current block; determining a vector difference based on the vector predictor and the vector for the current block; and signaling an index into the vector difference candidate list that identifies the vector difference.

[0289]

[0304] Clause 3. A method according to any of clauses 1 and 2, wherein the vector predictor is one of a block vector predictor or a motion vector predictor, the set of vector difference candidates is one of a set of block vector difference candidates or a set of motion vector difference candidates, the vector difference candidate list is one of a block vector difference candidate list or a motion vector difference candidate list, and the vector for the current block is one of a block vector or a motion vector for the current block.

[0290]

[0305] Clause 4. The method of any of clauses 1-3, wherein determining the set of vector difference candidates includes determining the set of vector difference candidates based on a set of positions surrounding a position of the vector predictor.

[0291]

[0306] Clause 5. The method of clause 4, wherein the set of locations includes all locations surrounding the vector predictor location or fewer than all locations surrounding the vector predictor location.

[0292]

[0307] Clause 6. The method of any of clauses 1 to 5, further comprising determining a subset of vector difference candidates from the set of vector difference candidates, and constructing the vector difference candidate list comprises constructing the vector difference candidate list based on the subset of vector difference candidates.

[0293]

[0308] Clause 7. The method of any of clauses 1 to 6, wherein constructing the vector difference candidate list includes determining a first subset of the set of vector difference candidates based on one of selecting candidates along each direction using different pels, or selecting candidates along each direction using different pels and reordering, and iteratively determining one or more additional subsets of vector difference candidates by adding candidates at different pel distances based on a previous subset of the set of vector difference candidates, wherein the final subset of vector difference candidates from the additional subsets is the vector difference candidate list.

[0294]

[0309] Clause 8. The method of any one of clauses 1 to 7, further comprising determining that a vector difference candidate in the vector difference candidate list is valid based on one or more of: when the vector difference candidate is summed with the vector predictor, a hypothetical vector within the search range is obtained; when the vector predictor is derived from an adjacent block and the adjacent block is coded with a horizontal flip type, the vector difference candidate has a vertical component value equal to 0; when the vector predictor is derived from an adjacent block and the adjacent block is coded with a vertical flip type, the vector difference candidate has a horizontal component value equal to 0; when the vector predictor is derived from an adjacent block and the adjacent block is coded with a vertical flip type, the vector difference candidate has a vertical component value equal to 0; and when the current block is coded with a vertical flip type, the vector difference candidate has a horizontal component value equal to 0.

[0295]

[0310] Clause 9. The method of any of clauses 1 to 8, further comprising determining a threshold size of the set of vector difference candidates based on a block size of the current block, wherein determining the set of vector difference candidates comprises determining the set of vector difference candidates until the number of vector difference candidates is equal to the threshold size.

[0296]

[0311] Clause 10. A device for decoding video data, the device comprising: a memory configured to store the video data; and processing circuitry configured to perform the method of any of clauses 1 and 3 to 9.

[0297]

[0312] Clause 11. The device of clause 10, further comprising a display configured to display the decoded video data.

[0298]

[0313] Clause 12. A device according to any of clauses 10 and 11, wherein the device comprises a video decoder or a video encoder.

[0299]

[0314] Clause 13. A device for encoding video data, the device comprising: a memory configured to store the video data; and processing circuitry configured to perform the method of any of clauses 2 to 9.

[0300]

[0315] Clause 14. The device of clause 13, wherein the device comprises a video encoder.

[0301]

[0316] Clause 15. A device according to any of clauses 10 to 13, wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

[0302]

[0317] Clause 16. A computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to perform the method of any of clauses 1 and 3-9.

[0303]

[0318] Clause 17. A computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to perform the method of any of clauses 2-9.

[0304]

[0319] Clause 18. A device for decoding video data, comprising means for implementing the method according to any one of clauses 1 and 3 to 9.

[0305]

[0320] Clause 19. A device for decoding video data, comprising means for implementing the method according to any one of clauses 2 to 9.

[0306]

[0321] Clause 1A. A method for decoding video data, comprising: determining a vector predictor for a current block of the video data; determining a first set of vector difference candidates based on a first pel interval defining a first distance between consecutive vector difference candidates in the first set of vector difference candidates starting from the vector predictor; determining a second set of vector difference candidates based on the first set of vector difference candidates and a second pel interval defining a second distance between consecutive vector difference candidates in the second set of vector difference candidates, the second pel interval being smaller than the first pel interval; constructing a vector difference candidate list based on the second set of vector difference candidates; determining a vector for the current block based on the vector predictor and a vector difference from the vector difference candidate list; and reconstructing the current block based on the vector for the current block.

[0307]

[0322] Clause 2A. The method of clause 1A, wherein determining a first set of vector difference candidates includes adding candidates that are a set of positions that each surround a position of the vector predictor and are an integer multiple of a first distance away from the vector predictor to the first set of vector difference candidates.

[0308]

[0323] Clause 3A. The method of clause 2A, wherein adding candidates includes adding candidates until a range threshold is met.

[0309]

[0324] Clause 4A. The method of either clause 2A or 3A, wherein adding candidates includes adding candidates until a cost value associated with one candidate in the first set of vector difference candidates meets a cost threshold.

[0310]

[0325] Clause 5A. The method of any of Clauses 1A to 4A, wherein determining the second set of vector difference candidates includes including at least a subset of candidates in the first set of vector difference candidates in the second set of vector difference candidates, and adding candidates between the subset of candidates in the first set of vector difference candidates based on an integer multiple of the second distance.

[0311]

[0326] Clause 6A. The method of clause 5A, wherein adding candidates among the subset of candidates includes adding candidates to the second set of vector difference candidates until cost values ​​associated with candidates in the second set of vector difference candidates satisfy a cost threshold.

[0312]

[0327] Clause 7A. The method of any of clauses 1A to 6A, further comprising: determining respective cost values ​​associated with candidates in a first set of vector difference candidates; and constructing a first subset of vector difference candidates based on the respective cost values; and wherein determining a second set of vector difference candidates comprises determining the second set of vector difference candidates based on the first subset of vector difference candidates.

[0313]

[0328] Clause 8A. The method of any of clauses 1A to 7A, further comprising determining respective cost values ​​associated with candidates in the second set of vector difference candidates, and constructing the vector difference candidate list comprises constructing the vector difference candidate list based on the respective cost values.

[0314]

[0329] Clause 9A. A method according to any one of clauses 1A to 8A, wherein the current block is coded in intra block copy (IBC) merge mode using block vector difference (IBC-MBVD), the vector predictor is a block vector predictor (BVP), and the vector difference candidate list is a block vector difference (BVD) candidate list.

[0315]

[0330] Clause 10A. The method of any of clauses 1A-9A, wherein the second distance is less than or equal to the second pel spacing.

[0316]

[0331] Clause 11A. A device for decoding video data, the device comprising: one or more memories configured to store the video data; and a processing circuit configuration coupled to the one or more memories configured to: determine a vector predictor for a current block; determine a first set of vector difference candidates based on a first pel interval defining a first distance between consecutive vector difference candidates in the first set of vector difference candidates starting from the vector predictor; determine a second set of vector difference candidates based on the first set of vector difference candidates and a second pel interval defining a second distance between consecutive vector difference candidates in the second set of vector difference candidates, the second pel interval being smaller than the first pel interval; construct a vector difference candidate list based on the second set of vector difference candidates; determine a vector for the current block based on the vector predictor and a vector difference from the vector difference candidate list; and reconstruct the current block based on the vector for the current block.

[0317]

[0332] Clause 12A. The device of Clause 11A, wherein to determine the first set of vector difference candidates, the processing circuitry is configured to add candidates that are a set of positions, each of which surrounds the position of the vector predictor and is an integer multiple of the first distance from the vector predictor, to the first set of vector difference candidates.

[0318]

[0333] Clause 13A. The device of clause 12A, wherein to add candidates, the processing circuitry is configured to add candidates until a range threshold is met.

[0319]

[0334] Clause 14A. The device of clause 12A or 13A, wherein to add a candidate, the processing circuitry is configured to add a candidate until a cost value associated with one candidate in the first set of vector difference candidates satisfies a cost threshold.

[0320]

[0335] Clause 15A. The device of any of Clauses 11A to 14A, wherein the processing circuitry is configured to: determine the second set of vector difference candidates by including at least a subset of candidates in the first set of vector difference candidates in the second set of vector difference candidates, and adding candidates between the subset of candidates in the first set of vector difference candidates based on an integer multiple of the second distance.

[0321]

[0336] Clause 16A. The device of clause 15A, wherein to add candidates between the subset of candidates, the processing circuitry is configured to add candidates to the second set of vector difference candidates until a cost value associated with a candidate in the second set of vector difference candidates satisfies a cost threshold.

[0322]

[0337] Clause 17A. A device described in any of Clauses 11A to 16A, wherein the processing circuitry is configured to determine respective cost values ​​associated with candidates in a first set of vector difference candidates and construct a first subset of vector difference candidates based on the respective cost values, and to determine a second set of vector difference candidates, the processing circuitry is configured to determine the second set of vector difference candidates based on the first subset of vector difference candidates.

[0323]

[0338] Clause 18A. A device described in any of clauses 11A to 17A, wherein the processing circuitry is configured to determine respective cost values ​​associated with candidates in the second set of vector difference candidates, and to construct a vector difference candidate list, the processing circuitry is configured to construct the vector difference candidate list based on the respective cost values.

[0324]

[0339] Clause 19A. A device described in any of clauses 11A to 18A, wherein the current block is encoded in intra block copy (IBC) merge mode using block vector difference (IBC-MBVD), the vector predictor is a block vector predictor (BVP), and the vector difference candidate list is a block vector difference (BVD) candidate list.

[0325]

[0340] Clause 20A. The device of any of clauses 11A to 19A, wherein the second distance is less than or equal to the second Pel spacing.

[0326]

[0341] Clause 21A. The device of any of clauses 11A to 20A, wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

[0327]

[0342] Clause 22A. A computer-readable storage medium having stored thereon instructions that, when executed, are configured to cause one or more processors to determine a vector predictor for a current block of video data; determine a first set of vector difference candidates based on a first pel interval defining a first distance between consecutive vector difference candidates in the first set of vector difference candidates starting from the vector predictor; determine a second set of vector difference candidates based on the first set of vector difference candidates and a second pel interval defining a second distance between consecutive vector difference candidates in the second set of vector difference candidates, the second pel interval being smaller than the first pel interval; construct a vector difference candidate list based on the second set of vector difference candidates; determine a vector for the current block based on the vector predictor and a vector difference from the vector difference candidate list; and reconstruct the current block based on the vector for the current block.

[0328]

[0343] Clause 23A. A device for encoding video data, the device including: one or more memories configured to store the video data; and processing circuitry coupled to the one or more memories configured to: determine a vector predictor for a current block of the video data; determine a first set of vector difference candidates based on a first pel interval defining a first distance between consecutive vector difference candidates in the first set of vector difference candidates starting from the vector predictor; determine a second set of vector difference candidates based on the first set of vector difference candidates and a second pel interval defining a second distance between consecutive vector difference candidates in the second set of vector difference candidates, the second pel interval being smaller than the first pel interval; construct a vector difference candidate list based on the second set of vector difference candidates; and signal an index into the vector difference candidate list for decoding the current block.

[0329]

[0344] It should be recognized that, in some examples, certain acts or events of any of the techniques described herein may be performed in a different order, added, merged, or omitted entirely (e.g., not all acts or events described may be required to practice the techniques). Moreover, in certain examples, acts or events may be performed in parallel rather than sequentially, for example, through multithreading, interrupt processing, or multiple processors.

[0330]

[0345] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which correspond to tangible media such as data storage media, or communication media, including any medium that facilitates transfer of a computer program from one place to another, for example, according to a communications protocol. As such, computer-readable media may generally correspond to (1) tangible computer-readable storage media that is non-transitory, or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures to implement the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0331]

[0346] By way of example, and not limitation, such computer-readable storage media may include one or more of RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included within the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but instead cover non-transitory tangible storage media. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks typically reproduce data magnetically, while discs reproduce data optically using a laser. Combinations of the above should also be included within the scope of computer-readable media.

[0332]

[0347] The instructions may be executed by one or more processors, such as one or more DSPs, general-purpose microprocessors, ASICs, FPGAs, or other equivalent integrated or discrete logic circuitry. Accordingly, the terms "processor" and "processing circuitry," as used herein, may refer to any of the above structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or may be incorporated into a combined codec. It is also possible for these techniques to be implemented entirely in one or more circuits or logic elements.

[0333]

[0348] The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units have been described in this disclosure to highlight functional aspects of devices configured to implement the disclosed techniques, but they do not necessarily require realization by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit or may be provided by a collection of interoperable hardware units, including one or more processors as described above, in conjunction with suitable software and / or firmware.

[0334]

[0349] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

1. 1. A method for decoding video data, comprising: determining a vector predictor for a current block of said video data; determining a first set of vector difference candidates based on a first pel interval defining a first distance between consecutive vector difference candidates in the first set of vector difference candidates starting from the vector predictor; determining a second set of vector difference candidates based on the first set of vector difference candidates and a second pel interval that defines a second distance between consecutive vector difference candidates in the second set of vector difference candidates, the second pel interval being smaller than the first pel interval; constructing a vector difference candidate list based on the second set of vector difference candidates; determining a vector for the current block based on the vector predictor and a vector difference from the vector difference candidate list; reconstructing the current block based on the vector for the current block; A method comprising:

2. determining the first set of vector difference candidates; 2. The method of claim 1, comprising adding candidates to the first set of vector difference candidates that are a set of positions each surrounding a position of the vector predictor and spaced from the vector predictor by an integer multiple of the first distance.

3. The method of claim 2 , wherein adding candidates comprises adding candidates until a range threshold is met.

4. The method of claim 2 , wherein adding candidates comprises adding candidates until a cost value associated with a candidate in the first set of vector difference candidates meets a cost threshold.

5. determining a second set of vector difference candidates; including at least a subset of candidates from the first set of vector difference candidates in the second set of vector difference candidates; adding candidates among a subset of the candidates in the first set of vector difference candidates based on an integer multiple of the second distance; The method of claim 1 , comprising:

6. 6. The method of claim 5, wherein adding candidates among the subset of candidates comprises adding candidates to the second set of vector difference candidates until cost values ​​associated with candidates in the second set of vector difference candidates satisfy a cost threshold.

7. determining a respective cost value associated with a first set of candidates of the vector difference candidates; constructing a first subset of vector difference candidates based on the respective cost values; determining the second set of vector difference candidates includes determining the second set of vector difference candidates based on the first subset of vector difference candidates. The method of claim 1.

8. determining respective cost values ​​associated with candidates of the second set of vector difference candidates; constructing the vector difference candidate list includes constructing the vector difference candidate list based on the respective cost values. The method of claim 1.

9. 2. The method of claim 1, wherein the current block is coded in intra block copy (IBC) merge mode with block vector difference (IBC-MBVD), the vector predictor is a block vector predictor (BVP), and the vector difference candidate list is a block vector difference (BVD) candidate list.

10. The method of claim 1 , wherein the second distance is less than or equal to the second pel spacing.

11. 1. A device for decoding video data, comprising: one or more memories configured to store the video data; processing circuitry coupled to the one or more memories, determining a vector predictor for the current block; determining a first set of vector difference candidates based on a first pel interval defining a first distance between successive vector difference candidates in the first set of vector difference candidates starting from the vector predictor; determining a second set of vector difference candidates based on the first set of vector difference candidates and a second pel spacing that defines a second distance between consecutive vector difference candidates in the second set of vector difference candidates, the second pel spacing being smaller than the first pel spacing; constructing a vector difference candidate list based on the second set of vector difference candidates; determining a vector for the current block based on the vector predictor and a vector difference from the vector difference candidate list; reconstructing the current block based on the vector for the current block; a processing circuit arrangement configured to A device comprising:

12. To determine the first set of vector difference candidates, the processing circuitry adding candidates that are a set of positions that each surround the vector predictor position and are an integer multiple of the first distance from the vector predictor to the first set of vector difference candidates; The device of claim 11 configured to:

13. The device of claim 12 , wherein to add candidates, the processing circuitry is configured to add candidates until a range threshold is met.

14. 13. The device of claim 12, wherein to add candidates, the processing circuitry is configured to add candidates until a cost value associated with a candidate in the first set of vector difference candidates meets a cost threshold.

15. to determine the second set of vector difference candidates, the processing circuitry including at least a subset of candidates from the first set of vector difference candidates in the second set of vector difference candidates; adding candidates among a subset of the candidates in the first set of vector difference candidates based on an integer multiple of the second distance; It is configured as follows: The device of claim 11.

16. 16. The device of claim 15, wherein to add candidates among the subset of candidates, the processing circuitry is configured to add candidates to the second set of vector difference candidates until cost values ​​associated with candidates in the second set of vector difference candidates satisfy a cost threshold.

17. the processing circuitry comprising: determining a respective cost value associated with a first set of candidates of the vector difference candidates; constructing a first subset of vector difference candidates based on the respective cost values; to determine the second set of vector difference candidates, the processing circuitry being configured to determine the second set of vector difference candidates based on the first subset of vector difference candidates. The device of claim 11.

18. the processing circuitry comprising: configured to determine respective cost values ​​associated with candidates of the second set of vector difference candidates; and to construct the vector difference candidate list, the processing circuitry is configured to configure the vector difference candidate list based on the respective cost values. The device of claim 11.

19. 12. The device of claim 11, wherein the current block is encoded in intra block copy (IBC) merge mode with block vector difference (IBC-MBVD), the vector predictor is a block vector predictor (BVP), and the vector difference candidate list is a block vector difference (BVD) candidate list.

20. The device of claim 11 , wherein the second distance is less than or equal to the second pel spacing.

21. The device of claim 11 , wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

22. 1. A device for encoding video data, comprising: one or more memories configured to store the video data; processing circuitry coupled to the one or more memories, determining a vector predictor for a current block of said video data; determining a first set of vector difference candidates based on a first pel interval defining a first distance between successive vector difference candidates in the first set of vector difference candidates starting from the vector predictor; determining a second set of vector difference candidates based on the first set of vector difference candidates and a second pel spacing that defines a second distance between consecutive vector difference candidates in the second set of vector difference candidates, the second pel spacing being smaller than the first pel spacing; constructing a vector difference candidate list based on the second set of vector difference candidates; processing circuitry configured to signal an index into the vector difference candidate list for decoding the current block; Including, the device.

Images