Improved prediction through local illumination compensation.

Abstract

The current block to be coded and the candidate reference block templates may be determined from the same picture. The bitstream may include an indication of whether to use local illumination compensation (LIC) on the current block. Based on the indication, a difference between the current block and the candidate reference block may be determined. Based on the difference, a reference block may be determined, and a prediction of the current block may be made based on the indication and the difference between the current block and the reference block.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Patent Application No. 63 / 426,664, filed November 18, 2022, and U.S. Provisional Patent Application No. 63 / 463,186, filed May 1, 2023. The above-referenced applications are incorporated herein by reference in their entireties. [Background technology]

[0002] The current block may be predicted with or without local illumination compensation (LIC). The current block may be predicted based on a reference block and the difference between the templates of the current block and the reference block. Summary of the Invention

[0003] The following summary provides a simplified overview of certain features. It is not an extensive overview and is not intended to identify key or critical elements.

[0004] To reduce the amount of data transmitted, intra-block copying (IBC) may be used when encoding a current block in a picture. A difference between a template of the current block and a template of a reference block may be determined. The current block may be predicted using the reference block and the difference between the templates. Local illumination compensation (LIC), a technique used to compensate for illumination variations between the current block template and the reference block template, may be used during encoding to further reduce the amount of data transmitted. Prediction of the current block may be with or without LIC, although using LIC may lead to inaccurate predictions in some situations. The encoder may include an indication in the bitstream of whether LIC is used to predict the current block. By including this indication in the bitstream, a decoder may then determine whether to include LIC when predicting the current block.

[0005] These and other features and advantages are described in more detail below. [Brief explanation of the drawings]

[0006] Some examples of various embodiments of the present disclosure are described herein with reference to the drawings.

[0007] [Figure 1] 1 illustrates an exemplary video coding / decoding system. [Figure 2] 1 illustrates an exemplary encoder. [Figure 3] 1 illustrates an exemplary decoder. [Figure 4] 1 illustrates an exemplary quadtree division of a coding tree block (CTB). [Figure 5] 5 illustrates an exemplary quadtree corresponding to the exemplary quadtree division of the CTB of FIG. 4. [Figure 6] 1 illustrates exemplary binary and ternary tree partitioning. [Figure 7] An example of a combined quadtree and multitype tree partitioning of CTB is shown. [Figure 8] The tree corresponding to the combination of the CTB quadtree and multitype tree partitions shown in Fig. 7 is shown. [Figure 9] 10 illustrates an exemplary set of reference samples determined for intra prediction of a current block. [Figure 10A] 1 illustrates exemplary intra-prediction modes. [Figure 10B] 1 illustrates exemplary intra-prediction modes. [Figure 11] The current block and the corresponding reference sample are shown. [Figure 12] 10 illustrates an exemplary application of intra-prediction modes for prediction of a current block. [Figure 13A] 10 illustrates an example of inter prediction. [Figure 13B] 1 shows exemplary motion vectors. [Figure 14]1 illustrates an example of bi-prediction. [Figure 15A] 1 illustrates exemplary spatial candidate neighboring blocks for a current block. [Figure 15B] 1 illustrates an exemplary temporally co-located block of the current block. [Figure 16] 1 illustrates an embodiment of intra block copy (IBC) for coding. [Figure 17] 1 shows an example of a block vector prediction (BVP), a block vector (BV), and the corresponding block vector difference (BVD). [Figure 18] 1 shows an example of a current block, a current block template, a reference block, and a reference block template. [Figure 19] 1 shows exemplary blocks and their respective templates. [Figure 20] The current block, the reference block, and the corresponding BV are shown. [Figure 21] 1 shows an encoder and decoder configured for intra-block copy local illumination compensation (IBC-LIC) and intra-template matching prediction (intra-TMP). [Figure 22] 1 illustrates an exemplary method implemented by a decoder. [Figure 23A] 10 illustrates an embodiment of a method for determining the difference between the template of each candidate reference block and the current block based on the LIC flag. [Figure 23B] 10 illustrates an embodiment of a method for determining the difference between the template of each candidate reference block and the current block based on the LIC flag. [Figure 23C] 10 illustrates an embodiment of a method for determining the difference between the template of each candidate reference block and the current block based on the LIC flag. [Figure 24] 1 shows the method implemented by the encoder. [Figure 25] 1 illustrates an exemplary determination of magnitude and sign of a block vector difference (BVD) from multiple candidate BVDs. [Figure 26] 1 illustrates an embodiment of a context-based adaptive binary arithmetic coding (CABAC) encoder. [Figure 27A] An example of intra-block copy (IBC) is shown. [Figure 27B] 10 shows an exemplary BVD candidate for entropy coding the BVD magnitude symbol. [Figure 27C] 1 shows an exemplary table containing ingredients and costs of BVD candidates. [Figure 27D] 10 shows an embodiment of a decoder for determining the magnitude signal of the BVD. [Figure 28] The entropy coding method is shown below. [Figure 29] 1 shows a method for entropy decoding. [Figure 30] 1 illustrates an embodiment of a computer system. [Figure 31] 1 illustrates exemplary elements of a computing device that may be used to implement any of the various devices described herein. DETAILED DESCRIPTION OF THE INVENTION

[0008] The accompanying drawings and description provide examples. It should be understood that the examples shown in the drawings and / or description are non-exclusive, and that the features shown and described may be practiced in other examples. Examples are provided for the operation of video encoding and decoding systems that may be used in the field of video data storage and / or transmission / reception. More specifically, the techniques disclosed herein may relate to video compression used in encoding and / or decoding devices and / or systems.

[0009] A video sequence including multiple pictures / frames may be represented in a digital format for storage and / or transmission. Representing a video sequence in a digital format may require a large number of bits. The large data size that may be associated with a video sequence may require significant resources for storage and / or transmission. Video encoding may be used to compress the size of the video sequence for more efficient storage and / or transmission. Video decoding may be used to expand the compressed video sequence for display and / or other forms of consumption.

[0010] 1 shows an exemplary video coding / decoding system. The video coding / decoding system 100 may include a source device 102, a transmission medium 104, and a destination device 106. The source device 102 may encode a video sequence 108 into a bitstream 110 for more efficient storage and / or transmission. The source device 102 may store and / or transmit / transmit the bitstream 110 to the destination device 106 via the transmission medium 104. The destination device 106 may decode the bitstream 110 to display the video sequence 108. The destination device 106 may receive the bitstream 110 from the source device 102 via the transmission medium 104. The source device 102 and / or the destination device 106 may be any of a number of different devices (e.g., a desktop computer, a laptop computer, a tablet computer, a smartphone, a wearable device, a television, a camera, a video game console, a set-top box, a video streaming device, etc.).

[0011] Source device 102 may comprise one or more of a video source 112, an encoder 114, and / or an output interface 116 (e.g., for encoding video sequence 108 into bitstream 110). Video source 112 may provide and / or generate video sequence 108 based on the capture of natural and / or synthetically generated scenes. Synthetically generated scenes may be scenes including computer-generated graphics and / or screen content. Video source 112 may comprise a video capture device (e.g., a video camera), a video archive containing previously captured natural and / or synthetically generated scenes, a video feed interface for receiving captured natural and / or synthetically generated scenes from a video content provider, and / or a processor for generating synthetic scenes.

[0012] A video sequence, such as video sequence 108, may include a series of pictures (also referred to as frames). A video sequence may achieve the impression of motion based on the sequential presentation of the pictures of the video sequence using fixed or variable time intervals between pictures. A picture may include one or more sample arrays of intensity values. The intensity values ​​may be obtained (e.g., measured, determined, provided) at a series of regularly spaced locations within the picture. A color picture may (e.g., typically does) include a luminance sample array and two chrominance sample arrays. The luminance sample array may include intensity values ​​representing the brightness of the picture (e.g., the luma component, Y). The chrominance sample array may include intensity values ​​representing the blue and red components of the picture (e.g., the chroma components, Cb and Cr), respectively, separate from the brightness. Other color picture sample arrays may be possible based on different color schemes (e.g., a red, green, blue (RGB) color scheme). A pixel in a color picture can point to / contain / associate all intensity values ​​(e.g., luma component, chroma component) for a given location in the sample array used to represent the color picture. A monochrome picture may contain a single luma sample array. A pixel in a monochrome picture can point to / contain / associate an intensity value (e.g., luma component) at a given location in the single luma sample array used to represent the monochrome picture.

[0013] The encoder 114 may encode the video sequence 108 into the bitstream 110. The encoder 114 may apply / use one or more prediction techniques (e.g., to encode the video sequence 108) to reduce redundant information in the video sequence 108. The redundant information may include information that may be predicted at a decoder and that does not need to be transmitted to the decoder for accurate decoding of the video sequence 108. For example, the encoder 114 may apply spatial prediction (e.g., intra-frame or intra-prediction), temporal prediction (e.g., inter-frame or inter-prediction), inter-layer prediction, and / or other prediction techniques to reduce redundant information in the video sequence 108. The encoder 114 may, for example, divide a picture including the video sequence 108 into rectangular regions called blocks before applying one or more prediction techniques. The encoder 114 may then encode the blocks using one or more of the prediction techniques.

[0014] The encoder 114 may search for a block similar to a block to be coded in another picture (e.g., a reference picture) of the video sequence 108, for example, for temporal prediction. It may then predict the block to be coded using a block (e.g., a predictive block) determined during the search. The encoder 114 may form a predictive block based on data from reconstructed neighboring samples of a block to be coded within the same picture of the video sequence 108, for example, for spatial prediction. The reconstructed samples may be coded and then decoded samples. The encoder 114 may determine a prediction error (e.g., a residual) based on the difference between the block to be coded and the predictive block. The prediction error may represent non-redundant information that may be transmitted / transmitted to a decoder for accurate decoding of the video sequence 108.

[0015] Encoder 114 may apply a transform to the prediction errors (e.g., using a discrete cosine transform (DCT) or any other transform) to generate transform coefficients. Encoder 114 may form bitstream 110 based on the transform coefficients used to determine the prediction blocks and other information using / based on the prediction type, motion vectors, and prediction mode. Encoder 114 may, for example, perform one or more of quantization and entropy coding of the transform coefficients and / or other information used to determine the prediction blocks before forming bitstream 110. The quantization and / or entropy coding may further reduce the number of bits required to store and / or transmit video sequence 108.

[0016] The output interface 116 may be configured to write and / or store the bitstream 110 onto the transmission medium 104 for transmission to the destination device 106. The output interface 116 may be configured to transmit / transmit, upload, and / or stream the bitstream 110 to the destination device 106 via the transmission medium 104. The output interface 116 may comprise a wired and / or wireless transmitter configured to transmit / transmit, upload, and / or stream the bitstream 110 according to one or more proprietary, open source, and / or standardized communication protocols (e.g., Digital Video Broadcasting (DVB) standard, Advanced Television Systems Committee (ATSC) standard, Integrated Services Digital Broadcasting (ISDB) standard, Data Over Cable Service Interface Specification (DOCSIS) standard, 3rd Generation Partnership Project (3GPP) standard, Institute of Electrical and Electronics Engineers (IEEE) standard, Internet Protocol (IP) standard, Wireless Application Protocol (WAP) standard, and / or any other communication protocol).

[0017] The transmission medium 104 may include wireless, wired, and / or computer-readable media. For example, the transmission medium 104 may comprise one or more wires, cables, air interfaces, optical disks, flash memory, and / or magnetic memory. The transmission medium 104 may comprise one or more networks (e.g., the Internet) or file servers configured to store and / or transmit / transmit encoded video data.

[0018] Destination device 106 may decode bitstream 110 into video sequence 108 for display. Destination device 106 may comprise one or more of input interface 118, decoder 120, and / or video display 122. Input interface 118 may be configured to read bitstream 110 stored on transmission medium 104 by source device 102. Input interface 118 may be configured to receive, download, and / or stream bitstream 110 from source device 102 via transmission medium 104. Input interface 118 may comprise a wired and / or wireless receiver configured to receive, download, and / or stream bitstream 110 according to one or more proprietary, open source, standardized communication protocols, and / or any other communication protocol (e.g., as referenced herein).

[0019] Decoder 120 may decode video sequence 108 from encoded bitstream 110. Decoder 120 may generate predictive blocks for pictures of video sequence 108 in a manner similar to encoder 114, e.g., determine prediction errors for blocks for encoding video sequence 108. Decoder 120 may generate predictive blocks using / based on prediction types, prediction modes, and / or motion vectors received in bitstream 110. Decoder 120 may determine prediction errors using transform coefficients received in bitstream 110. Decoder 120 may determine prediction errors by weighting transform basis functions using the transform coefficients. Decoder 120 may combine the predictive blocks and prediction errors to decode video sequence 108. The video sequence 108 at destination device 106 may, or may not necessarily, be the same video sequence as transmitted, such as the video sequence 108 transmitted by source device 102. The decoder 120 may decode a video sequence that approximates the video sequence 108 due to, for example, lossy compression of the video sequence 108 by the encoder 114 and / or errors introduced into the encoded bitstream 110 during transmission to the destination device 106.

[0020] Video display 122 may display video sequence 108 to a user. Video display 122 may include a cathode ray tube (CRT) display, a liquid crystal display (LCD), a plasma display, a light emitting diode (LED) display, and / or any other display device suitable for displaying video sequence 108.

[0021] Video encoding / decoding system 100 is merely one example, and different video encoding / decoding systems and / or modified versions of video encoding / decoding system 100 may implement the methods and processes described herein. For example, video encoding / decoding system 100 may include other components and / or arrangements. Video source 112 may be external to source device 102. Video display device 122 may be external to destination device 106 or may be omitted entirely (e.g., if video sequence 108 is intended for consumption by a machine and / or storage device). Source device 102 may further include a video decoder, and destination device 104 may further include a video encoder. For example, source device 102 may be configured to further receive an encoded bitstream from destination device 106 to support bidirectional video transmission between the devices.

[0022] Encoder 114 and / or decoder 120 may operate according to one or more proprietary or industry video coding standards. For example, encoder 114 and / or decoder 120 may operate according to one or more proprietary, open source, and / or standardized protocols (e.g., International Telecommunication Union Telecommunication Standardization Sector (ITU-T) H.263, ITU-T H.264, and Moving Picture Expert Group (MPEG)-4 Visual (also known as Advanced Video Coding (AVC)), ITU-T H.265 and MPEG-H Part 2 (also known as High Efficiency Video Coding (HEVC)), ITU-T H.265 and MPEG-I Part 3 (also known as Versatile Video Coding (VVC)), WebM VP8 and VP9 codecs, and / or AOMedia Video 1 (AV1), and / or any other video coding protocol).

[0023] FIG. 2 shows an example encoder. As shown in FIG. 2, the encoder 200 may implement one or more processes described herein. The encoder 200 may encode a video sequence 202 into a bitstream 204 for more efficient storage and / or transmission. The encoder 200 may be implemented in the video coding / decoding system 100 (e.g., as encoder 114) as shown in FIG. 1 or in any computing, communication, or electronic device (e.g., a desktop computer, a laptop computer, a tablet computer, a smartphone, a wearable device, a television, a camera, a video game console, a set-top box, a video streaming device, etc.). The encoder 200 may comprise one or more of an inter-prediction unit 206, an intra-prediction unit 208, combiners 210 and 212, a transform and quantization unit (TR+Q) 214, an inverse transform and quantization unit (iTR+iQ) 216, an entropy coding unit 218, one or more filters 220, and / or a buffer 222.

[0024] The encoder 200 may divide a picture (e.g., a frame) of (e.g., including) the video sequence 202 into blocks and encode the video sequence 202 block by block. The encoder 200 may perform / apply a prediction technique on a block to be encoded using either an inter prediction unit 206 or an intra prediction unit 208. The inter prediction unit 206 may perform inter prediction by searching for a block similar to a block to be encoded in another reconstructed picture (e.g., a reference picture) of the video sequence 202. The reconstructed picture may be a coded and subsequently decoded picture. The block (e.g., a predictive block) determined during the search may then be used to predict the block to be coded to remove redundant information. The inter prediction unit 206 may determine the predictive block by exploiting temporal redundancy or similarity in scene content from picture to picture of the video sequence 202. For example, scene content between pictures of the video sequence 202 may be similar over time except for differences due to motion and / or affine transformation of screen content.

[0025] The intra prediction unit 208 may perform intra prediction by forming a predictive block based on data from reconstructed neighboring samples of a block encoded within the same picture of the video sequence 202. The reconstructed samples may be encoded and then decoded samples. The intra prediction unit 208 may determine the predictive block by exploiting spatial redundancy or similarity in scene content within a picture of the video sequence 202. For example, the texture of a region of scene content within a picture may be similar to the texture of the area immediately surrounding the region of scene content within the same picture.

[0026] The combiner 210 may determine a prediction error (e.g., a residual) based on the difference between the block to be coded and the prediction block. The prediction error may represent non-redundant information that can be sent / transmitted to a decoder for accurate decoding of the video sequence 202.

[0027] The transform and quantization unit (TR+Q) 214 may transform and quantize the prediction errors. The transform and quantization unit 214 may convert the prediction errors into transform coefficients, for example, by applying a DCT to reduce correlation information in the prediction errors. The transform and quantization unit 214 may quantize the coefficients by mapping the data of the transform coefficients to a set of predefined representative values. The transform and quantization unit 214 may quantize the coefficients to reduce irrelevant information in the bitstream 204. The irrelevant information may be information that can be removed from the coefficients without producing visible and / or perceptible distortion in the video sequence 202 after decoding (e.g., at a receiving device).

[0028] The entropy coding unit 218 may apply one or more entropy coding methods to the quantized transform coefficients to further reduce the bit rate. For example, the entropy coding unit 218 may apply context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), and / or syntax-based context-based binary arithmetic coding (SBAC). The entropy-coded coefficients may be packed to form the bitstream 204.

[0029] The inverse transform and quantization unit (iTR+iQ) 216 may inverse quantize and inverse transform the quantized transform coefficients to determine a reconstructed prediction error. The combiner 212 may combine the reconstructed prediction error with the prediction block to form a reconstructed block. The filter 220 may filter the reconstructed block using, for example, a deblocking filter and / or a sample adaptive offset (SAO) filter. The buffer 222 may store the reconstructed block for prediction of one or more other blocks in the same and / or different pictures of the video sequence 202.

[0030] The encoder 200 may further include an encoder control unit. The encoder control unit may be configured to control one or more units of the encoder 200 shown in FIG. 2. The encoder control unit may control one or more units of the encoder 200 so that the bitstream 204 may be generated in accordance with the requirements of one or more proprietary coding protocols, industry video coding standards, and / or any other video coding protocol. For example, the encoder control unit may control one or more units of the encoder 200 so that the bitstream 204 may be generated in accordance with one or more of ITU-T H.263, AVC, HEVC, VVC, VP8, VP9, ​​AV1, and / or any other video coding standard / format.

[0031] The encoder control unit may attempt to minimize (or reduce) the bitrate of bitstream 204 and / or maximize (or increase) the reconstructed video quality (e.g., within the constraints of a proprietary coding protocol, an industry video coding standard, and / or any other video coding protocol). For example, the encoder control unit may attempt to minimize or reduce the bitrate of bitstream 204 so that the reconstructed video quality does not fall below a certain level / threshold, and / or may attempt to maximize or increase the reconstructed video quality so that the bitrate of bitstream 204 does not exceed a certain level / threshold. The encoder control unit may determine / control one or more of: dividing a picture of the video sequence 202 into blocks; whether a block is inter predicted by the inter prediction unit 206 or intra predicted by the intra prediction unit 208; a motion vector for the inter prediction of the block; an intra prediction mode among multiple intra prediction modes for the intra prediction of the block; filtering performed by the filter 220; and / or one or more transform types and / or quantization parameters applied by the transform and quantization unit 214. The encoder control unit may determine / control one or more of the above based on a rate-distortion measurement for the block or picture being coded. The encoder control unit may determine / control one or more of the above to reduce the rate-distortion measurement for the block or picture being coded.

[0032] The prediction type (intra- or inter-prediction) used to code the block, the prediction information for the block (intra-prediction mode, motion vectors, etc., in the case of intra-prediction), and / or the transform and / or quantization parameters may be transmitted to entropy coding unit 218 for further compression (e.g., to reduce bitrate). The prediction type, prediction information, and / or the transform and / or quantization parameters may be packed with the prediction error to form bitstream 204.

[0033] Encoder 200 is merely one example, and encoders different from encoder 200 and / or improved versions of encoder 200 may implement the methods and processes described herein. For example, encoder 200 may include other components and / or arrangements. One or more of the components shown in FIG. 2 may optionally be included in encoder 200 (e.g., entropy coding unit 218 and / or filter 220).

[0034] FIG. 3 shows an exemplary decoder. The decoder 300 shown in FIG. 3 may implement one or more processes described herein. The decoder 300 may decode a bitstream 302 into a decoded video sequence 304 for display and / or some other form of consumption. The decoder 300 may be implemented in the video encoding / decoding system 100 of FIG. 1 and / or in a computing, communication, or electronic device (e.g., a desktop computer, a laptop computer, a tablet computer, a smartphone, a wearable device, a television, a camera, a video game console, a set-top box, and / or a video streaming device). The decoder 300 may comprise an entropy decoding unit 306, an inverse transform and quantization (iTR+iQ) unit 308, a combiner 310, one or more filters 312, a buffer 314, an inter prediction unit 316, and / or an intra prediction unit 318.

[0035] The decoder 300 may comprise a decoder control unit configured to control one or more units of the decoder 300. The decoder control unit may control one or more units of the decoder 300 such that the bitstream 302 is decoded in accordance with the requirements of one or more proprietary coding protocols, industry video coding standards, and / or any other communication protocol. For example, the decoder control unit may control one or more units of the decoder 300 such that the bitstream 302 is decoded in accordance with one or more of ITU-T H.263, AVC, HEVC, VVC, VP8, VP9, ​​AV1, and / or any other video coding standards / formats.

[0036] The decoder control unit may determine / control one or more of: whether a block is inter predicted by inter prediction unit 316 or intra predicted by intra prediction unit 318, a motion vector for inter prediction of the block, an intra prediction mode among multiple intra prediction modes for intra prediction of the block, the filtering performed by filter 312, and / or one or more inverse transform types and / or inverse quantization parameters applied by inverse transform and quantization unit 308. One or more of the control parameters used by the decoder control unit may be packed within the bitstream 302.

[0037] The entropy decoding unit 306 may entropy decode the bitstream 302. The inverse transform and quantization unit 308 may inverse quantize and / or inverse transform the quantized transform coefficients to determine a decoded prediction error. The combiner 310 may combine the decoded prediction error with a prediction block to form a decoded block. The prediction block may be generated by the intra prediction unit 318 or the inter prediction unit 316 (e.g., as described above with respect to the encoder 200 of FIG. 2). The filter 312 may filter the decoded block using, for example, a deblocking filter and / or a sample adaptive offset (SAO) filter. The buffer 314 may store the decoded block for prediction of one or more other blocks in the same and / or different pictures of the video sequence in the bitstream 302. As shown in FIG. 3, the decoded video sequence 304 may be output from the filter 312.

[0038] Decoder 300 is merely one example, and decoders different from decoder 300 and / or modified versions of decoder 300 may implement the methods and processes described herein. For example, decoder 300 may have other components and / or arrangements. One or more of the components shown in Figure 3 may optionally be included in decoder 300 (e.g., entropy decoding unit 306 and / or filter 312).

[0039] Although not shown in Figures 2 and 3, each of the encoder 200 and the decoder 300 may further include an intra block copy unit in addition to the inter prediction and intra prediction units. The intra block copy unit may be implemented / operate similarly to the inter prediction unit, but may predict blocks within the same picture. For example, the intra block copy unit may exploit repetitive patterns that appear in screen content. The screen content may include computer-generated text, graphics, animation, etc.

[0040] Video encoding and / or decoding may be performed on a block-by-block basis. The process of dividing a picture into blocks may be adaptive based on the content of the picture. For example, to improve coding efficiency, larger block divisions may be used in areas of a picture that have a higher level of homogeneity.

[0041] A picture (e.g., HEVC, or any other coding standard / format) may be divided into non-overlapping square blocks, which may be referred to as coding tree blocks (CTBs). A CTB may contain samples of a sample array. A CTB may have a size of 2n×2n samples, where n may be specified by parameters of the coding system. For example, n may be 4, 5, 6, or any other value. A CTB may have any other size. A CTB may be further divided by a recursive quadtree division into coding blocks (CBs) of half-vertical and half-horizontal size. A CTB may form the root of the quadtree. A CB that is not further divided as part of the recursive quadtree division may be referred to as a leaf CB of the quadtree, or otherwise may be referred to as a non-leaf CB of the quadtree. A CB may have a minimum size specified by parameters of the coding system. For example, a CB may have a minimum size of 4×4, 8×8, 16×16, 32×32, 64×64 samples, or any other minimum size. The CB may be further divided into one or more prediction blocks (PBs) to perform inter-prediction and / or intra-prediction. A PB may be a rectangular block of samples to which the same prediction type / mode may be applied. For transforms, the CB may be divided into one or more transform blocks (TBs). A TB may be a rectangular block of samples that may determine / indicate the applied transform size.

[0042] FIG. 4 shows an exemplary quadtree partitioning of a CTB. FIG. 5 shows a quadtree corresponding to the exemplary quadtree partitioning of the CTB 400 of FIG. 4. As shown in FIGS. 4 and 5, the CTB 400 may initially be partitioned into four CBs of semi-vertical and semi-horizontal size. Three of the CBs resulting from the first level partitioning of the CTB 400 may be leaf CBs. The three leaf CBs of the first level partitioning of the CTB 400 are labeled 7, 8, and 9, respectively, in FIGS. 4 and 5. The non-leaf CBs of the first level partitioning of the CTB 400 may be partitioned into four sub-CBs of semi-vertical and semi-horizontal size. Three of the sub-CBs resulting from the second level partitioning of the CTB 400 may be leaf CBs. The three leaf CBs of the second level partitioning of the CTB 400 are labeled 0, 5, and 6, respectively, in FIGS. 4 and 5. The non-leaf CBs of the second level partitioning of the CTB 400 may be partitioned into four leaf CBs of semi-vertical and semi-horizontal size. The four lobes CB can be labeled 1, 2, 3, and 4 in Figures 4 and 5, respectively.

[0043] The CTB 400 of FIG. 4 may be divided into ten leaf CBs, labeled 0 through 9, and / or any other number of leaf CBs. The ten leaf CBs may correspond to ten CB leaf nodes (e.g., the ten CB leaf nodes of quadtree 500, as shown in FIG. 5). In other embodiments, the CTB may be divided into a different number of leaf CBs. The resulting quadtree division of the CTB 400 may be scanned using a z-scan (e.g., left to right, top to bottom) to form a sequence order for encoding / decoding the CB leaf nodes. The numeric indicator (e.g., indicator, index) of each CB leaf node in FIGS. 4 and 5 may correspond to the sequence order for encoding / decoding. For example, CB leaf node 0 may be encoded / decoded first, and CB leaf node 9 may be encoded / decoded last. Although not shown in FIGS. 4 and 5, each CB leaf node may include one or more PBs and / or TBs.

[0044] Pictures in VVC (or in any other coding standard / format) can be partitioned in a similar manner (such as HEVC). A picture can first be partitioned into non-overlapping square CTBs. The CTBs can then be partitioned into half-vertical and half-horizontal sized CBs using recursive quadtree partitioning. The quadtree leaf nodes (e.g., in VVC) can be further partitioned into unequal sized CBs by binary or ternary tree partitioning (or any other partitioning).

[0045] FIG. 6 illustrates exemplary binary tree and ternary tree partitioning. Binary tree partitioning may divide a parent block in half either vertically 602 or horizontally 604. The resulting partitions may be half the size compared to the parent block. The resulting partitions may correspond to sizes less than and / or more than half the parent block size. Ternary tree partitioning may divide a parent block into three parts either vertically 606 or horizontally 608. FIG. 6 illustrates an example in which a middle partition may be twice the size of the other two end partitions in the ternary tree partitioning. In other examples, the partitions may be other sizes relative to each other and to the parent block. Binary tree and ternary tree partitioning are examples of multi-type tree partitioning. Multi-type tree partitioning may include dividing a parent block into other numbers of smaller blocks. A block partitioning strategy (e.g., in VVC) may be referred to as a combination of quadtree and multi-type tree partitioning (quadtree + multi-type tree partitioning) because binary tree and / or ternary tree partitioning is added to quadtree partitioning.

[0046] FIG. 7 shows an example of a combined quadtree and multitype tree partitioning of a CTB. FIG. 8 shows a tree corresponding to the combined quadtree and multitype tree partitioning of CTB 700 shown in FIG. 7. In both FIG. 7 and FIG. 8, the quadtree partitioning is shown with solid lines, and the multitype tree partitioning is shown with dashed lines. CTB 700 is shown with the same quadtree partitioning as CTB 400 illustrated in FIG. 4, and an explanation of the quadtree partitioning of CTB 700 is omitted. The quadtree partitioning of CTB 700 is merely an example, and the CTB may be quadtree partitioned in a manner different from CTB 700. Additional multitype tree partitioning of CTB 700 may be performed on the three leaf CBs shown in FIG. 4. The three leaf CBs of FIG. 4 shown in FIG. 7 as being further partitioned may be leaves CBs 5, 8, and 9. The three leaf CBs may be further partitioned using one or more binary and / or ternary tree partitioning.

[0047] Leaf CB5 in FIG. 4 may be split into two CBs based on a vertical binary tree division. The resulting two CBs may be leaf CBs labeled 5 and 6 in FIGS. 7 and 8, respectively. Leaf CB8 in FIG. 4 may be split into three CBs based on a vertical ternary tree division. Two of the resulting three CBs may be leaf CBs labeled 9 and 14 in FIGS. 7 and 8, respectively. The remaining non-leaf CBs may be initially split into two CBs based on a horizontal binary tree division. One of the two CBs may be the leaf CB labeled 10. The other of the two CBs may be further split into three CBs based on a vertical ternary tree division. The resulting three CBs may be leaf CBs labeled 11, 12, and 13 in FIGS. 7 and 8, respectively. Leaf CB9 in FIG. 4 may be split into three CBs based on a horizontal ternary tree division. Two of the three CBs may be leaf CBs labeled 15 and 19 in FIGS. 7 and 8, respectively. The remaining non-leaf CBs can be split into three CBs based on another horizontal ternary tree division, and all three resulting CBs can be leaf CBs, labeled 16, 17, and 18 in Figures 7 and 8, respectively.

[0048] Overall, the CTB 700 may be divided into 20 leaf CBs, labeled 0 through 19, respectively. The 20 leaf CBs may correspond to 20 leaf nodes (e.g., the 20 leaf nodes of the tree 800 shown in FIG. 8). The resulting combined quadtree and multitype tree division of the CTB 700 may be scanned using a z-scan (left to right, top to bottom) to form a sequence order for encoding / decoding the CB leaf nodes. The numeric labels of each CB leaf node in FIGS. 7 and 8 may correspond to the sequence order for encoding / decoding, with CB leaf node 0 being encoded / decoded first and CB leaf node 19 being encoded / decoded last. Note that, although not shown in FIGS. 7 and 8, each CB leaf node may include one or more PBs and / or TBs.

[0049] A coding standard / format (e.g., HEVC, VVC, or any other coding standard / format) may define various units (e.g., in addition to specifying various blocks (e.g., CTB, CB, PB, TB)). A block may include a rectangular area of ​​samples within a sample array. A unit may include collocated blocks of samples from different sample arrays (e.g., luma and chroma sample arrays) that form a picture, as well as syntax elements and prediction data for the block. A coding tree unit (CTU) may include collocated CTBs of different sample arrays and may form a complete entity in the encoded bitstream. A coding unit (CU) may include collocated CBs of different sample arrays and syntax structures used to code samples of the CBs. A prediction unit (PU) may include collocated PBs of different sample arrays and syntax elements used to predict the PBs. A transform unit (TU) may include TBs of different sample arrays and syntax elements used to transform the TBs.

[0050] A block may refer to any of a CTB, CB, PB, TB, CTU, CU, PU, ​​and / or TU (e.g., in the context of HEVC, VVC, or any other coding format / standard). A block may be used to refer to a similar data structure in the context of any video coding format / standard / protocol. For example, a block may refer to a macroblock in the AVC standard, a macroblock or sub-block in the VP8 coding format, a superblock or sub-block in the VP9 coding format, and / or a superblock or sub-block in the AV1 coding format.

[0051] Samples of a block to be coded (e.g., the current block) may be predicted from samples in columns immediately adjacent to the leftmost column of the current block and samples in rows adjacent to the top row of the current block, such as in intra-prediction. Samples from immediately adjacent columns and rows may collectively be referred to as reference samples. Each sample of the current block may be predicted by projecting the position of the sample in the current block in a given direction onto a point along the reference sample (e.g., in intra-prediction mode). If the projection does not fall directly on the reference sample, the sample may be predicted by interpolating between the two closest reference samples to the projection point. A prediction error (e.g., a residual) may be determined for the current block based on the difference between the predicted sample values ​​and the original sample values ​​of the current block.

[0052] Predicting samples and determining a prediction error based on a difference between the predicted sample and the original sample may be performed (e.g., in an encoder) for multiple different intra-prediction modes (e.g., including a non-directional intra-prediction mode). The encoder may select one of the multiple intra-prediction modes and its corresponding prediction error to encode the current block. The encoder may send an indication of the selected prediction mode and its corresponding prediction error to a decoder for decoding the current block. The decoder may decode the current block by predicting samples of the current block using the intra-prediction mode indicated by the encoder and / or combining the predicted samples with the prediction error.

[0053] 9 shows an example set of reference samples determined for intra-prediction of a current block. The current block 904 may correspond to a block to be coded and / or decoded. The current block 904 may correspond to block 3 of the split CTB 700 as shown in FIG. 7. As described herein, the numeric labels 0-19 of the blocks of the split CTB 700 may correspond to a sequence order for coding / decoding the blocks and may be used as in the example of FIG. 9.

[0054] The current block 904 may be samples of size w × h. The reference samples 902 may include 2 w samples (or any other number of samples) in a row adjacent to the top row of the current block 904, 2 h samples (or any other number of samples) in a column directly adjacent to the leftmost column of the current block 904, and an upper-left adjacent corner sample for the current block 904. The current block 904 may be square, such that w = h = s. In other embodiments, the current block need not be square, such that w ≠ h. Available samples from neighboring blocks of the current block 904 may be used to construct the set of reference samples 902. A sample may not be available to construct the set of reference samples 902, for example, if the sample is outside the picture of the current block, if the sample is part of a different slice from the current block (e.g., if the slice concept is used), and / or if the sample belongs to an inter-coded block and constrained intra prediction is indicated. For example, if constrained intra prediction is indicated, the intra prediction may not depend on the inter-predicted block.

[0055] Samples that may not be available for constructing the set of reference samples 902 may include samples within blocks that have not yet been coded and reconstructed at the encoder and / or decoded at the decoder based on the sequential order for encoding / decoding. Restricting such samples from inclusion in the set of reference samples 902 may allow for determining the same prediction result at both the encoder and the decoder. Samples from neighboring blocks 0, 1, and 2 may be available for constructing reference samples 902, considering that these blocks are coded and reconstructed at the encoder and decoded at the decoder before coding of the current block 904. Samples from neighboring blocks 0, 1, and 2 may be available for constructing reference samples 902, for example, if there are no other issues (e.g., as described above) that prevent the availability of samples from neighboring blocks 0, 1, and 2. A portion of reference samples 902 from neighboring block 6 may not be available due to the sequential order for encoding / decoding (e.g., because block 6 has not yet been coded and reconstructed at the encoder and / or decoded at the decoder based on the sequential order for encoding / decoding).

[0056] Unavailable samples from the reference samples 902 may be filled with one or more of the available reference samples 902. For example, the unavailable reference sample may be filled with the nearest available reference sample. The nearest available reference sample may be determined by moving clockwise from the position of the unavailable reference through the reference samples 902. The reference samples 902 may be filled with, for example, an intermediate value of the dynamic range in which the picture is coded when a reference sample is unavailable.

[0057] The reference samples 902 may be filtered based on the size of the current block 904 being coded and the applied intra-prediction mode. Figure 9 shows an example determination of reference samples for intra-prediction of a block. The reference samples may be determined in a manner different from that described above. For example, multiple reference lines may be used in other instances (e.g., in VVC).

[0058] The samples of the current block 904 may be intra predicted based on the reference sample 902, e.g., based on (e.g., after) determining and (optionally) filtering the reference sample. At least some (e.g., most) encoders / decoders may support multiple intra prediction modes according to one or more video coding standards. For example, HEVC supports 35 intra prediction modes, including planar mode, direct current (DC) mode, and 33 angular modes. VVC supports 67 intra prediction modes, including planar mode, DC mode, and 65 angular modes. Planar and DC modes may be used to predict smooth and gradually changing regions of a picture. Angular modes may be used to predict directional structure within a region of a picture. Any number of intra prediction modes may be supported.

[0059] 10A and 10B show exemplary intra prediction modes. FIG. 10A shows 35 intra prediction modes such as those supported by HEVC. The 35 intra prediction modes may be indicated / identified by indexes 0 through 34. Prediction mode 0 may correspond to planar mode. Prediction mode 1 may correspond to DC mode. Prediction modes 2 through 34 may correspond to angular modes. Prediction modes 2 through 18 may be referred to as horizontal prediction modes because the primary prediction source is horizontal. Prediction modes 19 through 34 may be referred to as vertical prediction modes because the primary prediction source is vertical.

[0060] FIG. 10B shows 67 intra prediction modes such as those supported by VVC. The 67 intra prediction modes may be indicated / identified by indexes 0 through 66. Prediction mode 0 may correspond to planar mode. Prediction mode 1 may correspond to DC mode. Prediction modes 2 through 66 may correspond to angular modes. Prediction modes 2 through 34 may be referred to as horizontal prediction modes because the primary prediction source is in the horizontal direction. Prediction modes 35 through 66 may be referred to as vertical prediction modes because the primary prediction source is in the vertical direction. Some of the intra prediction modes illustrated in FIG. 10B may be adaptively replaced by wide-angle directions because VVC blocks need not be square.

[0061] Figure 11 shows a current block and corresponding reference samples. In Figure 11, the current block 904 and reference samples 902 of Figure 9 are shown in a two-dimensional x, y plane, where the samples can be referenced as p[x][y]. To simplify the prediction process, the reference samples 902 can be located in two one-dimensional arrays. The reference samples 902 above the current block 904 can be located in a one-dimensional array ref1[x].

number

[0062] The reference samples 902 to the left of the current block 904 may be located in a one-dimensional array ref2[y].

number

[0063] The prediction process may include determining a predicted sample p[x][y] (e.g., a predicted value) at location [x][y] within the current block 904. In the planar mode, the sample at location [x][y] within the current block 904 may be predicted by determining / calculating the average of two interpolated values. The first of the two interpolated values ​​may be based on a horizontal linear interpolation at location [x][y] within the current block 904. The second of the two interpolated values ​​may be based on a vertical linear interpolation at location [x][y] within the current block 904. The predicted sample p[x][y] in the current block 904 may be determined / calculated as follows:

number

number

number

[0064] The sample at location [x][y] in the current block 904 may be predicted by the average of the reference samples 902, such as DC mode. The predicted sample p[x][y] in the current block 904 may be determined / calculated as follows:

number

[0065] The sample at location [x][y] within the current block 904 may be predicted by projecting location [x][y] onto a point on a horizontal or vertical line of samples that includes the reference sample 902, for example, for a given angular mode, in a direction specified by the given angular mode. The sample at location [x][y] may be predicted by interpolating between the two nearest reference samples to the projection point if the projection does not fall directly on the reference sample. The direction specified by the angular mode may be given by an angle φ defined with respect to the y-axis for vertical prediction modes (e.g., modes 19-34 of HEVC and modes 35-66 of VVC). The direction specified by the angular mode may be given by an angle φ defined with respect to the x-axis for horizontal prediction modes (e.g., modes 2-18 of HEVC and modes 2-34 of VVC).

[0066] FIG. 12 illustrates an exemplary application of intra prediction modes for predicting a current block. FIG. 12 specifically illustrates prediction of a sample at location [x][y] in a current block 904 for a vertical prediction mode 906. The vertical prediction mode 906 may be given by an angle φ with respect to the vertical axis. In the vertical prediction mode, the location [x][y] in the current block 904 may be projected to a point (e.g., a projection point) on the horizon of the reference sample ref1[x]. The reference sample 902 is only partially illustrated in FIG. 12 for ease of illustration. As illustrated in FIG. 12, the projection point on the horizon of the reference sample ref1[x] may not lie exactly on the reference sample. The predicted sample p[x][y] in the current block 904 may be determined / calculated by linearly interpolating between the two reference samples, for example, if the projection point is at a fractional sample position between two reference samples. The predicted sample p[x][y] may be determined / calculated as follows:

number

number

number

number

[0067] The location [x][y] of the sample in the current block 904 may be projected onto the vertical line of the reference sample ref2[y], such as for a horizontal prediction mode. The predicted sample p[x][y] for the horizontal prediction mode may be determined / calculated as follows:

number

number

number

number

[0068] The interpolation functions given by equations (7) and (10) may be implemented by an encoder and / or a decoder (e.g., encoder 200 of FIG. 2 and / or decoder 300 of FIG. 3). The interpolation functions may be implemented by finite impulse response (FIR) filters. For example, the interpolation functions may be implemented as a set of 2-tap FIR filters. The coefficients of the 2-tap FIR filters are (1-i f ) and i f The predicted samples p[x][y] in angular intra prediction may be calculated at some predefined level of sample accuracy (e.g., 1 / 32 sample accuracy, or accuracy defined by any other metric). For 1 / 32 sample accuracy, the set of 2-tap FIR interpolation filters may include up to 32 different 2-tap FIR interpolation filters, which are used to calculate the projection displacement i f In other embodiments, different levels of sample precision may be used.

[0069] FIR filters may be used to predict chroma samples and / or luma samples. For example, a 2-tap interpolating FIR filter may be used to predict chroma samples, and the same and / or a different interpolation technique / filter may be used for luma samples. For example, a 4-tap FIR filter may be used to determine predicted values ​​for luma samples. The coefficients of the 4-tap FIR filter may be (e.g., similar to a 2-tap FIR filter) i f For 1 / 32 sample accuracy, the set of 32 different 4-tap FIR filters may include up to 32 different 4-tap FIR filters, which can be determined based on the projection displacement i f One for each of the 32 possible values ​​of the fractional part of i. In other embodiments, different levels of sample precision may be used. The set of 4-tap FIR filters is stored in a look-up table (LUT) and f The predicted samples p[x][y] for the vertical prediction mode may be determined based on a 4-tap FIR filter as follows:

Number

Number

[0070] When the location [x][y] of the sample in the current block 904 to be predicted is projected onto a negative x coordinate, supplementary reference samples can be determined / constructed. The location [x][y] of the sample can be projected onto a negative x coordinate, for example, when a negative vertical prediction angle φ is used. The supplementary reference samples are determined / constructed by projecting the reference samples within the vertical line of the reference sample 902 using the negative vertical prediction angle φ

Number

Number

[0071] The encoder may determine / predict samples of a current block (e.g., current block 904) to be coded for multiple intra prediction modes (e.g., using one or more of the functions described herein). For example, the encoder may determine / predict samples of the current block for each of the 35 intra prediction modes in HEVC and / or the 67 intra prediction modes in VVC. For each applied intra prediction mode, the encoder may determine a corresponding prediction error for the current block based on the difference (e.g., sum of squared differences (SSD), sum of absolute differences (SAD), or sum of absolute transform differences (SATD)) between the predicted samples determined for the intra prediction mode and the original samples of the current block. The encoder may determine / select one of the intra prediction modes to code the current block based on the determined prediction error. For example, the encoder may determine / select one of the intra prediction modes that results in the smallest prediction error for the current block. The encoder may determine / select an intra-prediction mode to encode the current block based on a rate-distortion measure (e.g., a Lagrangian rate-distortion cost) determined using the prediction error. The encoder may send an indication of the determined / selected intra-prediction mode and its corresponding prediction error (e.g., a residual) to the decoder for decoding the current block.

[0072] The decoder may determine / predict samples of a current block (e.g., current block 904) to be decoded for an intra-prediction mode. For example, the decoder may receive an indication of an intra-prediction mode (e.g., an angular intra-prediction mode) from the encoder for the current block. The decoder may construct a set of reference samples and perform intra-prediction based on the intra-prediction mode indicated by the encoder for the current block in a similar manner (e.g., as described above for the encoder). The decoder may add predicted values ​​of the samples (e.g., determined based on the intra-prediction mode) of the current block to the residual of the current block to reconstruct the current block. The decoder does not need to receive an indication of the angular intra-prediction mode from the encoder for the current block. The decoder may determine the intra-prediction mode based on, for example, other criteria. Although various embodiments herein correspond to intra-prediction modes in HEVC and VVC, the methods, devices, and systems described herein may be applied / used for other intra-prediction modes (e.g., as used in other video coding standards / formats such as VP8, VP9, ​​AV1, etc.).

[0073] Intra prediction may exploit correlation between spatially adjacent samples in the same picture of a video sequence to enhance video compression. Inter prediction is another coding tool that can be used to implement video compression. Inter prediction may exploit time-domain correlation between sample blocks in different pictures of a video sequence. For example, an object may be visible across multiple pictures of a video sequence. The object may move (e.g., with some translational and / or affine motion) or remain stationary across multiple pictures. A current block of samples in a current picture to be encoded may have / be associated with a corresponding block of samples in a previously decoded picture. The corresponding block of samples may accurately predict the current block of samples. The corresponding block of samples may be displaced from the current block of samples due, for example, to an object represented by both blocks moving across the respective pictures of the block. The previously decoded picture may be a reference picture. The corresponding block of samples in the reference picture may be a reference block for motion-compensated prediction. The encoder may use block matching techniques to estimate the displacement (or motion) of an object and / or determine a reference block in a reference picture.

[0074] The encoder may determine a difference between the current block and a prediction for the current block. The encoder may determine the difference, for example, based on / after determining / generating a prediction for the current block (e.g., using inter-prediction). The difference may be a prediction error and / or a residual. The encoder may store and / or transmit (e.g., signal) the bitstream, the prediction error, and / or other related prediction information. The prediction error and / or other related prediction information may be used for decoding and / or other forms of consumption. The decoder may decode the current block by predicting samples of the current block (e.g., by using the related prediction information) and combining the predicted samples with the prediction error.

[0075] FIG. 13A shows an example of inter prediction. Inter prediction may be performed on a current block 1300 of a current picture 1302 to be encoded. An encoder (e.g., encoder 200, as shown in FIG. 2) may perform inter prediction to determine and / or generate a reference block 1304 in a reference picture 1306. The reference block 1304 may be used to predict the current block 1300. The reference picture (e.g., reference picture 1306) may be a previously decoded picture available at the encoder and / or decoder. The availability of a previously decoded picture may depend / be based on whether a previously decoded picture is available in a decoded picture buffer when the current block 1300 is encoded and / or decoded. The encoder may search one or more reference pictures 1306 for a block that is similar (or substantially similar) to the current block 1300. The encoder may determine a best matching block from the blocks tested during the search process. The best matching block may be the reference block 1304. The encoder may determine that the reference block 1304 is the best matching reference block based on one or more cost criteria. The one or more cost criteria may include a rate-distortion criterion (e.g., a Lagrangian rate-distortion cost). The one or more cost criteria may also be based on differences (e.g., SSD, SAD, and / or SATD) between predicted samples of the reference block 1304 and the original samples of the current block 1300.

[0076] The encoder may search for the reference block 1304 within a reference region (e.g., search range 1308). The reference region (e.g., search range 1308) may be positioned around a co-located position (or block) 1310 of the current block 1300 in the reference picture 1306. The co-located block 1310 may have the same position in the reference picture 1306 as the current block 1300 in the current picture 1302. The reference region (e.g., search range 1308) may extend at least partially outside the reference picture 1306. A fixed boundary extension may be used, for example, when the reference region (e.g., search range 1308) extends outside the reference picture 1306. A fixed boundary extension may be used so that values ​​of samples in a row or column of the reference picture 1306 that are immediately adjacent to a portion of the reference region (e.g., search range 1308) that extends outside the reference picture 1306 can be used for the location of samples outside the reference picture 1306. A subset of, or all of, the potential locations within the reference region (e.g., search range 1308) may be searched for the reference block 1304. The encoder may utilize one or more search implementations to determine and / or generate the reference block 1304. For example, the encoder may determine a set of candidate search locations based on motion information of blocks (e.g., motion vectors 1312) that neighbor the current block 1300.

[0077] One or more reference pictures may be searched by the encoder during inter-prediction to determine and / or generate a best-matching reference block. The reference pictures searched by the encoder may be included in (e.g., added to) one or more reference picture lists. For example, in HEVC and VVC (and / or in one or more other communication protocols), two reference picture lists (e.g., reference picture list 0 and reference picture list 1) may be used. A reference picture list may contain one or more pictures. A reference picture 1306 of a reference block 1304 may be indicated by a reference index that points to a reference picture list that contains the reference picture 1306.

[0078] Figure 13B shows an example motion vector. The displacement between the reference block 1304 and the current block 1300 may be interpreted as an estimate of the motion between the reference block 1304 and the current block 1300 across their respective pictures. The displacement may be represented by a motion vector 1312. For example, the motion vector 1312 may be indicated by a horizontal component (MVx) and a vertical component (MVy) relative to the position of the current block 1300. A motion vector (e.g., the motion vector 1312) may have fractional or integer resolution. A motion vector with fractional resolution may point between two samples in the reference picture to provide a better estimate of the motion of the current block 1300. For example, the motion vector may have ½, ¼, ⅛, ⅙6, ⅛, or any other fractional sample resolution. Interpolation between two samples at integer positions may be used to generate a reference block and its corresponding sample at a fractional position, for example, if the motion vector points to a non-integer sample value in the reference picture. The interpolation may be performed by a filter with two or more taps.

[0079] The encoder may determine a difference (e.g., a corresponding sample-by-sample difference) between the reference block 1304 and the current block 1300. The encoder may determine the difference between the reference block 1304 and the current block 1300, for example, based on / after the reference block 1304 was determined and / or generated using inter-prediction for the current block 1300. The difference may be a prediction error and / or a residual. The encoder may store and / or transmit (e.g., a signal) within / via the bitstream, the prediction error and / or associated motion information. The prediction error and / or associated motion information may be used for decoding (e.g., decoding the current block 1300) and / or other forms of consumption. The motion information may include a motion vector 1312 and / or a reference indicator / index. The reference indicator may point to a reference picture 1306 within a reference picture list. The motion information may include an indication of the motion vector 1312 and / or an indication of the reference index. The reference index may point to a reference picture 1306 in a reference picture list. The decoder may decode the current block 1300 by determining and / or generating a reference block 1304. The decoder may determine and / or generate the reference block 1304 based on, for example, a prediction error and / or associated motion information. The reference block 1304 may correspond to / form (e.g., be considered as) a prediction of the current block 1300. The decoder may decode the current block 1300 based on combining the prediction with the prediction error.

[0080] As shown in Figure 13A, inter prediction may be performed using one reference picture 1306 as the source of prediction for a current block 1300. Inter prediction based on prediction of a current block using a single picture may be referred to as uni-prediction.

[0081] Inter-prediction of the current block is obtained based on two pictures using bi-prediction. Bi-prediction may be useful, for example, when a video sequence includes fast motion, camera pans, zooms, and / or scene changes. Bi-prediction may also be useful for capturing a fade-out of one scene or a fade-out from one scene to another, where two pictures may be effectively displayed simultaneously at different levels of intensity.

[0082] One or both of uni-prediction and bi-prediction may be available / used to perform inter-prediction (e.g., an encoder and / or a decoder). Performing a particular type of inter-prediction (e.g., uni-prediction and / or bi-prediction) may depend on the slice type of the current block. For example, for a P slice, only uni-prediction may be available / used to perform inter-prediction. For a B slice, either uni-prediction or bi-prediction may be available / used to perform inter-prediction. The encoder may determine and / or generate a reference block for predicting the current block, for example, from reference picture list 0 if the encoder uses uni-prediction. The encoder may determine and / or generate a first reference block for predicting the current block from reference picture list 0, and a second reference block for predicting the current block, for example, from reference picture list 1 if the encoder uses bi-prediction.

[0083] Figure 14 shows an example of bi-prediction. Two reference blocks 1402 and 1404 may be used to predict a current block 1400. Reference block 1402 may be in one reference picture of reference picture list 0 or reference picture list 1. Reference block 1404 may be in another reference picture of reference picture list 0 or reference picture list 1. As shown in Figure 14, reference block 1402 may be in a first picture that precedes (e.g., temporally) the current picture of current block 1400, and reference block 1404 may be in a second picture that follows (e.g., temporally) the current picture of current block 1400. The first picture may precede the current picture in terms of picture order count (POC). The second picture may follow the current picture in terms of POC. The reference pictures may both precede or both follow the current picture in terms of POC. The POC may be / indicate the order in which pictures are output (e.g., from a decoded picture buffer). The POC may be / indicate the order in which pictures are approximately intended to be displayed. Pictures that are output may not necessarily be displayed, but may undergo different processing and / or consumption (e.g., transcoding). Two reference blocks determined and / or generated using / for bi-prediction may correspond to (e.g., be included in) the same reference picture. A reference picture may be included in both reference picture list 0 and reference picture list 1, for example, if the two reference blocks correspond to the same reference picture.

[0084] Configurable weight and / or offset values ​​may be applied to one or more inter-prediction reference blocks. The encoder may enable the use of weighted prediction using a flag in a picture parameter set (PPS). The encoder may transmit / signal the weight and / or offset parameters in the slice segment header of the current block 1400. Different weight and / or offset parameters may be transmitted / signaled for the luma and / or chroma components.

[0085] The encoder may use inter prediction to determine and / or generate reference blocks 1402 and 1404 for the current block 1400. The encoder may determine a difference between the current block 1400 and each of the reference blocks 1402 and 1404. The difference may be a prediction error or a residual. The encoder may store and / or transmit / signal the prediction errors and / or their respective associated motion information in / via the bitstream. The prediction errors and their respective associated motion information may be used for decoding and / or other forms of consumption. The motion information for the reference block 1402 may include a motion vector 1406 and / or a reference indicator / index. The reference indicator may point to a reference picture for the reference block 1402 in a reference picture list. The motion information for the reference block 1402 may include an indication of the motion vector 1406 and / or an indication of a reference index. The reference index may point to a reference picture for the reference block 1402 in a reference picture list.

[0086] The motion information for the reference block 1404 may include a motion vector 1408 and / or a reference index / indicator. The reference indicator may point to a reference picture for the reference block 1408 in a reference picture list. The motion information for the reference block 1404 may include an indication of the motion vector 1408 and / or an indication of a reference index. The reference index may point to a reference picture for the reference block 1404 in a reference picture list.

[0087] A decoder may decode current block 1400 by determining and / or generating reference blocks 1402 and 1404. The decoder may determine and / or generate reference blocks 1402 and 1404, for example, based on a prediction error and / or associated motion information of reference blocks 1402 and 1404, respectively. Reference blocks 1402 and 1404 may correspond to / form (e.g., be considered) a prediction of current block 1400. The decoder may decode current block 1400 based on combining the prediction with the prediction error.

[0088] The motion information may be predictively coded, for example, before being stored and / or transmitted / signaled in / via a bitstream (e.g., in HEVC, VVC, and / or other video coding standards / formats / protocols). The motion information for a current block may be predictively coded based on the motion information of one or more blocks neighboring the current block. The motion information of neighboring blocks may often be correlated with the motion information of the current block because the motion of an object represented in the current block is often the same as (or similar to) the motion of an object in the neighboring block. Motion information prediction techniques may include advanced motion vector prediction (AMVP) and / or inter-prediction block merging. An encoder (e.g., encoder 200, as shown in FIG. 2) may code a motion vector. The encoder may code the motion vector (e.g., using AMVP) as the difference between the motion vector of the current block being coded and a motion vector predictor (MVP). The encoder may determine / select an MVP from a list of candidate MVPs. The candidate MVP may be / correspond to previously decoded motion vectors of neighboring blocks in the current picture of the current block and / or blocks collocated or near the current block in other reference pictures. The encoder and / or decoder may generate and / or determine the list of candidate MVPs.

[0089] The encoder may determine / select an MVP from a list of candidate MVPs. The encoder may send / signal an indication of the selected MVP and / or motion vector differential (MVD) in / via the bitstream. The encoder may indicate the selected MVP in the bitstream using an index / indicator. The index may indicate the selected MVP in a list of candidate MVPs. The MVD may be determined / calculated based on the difference between the motion vector of the current block and the selected MVP. For example, for a motion vector indicating a position (e.g., represented by a horizontal component (MVx) and a vertical component (MVy)) relative to the position of the current block being coded, the MVD may be represented by two components MVD_x and MVD_y. MVD_x and MVD_y may be determined / calculated as follows:

number

number

[0090] MVDx and MVDy may represent the horizontal and vertical components of the MVD, respectively. MVPx and MVPy may represent the horizontal and vertical components of the MVP, respectively. A decoder (e.g., decoder 300 as shown in FIG. 3) may decode a motion vector by adding the MVD to an MVP indicated in / via the bitstream. The decoder may decode a current block by determining and / or generating a reference block. The decoder may determine and / or generate a reference block, for example, based on a decoded motion vector. The reference block may correspond to / form (e.g., be considered as) a prediction of the current block. The decoder may decode the current block by combining the prediction with a prediction error.

[0091] A list of candidate MVPs for AMVP (e.g., in HEVC, VVC, and / or one or more other communication protocols) may include two or more candidates (e.g., candidate A and candidate B). Candidate A and candidate B may include up to two (or any other number) spatial candidate MVPs determined or derived from five (or any other number) spatially neighboring blocks of the current block being coded, one (or any other number) temporal candidate MVPs determined or derived from two (or any other number) temporally co-located blocks (e.g., when both of the two spatial candidate MVPs are unavailable or are identical), and / or a zero motion vector candidate MVP (e.g., when one or both of the spatial candidate MVP or the temporal candidate MVP are unavailable). Other numbers of spatial candidate MVPs, spatially neighboring blocks, temporal candidate MVPs, and / or temporally co-located blocks may also be used in the list of candidate MVPs.

[0092] Figure 15A shows spatial candidate neighboring blocks for a current block. For example, five (or any other number) spatial candidate neighboring blocks may be located relative to the current block 1500 being coded. The five spatial candidate neighboring blocks may be A0, A1, B0, B1, and B2. Figure 15B shows temporally co-located blocks for the current block. For example, two (or any other number) temporally co-located blocks may be located relative to the current block 1500. The two temporally co-located blocks may be C0 and C1. The two temporally co-located blocks may be in one or more reference pictures that may be different from the current picture of the current block 1500.

[0093] An encoder (e.g., encoder 200 as shown in FIG. 2) may code motion vectors using inter-prediction block merging (e.g., merge mode). An encoder (e.g., using merge mode) may reuse the same motion information of a neighboring block (e.g., one of neighboring blocks A0, A1, B0, B1, and B2) for inter prediction of the current block. An encoder (e.g., using merge mode) may reuse the same motion information of a temporally co-located block (e.g., one of temporally co-located blocks C0 and C1) for inter prediction of the current block. MVD does not need to be transmitted (e.g., indicated or signaled) for the current block because the same motion information as that of the neighboring or temporally co-located block can be used for the current block (e.g., at the encoder and / or decoder). Because MVD does not need to be indicated for the current block, signaling overhead for transmitting / signaling motion information of the current block may be reduced. The encoder and / or decoder may generate a candidate list of motion information from neighboring blocks or temporally co-located blocks of the current block (e.g., in a manner similar to AMVP). The encoder may decide to use (e.g., inherit) motion information of one neighboring block or one temporally co-located block in the candidate list to predict motion information of the current block being coded. The encoder may signal / send an indication of the determined motion information from the candidate list in / via the bitstream. For example, the encoder may signal / send an indicator / index. The index may indicate the determined motion information in the list of candidate motion information. The encoder may signal / send an index to indicate the determined motion information.

[0094] A list of candidate motion information for merge mode (e.g., in HEVC, VVC, or any other coding format / standard / protocol) may include up to four (or any other number) spatial merge candidates derived / determined from five (or any other number) spatially neighboring blocks (e.g., as shown in Figure 15A), one (or any other number) temporal merge candidates derived from two (or any other number) temporally co-located blocks (e.g., as shown in Figure 15B), and / or additional merge candidates including both prediction candidates and zero motion vector candidates. The spatially neighboring blocks and temporally co-located blocks used for merge mode may be the same as the spatially neighboring blocks and temporally co-located blocks used for AMVP.

[0095] Inter prediction may be implemented in other ways and variations than those described herein. For example, motion information prediction techniques other than AMVP and merge mode may be used. While various embodiments herein correspond to inter prediction modes such as those used in HEVC and VVC, the methods, devices, and systems described herein may be applied / used with other inter prediction modes (e.g., as used in other video coding standards / formats such as VP8, VP9, ​​AV1, etc.). History-based motion vector prediction (HMVP), combined intra / inter prediction mode (CIIP), and / or merge mode with motion vector differential (MMVD) (e.g., as described in VVC) may also be implemented / used and are within the scope of this disclosure.

[0096] Block matching may be used (e.g., in inter prediction) to determine a reference block in a picture different from the picture of the current block being coded. Block matching may be used to determine a reference block in the same picture as the picture of the current block being coded. A reference block in the same picture as the current block determined using block matching may often not accurately predict the current block (e.g., in the case of camera-captured video). Prediction accuracy for screen content video is not similarly affected, for example, when a reference block in the same picture as the current block is used for coding. Screen content video may include, for example, computer-generated text, graphics, animation, etc. Screen content video may (e.g., often does) include repetitive patterns (e.g., repetitive patterns of text and / or graphics) within the same picture. Using a reference block (e.g., determined using block matching) in the same picture as the current block being coded may provide efficient compression for screen content video.

[0097] A prediction technique may be used (e.g., in HEVC, VVC, and / or any other coding standard / format / protocol) to exploit correlation between blocks of samples within the same picture (e.g., of screen content video). The prediction technique may be intra block copying (IBC) or current picture reference (CPR). The encoder may apply / use a block matching technique (e.g., similar to inter prediction) to determine a displacement vector (e.g., block vector (BV)). The BV may indicate the relative position of a reference block that best matches the current block from the position of the current block (e.g., according to intra block compensated prediction). For example, the relative position of the reference block may be the relative position of the upper left corner (or any other point / sample) of the reference block. The BV may indicate the relative displacement from the current block to the reference block that best matches the current block. The encoder may determine the best matching reference block from the blocks tested during the search process (e.g., in a manner similar to that used for inter prediction). The encoder may determine that the reference block is the best matching reference block based on one or more cost criteria. The one or more cost criteria may include a rate-distortion criterion (e.g., a Lagrangian rate-distortion cost). The one or more cost criteria may be based, for example, on one or more differences (e.g., differences determined based on SSD, SAD, SATD, and / or a hash function) between predicted samples of the reference block and original samples of the current block. The reference block may correspond to / include a previously decoded block of samples of the current picture. The reference block may include a decoded block of samples of the current picture before being processed by an in-loop filtering operation (e.g., deblocking and / or SAO filtering).

[0098] Figure 16 shows an example of an IBC for encoding. The exemplary IBC shown in Figure 16 may correspond to screen content. The rectangular portion / section with the arrow starting at their boundary may be the current block to be encoded. The rectangular portion / section to which the arrow points may be a reference block for predicting the current block.

[0099] A reference block may be determined and / or generated for the current block for IBC. The encoder may determine a difference between the reference block and the current block (e.g., a corresponding sample-by-sample difference). The difference may be a prediction error or a residual. The encoder may store and / or transmit / signal the prediction error and / or associated prediction information in / via the bitstream. The prediction error and / or associated prediction information may be used for decoding and / or other forms of consumption. The prediction information may include a BV. The prediction information may include an indication of the BV. A decoder (e.g., decoder 300 as shown in FIG. 3) may decode the current block by determining and / or generating a reference block. The decoder may determine and / or generate the current block, for example, based on the prediction information (e.g., BV). The reference block may correspond to / form (e.g., be considered as) a prediction of the current block. The decoder may decode the current block by combining the prediction with the prediction error.

[0100] The BVs may be predictively coded (e.g., in HEVC, VVC, and / or any other coding standard / format / protocol) before being stored and / or transmitted / signaled in / via a bitstream. The BVs for a current block may be predictively coded based on the BVs of one or more blocks neighboring the current block. For example, the encoder may predictively code the BVs using merge mode (e.g., in a manner similar to that described herein for inter prediction), AMVP (e.g., as described herein for inter prediction), or an AMVP-like technique. The AMVP-like technique may be BV prediction and differential coding (or AMVP for IBC).

[0101] An encoder that performs BV prediction and coding (e.g., encoder 200 as shown in FIG. 2) may code the BV as the difference between the BV of the current block being coded and a block vector predictor (BVP). The encoder may select / determine a BVP from a list of candidate BVPs. The candidate BVPs may include / correspond to previously decoded BVs of neighboring blocks in the current picture of the current block. The encoder and / or decoder may generate or determine the list of candidate BVPs.

[0102] The encoder may signal an indication of the selected BVP and / or BV differential (BVD) in the bitstream. The encoder may signal an indication of the BVP and / or BV differential (BVD) in the bitstream, for example, if the encoder selects a BVP from a list of candidate BVPs. Figure 17 shows an example of a BVP, a BV, and a corresponding BVD (e.g., BVP 1706, BVD 1708, and BV 1710). The encoder may indicate the selected BVP in the bitstream by an index that points into the list of candidate BVPs. The BVD may be calculated. The BVD may be calculated based on the difference between the BV of the current block and the selected BVP. The BVD may be represented by two components, for example, for the BV, a horizontal component (BV) relative to the position of the current block being coded. x ) and vertical component (BV y ) The BVD can be represented by two components calculated as follows:

number

number

[0103] In HEVC and VVC, the list of candidate BVPs may include, for example, two candidates referred to as candidates A and B. Candidates A and B may include up to two spatial candidate BVPs derived from the five spatial neighboring blocks of the current block being coded, or one or more of the last two coded BVPs if no spatial neighboring candidates are available. The spatial neighboring candidates may not be available, for example, because they are coded in intra or inter mode. The locations of the five spatial candidate neighboring blocks relative to the current block coded using IBC may be the same as those shown in Figure 15A for inter prediction. The five spatial candidate neighboring blocks are denoted as A0, A1, B0, B1, and B2, respectively.

[0104] Intra prediction modes may enable a reduction in the amount of bits that can be transmitted (e.g., transmitted) from an encoder to a decoder. For example, intra template match prediction (intra-TMP) may enable a reduction in the amount of bits that can be transmitted (e.g., transmitted) from an encoder to a decoder. In intra-TMP mode, the best reference block (e.g., the smallest difference or smallest error between the current block template and the reference block template) from the reconstructed portion of the current frame may be used as the reference block. The reconstructed portion of the current frame may have an L-shaped template that can match the current template (i.e., the L-shaped template of the current block). Sum of absolute differences (SAD) may be used as a cost function in determining a template that may be most similar to the current template. The decoder may search for the template with the smallest SAD value relative to the current one. The decoder may use the corresponding block as the predicted block. SAD may be used as an exemplary difference calculation technique, although other difference calculation techniques may be used instead of SAD (e.g., sum of squared errors (SSE), sum of transformed difference values ​​(SATD), mean-removed SAD (MR-SAD)). The SATD may include a Hadamard transform that may be used to differencing, and the absolute values ​​of the resulting transform coefficients may be summed. The encoder may search for the template that is most similar to the current template in the reconstructed portion of the current frame. The encoder may use the corresponding block as the prediction block, for example, for a predefined search range. The encoder may signal the use of this mode to the decoder, and the same prediction operation may be performed on the decoder side.

[0105] The prediction signal may be generated by matching the L-shaped template of the current block with the L-shaped template of another block. The prediction signal may be generated by matching the L-shaped template of the current block with the L-shaped template of another block within a predefined search area, for example, within the current frame. The predefined area may be the area in which all blocks are reconstructed. The predefined area may include the entire reconstructed region of the current frame. Alternatively, the predefined search area may be limited to a subset of already decoded blocks of the current frame. Intra-TMP is described in the Enhanced Compression Model (ECM) software algorithm, which is currently being coordinated exploratory research by the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC MPEG Joint Video Exploration Team (JVET) as a potential enhancement video coding technique beyond the capabilities of VVC.

[0106] Figure 18 shows an example of a current block 1802, a current block template 1804, a reference block 1806, and a reference block template 1808. A "template" of a block, as the term is used herein, may be formed by adjacent samples to the left and above the block, as shown in Figure 18. A template of a block may consist of adjacent samples adjacent to the left boundary of the block and adjacent samples adjacent to the top boundary of the block. The method is not limited to a particular number of samples in the template.

[0107] Local illumination compensation may be one technique for improving motion compensation in VVC. Local illumination compensation (LIC) may be a prediction technique for modeling local illumination variation between a current block and its predicted block as a function of the local illumination variation between the current block template and a reference block template. The parameters of the LIC function may be represented by a scale α and an offset β of a linear equation, i.e., α*p[x]+β, to compensate for illumination changes, where p[x] is a reference sample pointed to by a displacement vector at location x on the reference picture (e.g., a motion vector (MV) in inter prediction or a block vector (BV) in intra prediction). Because the parameters α and β may be derived based on the current block template and the reference block template, no signaling overhead is required for them, except that a LIC flag may be signaled to indicate the use of LIC.

[0108] Applying LIC to a block involves adjusting the predicted samples (e.g., samples of the predicted block) by multiplying the predicted samples (e.g., sample values) by α (e.g., the value of α) and adding β (e.g., the value of β) according to the linear equation described above to compensate for local illumination differences. The parameters α and β may be derived from samples in templates of the current block and reference block, for example, by using a least-squares method. The parameters α and β may be derived using all, a subset, or multiple subsets of the samples in the template.

[0109] Figure 19 shows exemplary blocks and their respective templates. Figure 19 shows examples of a current block and samples, and a reference block and reference samples. On the left side of Figure 19, a current block 1902 (e.g., a current CU) and samples 1906 (e.g., neighboring samples) in a template 1904 of the current block are shown as empty circles. The reference samples (e.g., neighboring samples) may be samples adjacent to the left boundary (e.g., boundary) and the top boundary of the current block. The reference samples may belong to previously reconstructed neighboring blocks of the current block. On the right side of Figure 19, a reference block, e.g., a reference block used to predict the current block, and reference samples (e.g., neighboring samples) of the reference block are shown as empty circles. The reference samples (e.g., neighboring samples of the reference block) may be samples adjacent to the boundaries (e.g., left and top boundaries) of the reference block 1908. The neighboring samples of the reference block 1908 may have the same relative positions with respect to the reference block as the neighboring samples of the current block. The relative positions of adjacent samples in the template 1910 of the reference block and the current block match. The LIC parameter calculation may involve only a subset of samples. LIC has been described for inter-prediction in the Enhanced Compression Model (ECM) software algorithm, and is currently being coordinated exploratory research by the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC MPEG Joint Video Exploration Team (JVET) as a potential enhancement video coding technique beyond the capabilities of VVC.

[0110] The local illumination variation between a current block and its predicted block may be modeled as a linear equation. The local illumination variation between a current block and its predicted block may be modeled as a linear equation, for example, when IBC-LIC is used in the current block (e.g., CU). The parameters of the linear equation may be derived in a manner similar to LIC for inter prediction, where, for example, the reference block 1908 (shown in FIG. 19) may be from a different reference picture than that of the current block 1902, as described herein with respect to FIG. 14. IBC-LIC may be used with IBC AMVP mode and IBC merge mode. An IBC-LIC flag may indicate the use of IBC-LIC for IBC AMVP mode. The IBC-LIC flag may be estimated from merge candidates for the IBC merge mode.

[0111] IBC may be used, for example, when encoding a current block in a picture, to reduce the amount of data transmitted (e.g., transmitted) from the encoder to the decoder in a bitstream. Furthermore, the encoder may use LIC on the reference block to further reduce the transmitted (e.g., transmitted) data by compensating for illumination differences between the current block and the reference block, thereby reducing the residual that needs to be encoded. A difference calculation to determine the difference between a template of a current block and a template of a reference block, as determined by sum of absolute differences (SAD), may not accurately measure similarity. A difference calculation to determine the difference between a template of a current block and a template of a reference block, as determined by sum of absolute differences (SAD), may not accurately measure similarity, for example, when a picture and / or block is encoded using IBC with LIC (IBC-LIC). This occurs because illumination differences between the reference block and the current block are reduced or eliminated when LIC is used with the reference block. Therefore, using SAD as a difference calculation that is sensitive to illumination differences may lead to inaccurate results.

[0112] Various embodiments herein describe approaches for improving the accuracy of template matching in IBC-LIC coded bitstreams. To minimize problems associated with illumination differences and the inaccurate results they cause, an indicator may be used to indicate whether LIC is used. The indicator may be a flag. The indicator may be included in the bitstream. By determining whether LIC is used, prediction of the current block may take into account the effects of LIC. LIC may improve the accuracy of current block prediction by compensating for contrast and luminance differences between the current and reference blocks. More accurate prediction may result in improved compression efficiency, for example, if the number of bins encoding the residual signal is reduced, and higher reconstructed picture quality may be achieved using the same number of coded bins. This disclosure regarding improved prediction based on LIC may be further extended to templates of reference blocks determined for inter prediction. These and other features are further described below.

[0113] Both HEVC and VVC include a prediction technique that exploits the correlation between blocks of samples within the same picture. This technique is called intra-block copying (IBC). IBC, IBC with local illumination compensation (IBC-LIC), and intra-template matching prediction (intra-TMP) are included in the Enhanced Compression Model (ECM) software algorithm, which is currently being coordinated by the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC MPEG Joint Video Exploration Team (JVET) as potential enhancement video coding techniques beyond the capabilities of VVC.

[0114] Figure 17 shows an example of the IBC disclosed herein. An encoder may determine a block vector (BV) 1710 that may indicate the displacement from a current block 1702 to a reference block 1704 (e.g., intra-block compensated prediction) in the same picture. For example, when performing IBC, the encoder may determine a block vector (BV) 1710 that may indicate the displacement from a current block 1702 to a reference block 1704 (e.g., intra-block compensated prediction) in the same picture. The encoder may use a search process to determine the reference block 1704 from among one or more reference blocks.

[0115] An IBC mode encoder can also be configured to set LIC on or off. IBC-LIC is an intra-prediction technique that models local illumination variations between a current block and its predicted block as a function of the difference between a current block template and a reference block template. LIC provides an encoder to reduce the amount of data that can be transmitted to a decoder. For example, LIC provides an encoder to reduce the amount of data that can be transmitted to a decoder by compensating for local illumination differences between a reference block and a current block. Compensating for local illumination differences between a reference block and a current block can be performed, for example, before the residual is encoded.

[0116] Intra-TMP is another technique that can reduce the amount of data sent (e.g., transmitted) from the encoder to the decoder. The encoder may search a predefined region (e.g., a "search region") for a block template that may be a best match to the template of the current block, for example, if IBC-LIC is on for the current block being coded. Figure 18 shows a current block 1802, a current block template 1804, a reference block 1806, and a reference block template 1808 in a search region 1810 consisting of areas R1-R4 within a current frame 1812 or picture. Block templates that may be tested with the current block template may first be updated according to the local illumination function described herein. The block templates may be updated according to the local illumination function described herein, for example, if IBC-LIC is on. More specifically, each template that may be tested may first be used with the current block template to determine parameters of a local illumination compensation function. The local illumination compensation function may be used with the template that may be tested. The local illumination compensation function may be used with the template that may be tested, for example, based on the determined parameters. The best match may be the block template that has the smallest difference from the template of the current block among all block templates tested within the search area. Figure 19 shows an exemplary current block and current block template (e.g., current block 1902 and current block template 1904) with a sample (e.g., sample 1906). An exemplary reference block 1908 and its neighboring sample template 1910 are also shown in Figure 19.

[0117] The difference may be calculated using sum of absolute differences (SAD). The difference may be calculated using sum of absolute differences (SAD), e.g., when IBC-LIC is not on, the difference may be calculated using mean-removed SAD (MR-SAD). The difference may be calculated using mean-removed SAD (MR-SAD), e.g., when IBC-LIC is on. The encoder may determine the difference between the template samples of the reference block and the template samples of the current block (e.g., current block 1702). The encoder may determine the difference between the template samples of the reference block and the template samples of the current block, for example, for each of one or more block templates tested during the search. The encoder may determine the reference block 1704 from among one or more reference blocks. The encoder may determine the reference block 1704 from among one or more reference blocks, for example, based on the template of the reference block 1704 having the smallest difference from the current block 1702 among the one or more reference blocks. The encoder may determine the reference block 1704 from among one or more reference blocks based on, for example, some other criteria. The reference block 1704 and one or more other reference blocks corresponding to templates tested during the search process may include decoded or reconstructed samples. The decoded (or reconstructed) samples may not have been processed by an in-loop filtering operation (e.g., deblocking or SAO filtering).

[0118] FIG. 20 shows a current block, a reference block, and a corresponding BV. The encoder may use the reference block 1704 to predict the current block 1702, for example, if the reference block 1704 is determined for the current block 1702 to be encoded. The encoder may determine and / or use a difference (e.g., a difference per corresponding sample) between the reference block 1704 and the current block 1702. The difference may be referred to as a prediction error and / or a residual. The samples of the reference block may be updated according to the LIC linear function described herein. The samples of the reference block may be updated according to the LIC linear function described herein, for example, before determining the residual. The samples of the reference block may be updated according to the LIC linear function described herein, for example, when IBC-LIC is on, before determining the residual. The update may be designed so that the residual between the updated reference block and the current block is less than the residual between the reference blocks. The update may be designed so that the residual between the updated reference block and the current block is less than the residual between the reference block and the current block, for example, before the update. The encoder may signal the residual (prediction error) and associated prediction information in the bitstream. The prediction information may include BV 1710. The prediction information may include an indication of BV 1710. A decoder, such as the decoder 300 shown in FIG. 3, may receive the bitstream and decode the current block 1702. The decoder may receive the bitstream and decode the current block 1702, for example, by determining a reference block 1704 that forms a prediction of the current block 1702 and using the prediction information to combine the prediction with the prediction error.

[0119] An encoder may not include BV 1710, a BV indication, BVP 1706, or BVD 1708 in the bitstream sent (e.g., transmitted) to a decoder. An encoder may not include BV 1710, a BV indication, BVP 1706, or BVD 1708 in the bitstream sent (e.g., transmitted) to a decoder, for example, if the encoder determined a reference block by searching for a best matching template in a search space as described herein. An indication in the bitstream that intra-TMP is used for the current block may be used by the decoder to perform an intra-TMP process on the current block to determine a reference block in the same manner as was done in the encoder.

[0120] The BV 1710 may be predictively coded. The BV 1710 may be predictively coded, for example, before being signaled in the bitstream. The BV 1710 may be predictively coded based on the BVs of neighboring blocks of the current block 1702 or the BVs of other blocks. The encoder may predictively code the BV 1710 using merge mode or AMVP described herein. The encoder may encode the BV 1710 as the difference between the BV 1710 and a BV predictor (BVP) 1706, with respect to AMVP described herein in FIG. 17. The encoder may select the BVP 1706 from a list of candidate BVPs. The candidate BVPs may be obtained from previously decoded BVs of neighboring blocks of the current block 1702 or from other sources. Both the encoder and the decoder may generate or determine the list of candidate BVPs.

[0121] The encoder may signal an indication of the BVP 1706 and the BV difference (BVD) 1708 in the bitstream. The encoder may signal an indication of the BVP 1706 and the BV difference (BVD) 1708 in the bitstream, for example, if the encoder selects the BVP 1706 from a list of candidate BVPs. The encoder may indicate the BVP 1706 in the bitstream by an index that points into the list of candidate BVPs or by one or more flags. The BVD 1708 may be calculated based on the difference between the BV 1710 and the BVP 1706. The BVD 1708 may be calculated based on the horizontal component (BVD), which may be determined according to equations (17) and (18), respectively, described herein. x ) and vertical component (BVD y ) may contain two components: BVD x and BVD y Each of the components may contain a magnitude and a sign. The encoder uses the two components, BVD x and BVD y The BVD 1708 in the bitstream can be pointed to via

[0122] The decoder may decode BV 1710. The decoder may decode BV 1710, for example, by adding BVD 1708 to BVP 1706. The decoder may decode current block 1702. The decoder may decode current block 1702, for example, by determining reference block 1704 using BV 1710 and combining the prediction with the prediction error to form a prediction of current block 1702. The decoder may determine reference block 1704, for example, by adding BV 1710 to the location of current block 1702, which may give the location of reference block 1704.

[0123] FIG. 21 shows an encoder and decoder configured for intra-block copy local illumination compensation (IBC-LIC) and intra-template match prediction (intra-TMP). FIG. 21 shows exemplary encoder 2102 and decoder 2106, each configured for IBC-LIC and intra-TMP (e.g., modules 2106 and 2108). The encoder 2102 may be similar to or identical to the encoders 114 and 200 described herein. The encoder 2102 may be configured to perform encoder operations as described herein. The decoder 2106 may be similar to or identical to the decoders 120 and 300 described herein. The decoder 2106 may be configured to perform decoder operations as described herein. The encoder 2102 may encode an input video stream 2110 and send (e.g., transmit) the encoded data in a bitstream 2112 to the decoder 2106. The decoder 2106 may generate a reconstructed video stream 2114 from the bitstream 2112. The operations performed by the encoder 2102 and decoder 2104 may be further extended to adjust templates of candidate reference blocks determined for inter prediction.

[0124] FIG. 22 shows an example method implemented by a decoder. FIG. 22 shows a flowchart 2200. One or more steps of the example flowchart 2200 may be implemented by a decoder, such as the decoder 2104 shown in FIG. 21. An example corresponding encoder process is described herein with respect to FIG. 24. The method may begin, for example, when a decoder (e.g., decoder) 2104 receives a bitstream of encoded video information transmitted (e.g., transmitted) by an encoder (e.g., encoder 2102). The encoder may be configured to use IBC and intra-TMP and to use LIC with IBC to encode at least some intra-predicted blocks. In step 2202, the decoder may receive a flag (e.g., a "LIC flag") in the bitstream. The flag may indicate whether to use a LIC process in connection with the current block. A value of "1" (e.g., to indicate on) for the LIC flag may indicate that LIC is used. A value of "0" (e.g., to indicate off) may indicate that LIC is not used. The bitstream may further include one or more other flags that indicate that the current block was predicted using IBC and / or intra-TMP. The bitstream may further include one or more other flags that indicate that the current block was predicted using inter-prediction and with TMP. The configuration and / or structure of any flags is not limited to any particular value.

[0125] In step 2204, the decoder may determine a difference between the template of the current block and a respective template of each candidate reference block among the plurality of candidate reference blocks. The decoder may determine a difference between the template of the current block and a respective template of each candidate reference block among the plurality of candidate reference blocks, for example, based on the value of the LIC flag. The decoder may use template matching to determine a candidate reference block for the current block within the same picture, for example, if IBC and intra-TMP are signaled as being used for the current block. As described herein, in IBC, the current block may be predicted using reference blocks of the same picture. Also, as described herein, the reference block may be determined using template matching, and the bitstream may not need to include a BV or BV indication when intra-TMP is used with IBC. The determination of the difference between templates may be implemented differently. The determination of the difference between templates may be implemented differently, for example, based on whether the LIC flag is on or off. More specifically, the difference may be calculated using a difference calculation technique, such as SAD, that is sensitive to lighting differences, for example, if the LIC flag is off, indicating that LIC will not be used for the current block. The difference calculation may be performed in a manner that can account for LIC, if the LIC flag is on, indicating that LIC will be used for the current block. The current block may be predicted in an inter-prediction mode, which may be indicated by one or more flags received in the bitstream. Template matching may be performed on a reference template of a candidate reference block from a different picture than that of the current block. Figures 23A-23C describe further details that may be used in different embodiments to calculate the difference between the template of each candidate reference template and the template of the current block.

[0126] In step 2206, the decoder may predict the current block. The decoder may predict the current block based on, for example, a reference block selected from multiple candidate reference blocks. The decoder may predict the current block based on a reference block selected from multiple candidate reference blocks, for example, based on the difference calculated in step 2204. The reference block corresponds to a template having the smallest calculated difference among the templates of all candidate reference blocks. An indicator other than the smallest difference may be used to select the reference block. The prediction may include using a local illumination compensation process for the reference block according to an instruction on whether to use an LIC process. The prediction may include, for example, updating sample values ​​of the reference block according to a linear LIC function if the instruction indicates to use an LIC process.

[0127] The reconstruction of the candidate block may be performed by copying the reference block and adding a residual, which may be obtained from the bitstream, for example, based on determining the reference block. The sample values ​​of the reference block may be updated according to a linear LIC function described herein. For example, if LIC is used, the sample values ​​of the reference block may be updated according to a linear LIC function described herein before the residual is added to determine the sample values ​​of the reconstructed current block. As described herein, the bitstream may include an indication of whether LIC is used (e.g., an LIC flag). Obtaining the indication from the bitstream is not required; the indication may be obtained from elsewhere. In at least some cases, the indication may be obtained from the candidate reference block, for example, in IBC merge mode.

[0128] 23A, 23B, and 23C show example methods for determining differences between templates of each candidate reference block and a current block based on the LIC flag. FIG. 23A shows an example method for determining differences between templates of each candidate reference block and a current block in a manner based on the LIC flag (e.g., as described herein in step 2204 of FIG. 22). FIG. 23A shows a flowchart 2300A. One or more steps of the example flowchart 2300 may be performed by a decoder. In step 2304, the decoder may identify a template for a block being considered as a possible candidate reference block. A candidate reference block may be identified using template matching, for example, if intra-TMP is signaled by the encoder. One or more candidate reference blocks may be identified based on a BV, a BVP, or an indication thereof.

[0129] In step 2306, the decoder may determine whether LIC can be used in connection with the current block. In step 2308, a difference between the template of the identified block and the current block may be determined, for example, by calculating the SAD of the two templates. The difference between the templates of the identified block and the current block may be determined, for example, if the LIC flag is set to "0" (e.g., set to off) or some other indicator indicates that LIC is not used. Both the encoder and the decoder may perform this step to maintain consistency of the predicted block, for example, to ensure that the prediction result is the same at the encoder and decoder sides.

[0130] The template differential may be calculated in a manner that takes into account the use of LIC, for example, if the LIC flag is set to "1" (e.g., set on). In step 2312, the differential may be calculated using MR-SAD, for example, if the LIC flag is on and the decoder is configured to use a second technique for calculating the differential (e.g., in step 2310).

[0131] In step 2314, parameters (e.g., alpha and beta) of a linear LIC function described herein may be calculated. The parameters (e.g., alpha and beta) of the linear LIC function may be calculated, for example, if the LIC flag is set to 1 and the decoder is configured to calculate differences using a first difference calculation technique (e.g., in step 2310). In step 2316, the LIC function may be used with the candidate template using the derived parameters. In step 2318, differences may be calculated using SAD. The differences may be calculated using SAD, for example, based on using LIC on the template. As described herein, SAD may be more sensitive to illumination differences than MR-SAD. The difference calculation is not limited to SAD and / or MR-SAD. The difference calculation in steps 2308 and 2312 may be performed by another difference calculation technique that may, for example, be more sensitive to illumination variations than the difference calculation technique used in step 2318.

[0132]

[00103] Figure 23B shows an example method for determining differences between each candidate reference block and the current block. Figure 23B shows a flowchart 2300B. The example method shown in Figure 23B operates the same as that described herein with respect to Figure 23A, except that a "yes" determination of IBC-LIC in step 2306 (e.g., LIC may be used for the current block) may cause the method to proceed to step 2314 to calculate LIC parameters without a determination in step 2310.

[0133]

[00103] Figure 23C shows an example method for determining differences between templates of each candidate reference block and a current block. Figure 23 shows a flowchart 2300C. The example method shown in Figure 23C operates the same as that described herein with respect to Figure 23A, except that a "yes" determination of IBC-LIC in step 2306 (e.g., LIC may be used for the current block) may cause the method to proceed to step 2312 to calculate differences between templates without a determination in step 2310.

[0134] FIG. 24 illustrates an exemplary method implemented by an encoder. FIG. 24 illustrates a flowchart 2400. One or more steps of the exemplary flowchart 2400 may be implemented by an encoder, such as, for example, the encoder 2102 described herein in FIG. 21. An exemplary corresponding decoding process 2200 was described herein with respect to FIG. 22. The method may begin, for example, when encoding a block in a current picture in a video sequence. The encoder may determine, based on a configuration, whether IBC and / or intra-TMP are applied to the current block being encoded. The configuration may be a parameter indicated in a header of the bitstream. The indication of the configuration may be implemented, for example, in a sequence parameter set (SPS), a picture parameter set (PPS), and / or a picture header or a slice header. A flag may indicate, for example, whether IBC and / or intra-TMP are applied when performing prediction (e.g., for a picture or slice of a video sequence being processed).

[0135] In step 2402, an encoder (e.g., encoder 2102) may determine a difference between a template of the current block and a respective template of each candidate reference block from a plurality of candidate reference blocks. The encoder (e.g., encoder 2102) may determine a difference between a template of the current block and a respective template of each candidate reference block from a plurality of candidate reference blocks based, for example, on whether LIC is used for the current block. The determination of whether LIC can be used for the current block may be based on a preconfigured illumination threshold of the current block and / or the candidate reference block. The determination of whether LIC can be used with the current block may be based on a preconfigured threshold of an illumination difference between the current block and the candidate reference block. The current block and the plurality of candidate reference blocks may be located within a current picture. The current block and the plurality of candidate reference blocks may be located within a current picture if, for example, IBC and intra-TMP are used for the current block.

[0136] Intra-TMP may be used to determine multiple candidate reference blocks. Also, as described herein with respect to Figures 22 and 23, calculation of template differences between each candidate reference block and the current block may be performed differently based on whether LIC is used for the current block or whether LIC is not used with the current block. The differences between templates may be calculated using SAD, for example, if LIC is not used. The differences between templates may be calculated using either MR-SAD or a combination of LIC adjustment and SAD, for example, if LIC is used.

[0137] In step 2404, the encoder may predict the current block. The encoder may predict the current block, for example, after a difference is calculated relative to a template of a candidate reference block. The encoder may predict the current block based on a reference block, which may be based on a calculated difference between a template of the current block and a template of the candidate reference block. From multiple candidate reference blocks, the block selected as the reference block may be the block corresponding to a template that may be determined to have the smallest calculated difference. An indicator other than the smallest calculated difference may be used to select the reference block. Sample values ​​of the reference block may be adjusted according to a linear LIC function described herein. The sample values ​​of the reference block may be adjusted according to a linear LIC function described herein, for example, before a prediction error (e.g., a residual) may be calculated for the current block. The sample values ​​of the reference block may be adjusted according to a linear LIC function described herein, for example, if LIC is to be used for the current block, before a prediction error (e.g., a residual) may be calculated for the current block. The parameters of the LIC function (e.g., the alpha and beta parameters described herein) may be calculated using templates of the reference block and the current block, for example, before the LIC function is used with the reference block. The prediction error may be calculated without adjusting the reference block of the LIC. The prediction error may be calculated without adjusting the reference block of the LIC, for example, if LIC is not used.

[0138] In step 2406, the encoder may send (e.g., transmit) the prediction error and a LIC flag indicating whether LIC is used for the current block to the decoder in the bitstream. The encoder may send (e.g., transmit) the prediction error and the LIC flag, for example, after the prediction error is calculated for the current block. The transmission of the bitstream is as described herein.

[0139] Both the encoder and decoder may use template matching to identify the reference block; for example, if IBC-LIC is used with intra-TMP for the current block, no BV or indication of the BV may be sent (e.g., transmitted) in the bitstream. The encoder may predictively encode the BV using IBC-merge mode or IBC-AMVP, as described herein, for example, if intra-TMP is not used for the current block. The encoder may use template matching to determine the reference block according to the IBC-LIC context, as described herein with respect to Figure 25 and other figures, for example, if intra-TMP is not used for the current block.

[0140] The encoder and decoder in IBC-AMVP and IBC merge modes may generate candidate lists of neighboring blocks for determining a BVP. The encoder and decoder in IBC-AMVP and IBC merge modes may generate candidate lists of neighboring blocks for determining a BVP as described herein for AMVP mode and merge mode in inter prediction. Template matching based on the LIC flag may be used to change the ordering sequence of the candidate BVPs. The candidate BVPs may be arranged in sequence based on template matching. The candidate BVPs may be arranged in sequence based on template matching so that a candidate BVP whose corresponding template has a smaller difference (e.g., greater similarity) with the template of the current block may have a higher probability of being selected.

[0141] The exemplary approaches discussed herein with respect to Figures 22-24 may be further used in inter prediction (e.g., affine model / mode, translational model / mode, or multi-hypothesis prediction mode) in addition to or instead of IBC mode. With respect to inter prediction, the terms BV, BVP, BVD, and BVD candidate may be replaced with the terms MV, MVP, MVD, and MVD candidate, respectively. The exemplary approaches discussed herein may also be used with IBC and inter prediction. The exemplary approaches discussed herein may also be used for IBC and inter prediction, for example, based on a translational motion model of the predictive block. In IBC, the candidate reference blocks associated with the candidate reference block template may be in the same picture as the current block's picture associated with the current block. In inter prediction, each of the candidate reference blocks may be determined from (and may belong to) a picture different from the current picture of the current block. The candidate reference blocks may be determined from one or more reference pictures, as described herein with respect to Figure 14. Additionally, for inter-prediction, as shown in Figures 23A-23C, instead of the decoder determining whether IBC-LIC is enabled as described herein with respect to step 2306, the decoder may determine whether LIC is enabled for inter-prediction.

[0142] FIG. 25 illustrates an exemplary determination of the magnitude and sign of a BVD from multiple candidate BVDs. A BVD 2516 for a current block 2502 and a candidate BVD (e.g., BVD 2518) may be determined based on a list of candidate reference blocks. The candidate reference blocks correspond, for example, to templates 2506, 2508, 2510, and 2512. The difference between each of templates 2506-2512 and template 2504 of current block 2502 may be determined using a technique that takes LIC into account, for example, if the LIC flag is set to "1" (e.g., set on, used, etc.). The difference may be calculated using either the first difference calculation technique described herein in connection with step 2312 if the LIC flag is on, or the second difference calculation technique described herein in connection with steps 2314-2318. The difference calculation may use the first difference calculation technique, for example, if the LIC flag is off or not present. The template that may have the smallest difference from the current block template 2504 may be selected as the reference block. The reference block may be the block that corresponds to the template 2508, as shown in FIG.

[0143] As shown in Figure 25, the four candidate templates correspond to different x and y component sign combinations of the BVD relative to the BVP. A decision based on the smallest difference may determine that the bottom right (corresponding to +, - x, y displacements) template is selected, which may enable the determination of the BVD 2518. The BV 2514 may be calculated, for example, as BV = BVP + BVD based on the determined BVP 2516 and BVD 2518. Both the encoder and / or decoder may perform these steps. Figures 26-29 further illustrate the determination of the sign and magnitude of the BVD.

[0144] Figure 26 shows an example of a context-based adaptive binary arithmetic coding (CABAC) encoder. A CABAC encoder (e.g., CABAC encoder 2600) may be implemented in a video encoder for entropy encoding syntax elements of a video sequence, such as the video encoder 200 described herein with reference to Figure 2 or the encoder 2102 described herein with reference to Figure 21. As shown in Figure 26, the CABAC encoder (e.g., CABAC encoder 2600) may comprise a binarizer 2602, an arithmetic encoder 2604, and / or a context modeler 2606.

[0145] A CABAC encoder (e.g., CABAC encoder 2600) may receive syntax elements 2608 for arithmetic encoding. The syntax elements (e.g., syntax elements 2608) may be generated by a video encoder and / or may describe how a video signal may be reconstructed by a video decoder. The syntax elements may include an intra-prediction mode based on which a coding unit (CU) is intra-predicted. The syntax elements may include motion data (e.g., MVD and MVP-related data) based on which a CU is inter-predicted. The syntax elements may include displacement data (e.g., BVD and BVP-related data) based on which a CU is predicted using IBC.

[0146] A binarizer (e.g., binarizer 2602) may map the values ​​of a syntax element (e.g., syntax element 2608) to a sequence of binary symbols (e.g., bins). Binarizer 2602 may define a unique mapping of the values ​​of syntax element 2608 to a sequence of binary symbols. Binarization of syntax elements may be useful for improving the implementation of probability modeling and arithmetic encoding. Binarizer 2602 may implement one or more binarization processes (e.g., unary, truncated unary, k-th truncated Rice, k-th Exponential-Golomb (EGk), fixed length, or some combination of two or more of these binarization processes). Binarizer 2602 may select a binarization process. The binarizer 2602 may select a binarization process based on the type of the syntax element 2608 and / or one or more syntax elements processed by the CABAC encoder 2600 before the syntax element 2608. The binarizer 2602 may not process the syntax element 2608. The binarizer 2602 may not process the syntax element 2608, for example, based on the syntax element 2608 already being represented by a sequence of one or more binary symbols. The binarizer 2602 may not be used, and the syntax element 2608 may be encoded directly by the CABAC encoder 2600, for example, if it is represented by a sequence of one or more non-binary symbols.

[0147] One or more of the binary symbols may be processed by an arithmetic encoder (e.g., arithmetic encoder 2604). One or more of the binary symbols may be processed by arithmetic encoder 2604, for example, based on binarizer 2602 optionally mapping values ​​of syntax elements 2608 to a sequence of binary symbols. Arithmetic encoder 2604 may process each of the one or more binary symbols in one of at least two modes: a normal arithmetic encoding mode or a bypass arithmetic encoding mode.

[0148] The arithmetic encoder (arithmetic encoder 2604) may process binary symbols that do not have a uniform or near-uniform probability distribution (e.g., binary symbols that do not have a probability distribution of 0.5 for each of their two possible values) in normal arithmetic encoding mode. In normal arithmetic encoding mode, the arithmetic encoder 2604 may perform arithmetic encoding as described herein. The arithmetic encoder 2604 may, for example, subdivide the current coding interval into m disjoint subintervals. Each of the m disjoint subintervals may have a width proportional to the probability of the binary symbol having a different one of the values ​​of the m-ary source alphabet. For example, for a binary symbol, the value of m may be equal to 2, and the current coding interval may be subdivided into two disjoint intervals, each having a width proportional to the probability of a different one of the two possible values ​​{0, 1} for the binary symbol being encoded. The probabilities of the two possible values ​​for the binary symbol may be dictated by a probability model for the binary symbol (e.g., probability model 2610). The arithmetic encoder 2604 may encode a binary symbol by selecting a subinterval that corresponds to the actual value of the binary symbol as the new coding interval for the next binary symbol to be encoded.

[0149] An arithmetic encoder (e.g., arithmetic encoder 2604) may receive a probability model 2610 from a context modeler (e.g., context modeler 2606). The context modeler 2606 may determine the probability model 2610 for the binary symbol by a fixed selection. The context modeler 2606 may determine the probability model 2610 for the binary symbol by a fixed selection, for example, based on the position of the binary symbol in a sequence of binary symbols representing the syntax element 2608. The context modeler 2606 may determine the probability model 2610 for the binary symbol by an adaptive selection from among two or more probability models. The context modeler 2606 may determine the probability model 2610 for the binary symbol by an adaptive selection from among two or more probability models, for example, based on information related to the binary symbol. As shown in FIG. 26 , the probability model (e.g., probability model 2610) may determine the probability P of the least likely symbol (LPS). LPS and the most likely symbol (MPS) value V MPS The probability model 2610 may include two parameters: LPS In addition to or instead of, the probability of MPS P MPS Similarly, a probabilistic model (e.g., probabilistic model 2610) may include the value V of the MPS. MPS In addition to or instead of, the value of LPS, V LPSThe arithmetic encoder (e.g., arithmetic encoder 2604) may provide one or more probability model update parameters (e.g., probability model update parameters 2612) to the context modeler (e.g., context modeler 2606). The arithmetic encoder 2604 may provide the one or more probability model update parameters 2612 to the context modeler 2606, for example, after the arithmetic encoder 2604 encodes the binary symbol. The context modeler 2606 may adapt the probability model 2610. The context modeler 2606 may adapt the probability model 2610 based on the one or more probability model update parameters 2612, for example. The one or more probability model update parameters 2612 may include an actual coded value of the binary symbol ... MPS If not equal to P LPS otherwise, by increasing P LPS The probabilistic model 2610 may be updated by decreasing

[0150] The arithmetic encoder (e.g., the arithmetic encoder 2604) may process binary symbols that may have a uniform or near-uniform probability distribution in the bypass arithmetic encoding mode. The arithmetic encoder 2604 may bypass the determination and adaptation of the probability model performed in the normal arithmetic encoding mode when encoding binary symbols to speed up the encoding process, for example, because the binary symbols processed by the arithmetic encoder 2604 in the bypass arithmetic encoding mode may have a uniform or near-uniform probability distribution. In addition, the subdivision of the current coding interval may be simplified due to the uniform or near-uniform probability distribution. The current coding interval may be partitioned into two disjoint subintervals of equal width, which may be realized using a simple implementation that may further speed up the encoding process. The arithmetic encoder (e.g., the arithmetic encoder 2604) may encode a binary symbol by selecting a subinterval corresponding to the value of the binary symbol as the new coding interval for the next binary symbol to be encoded. Because CABAC encoding may have throughput limitations, the resulting increase in encoding speed for binary symbols encoded by the arithmetic encoder 2604 in the bypass arithmetic encoding mode may be significant.

[0151] An arithmetic encoder (e.g., arithmetic encoder 2604) may determine a value of the range of the final coding interval as an arithmetic codeword (e.g., arithmetic codeword 2614) for a binary symbol. The arithmetic encoder 2604 may determine a value of the range of the final coding interval as an arithmetic codeword 2614 for a binary symbol, for example, based on processing several binary symbols (e.g., corresponding to one or more syntax elements). The arithmetic encoder 2604 may output the arithmetic codeword 2614. The arithmetic encoder 2604 may output the arithmetic codeword 2614 to a bitstream that may be received and processed by, for example, a video decoder.

[0152] Two syntax elements coded in the bypass arithmetic coding mode are the motion vector differential (MVD) magnitude and the block vector differential (BVD) magnitude. These syntax elements may be determined as part of advanced motion vector prediction (AMVP) for inter prediction and AMVP for intra block copy (IBC), respectively, as described herein. Although the bypass arithmetic coding mode may be used to speed up the arithmetic coding process, compression of symbols of these syntax elements coded in the bypass arithmetic coding mode may be limited because their probability distributions are uniformly, or at least nearly uniformly, distributed. From information theory, a symbol cannot be compressed at a rate lower than its entropy without losing information, and a symbol with a uniform probability distribution has the maximum entropy. Therefore, symbols coded using the bypass arithmetic coding mode may generally require more bits to code than symbols coded using the normal arithmetic coding mode.

[0153] Apparatuses and methods for improving the compression efficiency of one or more magnitude symbols of BVD are described herein. Instead of entropy coding the magnitude symbol of BVD, the methods described herein may entropy code an indication of whether the value of the magnitude symbol of BVD matches the value of the same magnitude symbol of a BVD candidate used as a predictor of BVD. The BVD predictor may be selected from among multiple BVD candidates. The BVD predictor may be selected from among multiple BVD candidates, for example, based on the cost of the multiple BVD candidates. The cost of each BVD candidate in the multiple BVD candidates may be calculated based on, for example, a difference between a template of the current block and a template of a candidate reference block. The candidate reference block may be displaced relative to the current block by the sum of the BVD candidate and a block vector predictor (BVP). The indication of whether the value of the magnitude symbol of BVD matches the value of the magnitude symbol of a BVD predictor may have a non-uniform probability distribution and, therefore, may provide improved compression efficiency over coding the magnitude symbol of BVD based on a uniform probability distribution.

[0154] Figure 27A shows an example of IBC. As shown in Figure 27A, BV 2702, current block 2704, reference block 2706, BVP 2708, and BVD 2710 may be considered to correspond to BV 1710, current block 1702, reference block 1704, BVP 1706, and BVD 1708, respectively, described herein with respect to Figure 17. A BV to be predictively coded (e.g., BV 2702) may be determined as described herein with respect to Figure 17.

[0155] The BV (e.g., BV 2702) may be predictively coded. The BV (e.g., BV 2702) may be predictively coded, for example, before being signaled in the bitstream. The BV (e.g., BV 2702) may be predictively coded, for example, based on the BVs of neighboring blocks of the current block 2704 or the BVs of other blocks. The encoder may predictively code the BV 2702, for example, using merge mode or AMVP described herein. The encoder may encode the BV 2702 as the difference between the BV 2702 and a BV predictor (BVP) 2708 with respect to AMVP, as shown in Figure 27A. The encoder may select the BVP 2708 from a list of candidate BVPs. The candidate BVPs may be obtained from previously decoded BVs of neighboring blocks of the current block 2704 or from other sources. Both the encoder and the decoder may generate or determine the list of candidate BVPs.

[0156] The encoder may signal an indication of the BVP (e.g., BVP2708) and the BV difference (e.g., BVD2710) in the bitstream. The encoder may signal an indication of BVP2708 and / or BVD2710 in the bitstream, for example, based on the encoder selecting BVP2708 from a list of candidate BVPs. The encoder may indicate BVP2708 in the bitstream by an index that points into the list of candidate BVPs and / or by one or more flags. BVD2710 may be calculated based on the difference between BV2702 and BVP2708. BVD2710 may be calculated based on the horizontal component (BVD), which may be determined according to equations (17) and (18) above, respectively. x )2712 and vertical component (BVD y ) 2714. Two components of BVD x 2712 and BVD y 2714 may each include a magnitude and a sign. As shown in FIG. x2712 has a magnitude of 10011 in fixed-length binary (or 19 in decimal) and has a negative sign. The positive horizontal direction points to the right in the example shown in FIG. 27A. As further shown in FIG. 27A, BVD y 2714 has a magnitude of 01011 in fixed-length binary (or 11 in decimal) and a positive sign. The positive vertical direction points down in the example shown in Figure 27A. The encoder calculates its two components, BVD x 2712 and BVD y The BVD 2710 may be indicated in the bitstream via 2714.

[0157] The decoder may decode a BV (e.g., BV 2702) by adding BVD 2710 to BVP 2708. The decoder may decode a current block (e.g., current block 2704) by using BV 2702 and determining a reference block (e.g., reference block 2706) that combines the prediction and prediction error to form a prediction of current block 2704. The decoder may determine reference block 2706 by adding BV 2702 to the location of current block 2704, which may give the location of reference block 2706.

[0158] As described herein, the magnitude of a BVD (e.g., BVD2710) may be coded in a bypass arithmetic coding mode. While the bypass arithmetic coding mode may be used to speed up the arithmetic coding process, the compression of BVD2710 magnitude symbols coded in a bypass arithmetic coding mode may be limited because their probability distribution may be uniformly or nearly uniformly distributed. From information theory, a symbol cannot be compressed at a rate lower than its entropy without loss of information, and symbols with uniform probability distributions have the maximum entropy. Therefore, symbols coded using a bypass arithmetic coding mode may generally require more bits to code than symbols coded using a normal arithmetic coding mode.

[0159] As described herein, the compression efficiency of one or more magnitude symbols of BVD (e.g., BVD 2710) may be improved. An encoder may entropy encode an indication of whether the value of the magnitude symbol of BVD 2710 matches the value of the magnitude symbol of a BVD candidate used as a predictor of BVD 2710. The encoder may entropy encode this indication, for example, instead of directly entropy encoding the magnitude symbol of BVD 2710. The indication of whether the value of the magnitude symbol of BVD 2710 matches the value of the magnitude symbol of a BVD predictor may have a non-uniform probability distribution and may therefore provide improved compression efficiency. An encoder may select a BVD predictor from among multiple BVD candidates. The encoder may select a BVD predictor from among multiple BVD candidates based, for example, on the costs of the multiple BVD candidates. The BVD candidates may include a BVD candidate for each possible value of the magnitude symbol of BVD 2710. The magnitude symbol of BVD 2710, which can be represented in binary form, has only two possible values. Thus, the BVD candidates may include two BVD candidates for this representation, one for each possible value of the magnitude symbol in BVD 2710 being encoded: a first BVD candidate equal to BVD 2710 itself, and a second BVD candidate equal to BVD 2710 but with the inverse (or other) value of the magnitude symbol of BVD 2710. The cost of each BVD candidate in the plurality of BVD candidates may be calculated based, for example, on the difference between the template of the current block 2704 and the template of the candidate reference block. The candidate reference block may be displaced relative to the current block by the sum of the BVD candidate and BVP 2708.

[0160] 27A shows an example magnitude symbol 2716 of a BVD 2710 that is entropy coded. The magnitude symbol 2716 of the BVD 2710 is the horizontal component of the BVD 2710, x2712 and may have a binary value of "0." As described herein, an encoder may, for example, instead of directly entropy encoding magnitude symbol 2716 of BVD 2710, entropy encode an indication of whether the value of magnitude symbol 2716 of BVD 2710 matches the value of the same magnitude symbol of a BVD candidate used as a predictor for BVD 2710. The encoder may, for example, select a BVD predictor from among multiple BVD candidates based on the costs of the multiple BVD candidates. The BVD candidates may include a BVD candidate for each of the two possible values ​​{0, 1} of the magnitude symbol 2716 of BVD2710, i.e., a first BVD candidate 2718 equal to BVD2710 itself, and a second BVD candidate 2720 equal to BVD2710 but having the opposite (or other) value of the magnitude symbol 2716 of BVD2710.

[0161] 27B shows exemplary BVD candidates for entropy coding the magnitude symbol 2716 of BVD 2710. Specifically, FIG. 27B shows exemplary BVD candidate 2718, which is equal to BVD 2710 itself, and BVD candidate 2720, which is equal to BVD 2710 but has the opposite (or other) value of the magnitude symbol 2716 of BVD 2710. According to the opposite (or other) value of the magnitude symbol 2716 of BVD candidate 2718, BVD candidate 2720 is a horizontal component BVD with a magnitude of 11011 in fixed-length binary (or decimal 27) and a negative sign. x 2722. The vertical component BVD of the BVD candidate 2720 y 2724 is the same magnitude of 01011 in fixed length binary (or decimal 11) and the vertical component BVD of BVD candidate 2718 (or BVD 2710). y 2714 and has a positive sign.

[0162] The cost of each BVD candidate in the plurality of BVD candidates may be calculated, for example, based on a difference between the template of the current block 2726 and a template of a candidate reference block that is displaced relative to the current block 2704 by the sum of the BVD candidate and BVP 2708. The encoder may determine the cost of the BVD candidate 2718, for example, based on a difference between the template 2726 of the current block 2704 and the template 2728 of the candidate reference block 2730 that is displaced relative to the current block 2704 by the sum of the BVD candidate 2718 and BVP 2708. The encoder may determine a difference between the template 2726 and the template 2728, for example, based on a difference between the samples of the template 2726 and the samples of the template 2728. The difference may include, for example, a sum of squared differences (SSD), a sum of absolute differences (SAD), a sum of absolute transformed differences (SATD), a mean removed SAD, and / or a mean removed SSD. The encoder may determine the cost of BVD candidate 2720 based on, for example, the difference between template 2726 of current block 2704 and template 2732 of candidate reference block 2734 that is displaced relative to current block 2704 by the sum of BVD candidate 2720 and BVP 2708. The encoder may determine the difference between template 2726 and template 2732 based on, for example, the difference (e.g., SSD, SAD, SATD, mean-removed SAD, or mean-removed SSD) between the samples of template 2726 and template 2728. Templates 2726, 2728, and 2732 may include one or more samples to the left and / or above their respective blocks. Templates 2726, 2728, and 2732 may include samples from one or more columns to the left of their respective blocks and / or from one or more rows above their respective blocks. FIG. 27B shows one exemplary position and shape (eg, an L-shape rotated 90 degrees clockwise) of templates 2726, 2728, and 2732.

[0163] The encoder may select one of the multiple BVD candidates as the BVD predictor. The encoder may select one of the multiple BVD candidates as the BVD predictor, for example, based on determining the cost of each of the multiple BVD candidates. The encoder may select the BVD candidate that may have the smallest cost among the multiple BVD candidates as the BVD predictor. Figure 27C shows an example table including components and costs of BVD candidates. BVD candidates 2718 and 2720 may be assumed to be the only BVD candidates. More BVD candidates may be used. The rows of the table may be sorted by the costs of BVD candidates 2718 and 2720, with the BVD candidates with the smallest costs at the top. BVD candidate 2718 may have the smallest cost among BVD candidates 2718 and 2720. The encoder may, for example, select BVD candidate 2718 as the BVD predictor 2736 for BVD 2710 because BVD candidate 2718 has the smallest cost among BVD candidates 2718 and 2720.

[0164] The encoder may entropy encode an indication (e.g., indication 2738) of whether the value of the magnitude symbol 2716 of the BVD 2710 matches the value of the magnitude symbol 2716 of the BVD predictor 2736. The encoder may entropy encode the indication 2738 of whether the value of the magnitude symbol 2716 of the BVD 2710 matches the value of the magnitude symbol 2716 of the BVD predictor 2736, for example, based on selecting the BVD candidate 2718 as the BVD predictor 2736. The magnitude symbol 2716 of the BVD predictor (e.g., BVD predictor 2736) may have a value of “0,” which matches the value of the magnitude symbol 2716 of the BVD 2710. The indication 2738 may indicate that the value of the magnitude symbol 2716 of the BVD 2710 matches the value of the magnitude symbol 2716 of the BVD predictor 2736. The indication (e.g., indication 2738) may be a single bit having a value of “0” if the value of the magnitude symbol 2716 of the BVD 2710 matches the value of the magnitude symbol 2716 of the BVD predictor 2736. The indication (e.g., indication 2738) may be a single bit having a value of “1” if the value of the magnitude symbol 2716 of the BVD 2710 does not match the value of the magnitude symbol 2716 of the BVD predictor 2736. Logic (e.g., logic 2740) may be used to determine the indication 2738. The logic 2740 may implement a logical exclusive or (XOR) function. The indication 2738 may indicate, for example, that if the magnitude symbol 2716 is not binary, a first candidate among multiple candidates (e.g., when sorted based on their respective costs) may have a magnitude symbol 2716 value that may match the magnitude symbol 2716 value of the BVD 2710.

[0165] In Figure 27C, the encoder may entropy code the instructions 2738 using an arithmetic encoder (e.g., arithmetic encoder 2742). The instructions 2738 may have a non-uniform probability distribution, for example, based on a method for determining the instructions (e.g., instructions 2738) as described herein. Accordingly, the arithmetic encoder 2742 may process the instructions 2738 in a normal arithmetic encoding mode, as described herein. The arithmetic encoder 2742 may subdivide the current coding interval into m disjoint subintervals. Each of the m disjoint subintervals may have a width proportional to the probability that the encoded symbol has a different one of the values ​​of the m-ary source alphabet. For instructions 2738 that are binary, the value of m is equal to 2, and the current coding interval may be subdivided into two disjoint intervals, each having a width proportional to the probability of a different one of the two possible values ​​{0, 1} for the encoded instruction 2738. The probabilities of two possible values ​​for the instruction 2738 may be indicated by a probability model 2744 for the instruction 2738. The arithmetic encoder 2742 may encode the instruction 2738 by selecting a subinterval corresponding to the actual value of the instruction 2738 as a new coding interval for the next binary symbol to be encoded.

[0166] An operation encoder (e.g., operation encoder 2742) may receive a probability model (e.g., probability model 2744) from a context modeler (e.g., context modeler 2746). The context modeler 2746 may determine the probability model 2744 for an instruction (e.g., instruction 2738) by fixed or adaptive selection from among two or more probability models. The context modeler 2746 may, for example, select the BVD of BVD 2710. x The position of the magnitude symbol 2716 in 2712 or the BVD in BVD 2710 x Based on the index of the position (e.g., value indicating the position) of the magnitude symbol 2716 in 2712, the probability model 2744 may be determined by a fixed or adaptive selection from among two or more probability models. xThe position (e.g., position index) of the magnitude symbol 2716 in 2712 may provide an indication of the distance 2753 (as described herein with respect to FIG. 27B ) between the two candidate BVDs. The likelihood that the value of the magnitude symbol 2716 of the BVD predictor 2736 matches the value of the magnitude symbol 2716 of the BVD 2710 may be related to the distance 2753. More specifically, the degree of difference between the respective templates of the candidate BVDs may be greater the greater the value of the distance 2753 between the candidate BVDs. The greater the difference between the respective templates of the BVD candidates, the more likely the cost of the BVD candidate accurately reflects the BVD candidate with a magnitude symbol 2716 value that matches the value of the magnitude symbol 2716 of the BVD 2710. x The position (eg, position index) of the magnitude symbol 2716 in 2712 may be useful in selecting a probability model 2744 for the instruction 2738.

[0167] A context modeler (e.g., context modeler 2746) may use the BVD 2710 BVD for adaptive selection among two or more probabilistic models. x The position (e.g., position index) of the magnitude symbol 2716 in 2712 may be compared to one or more thresholds. x The context modeler 2746 may compare the position (e.g., position index) of the magnitude symbol 2716 in 2712 to a first threshold. x The context modeler 2746 may select a first probability model for the instruction 2738 based on the position (e.g., position index) of the magnitude symbol 2716 in 2712 being less than a threshold. x The context modeler 2746 may select a second probability model for the instruction 2738 based on the position (e.g., position index) of the magnitude symbol 2716 in 2712 being above a threshold. xThe position (e.g., position index) of the magnitude symbol 2716 in 2712 is x The context modeler 2746 may compare the magnitude symbol 2716 in 2712 with a second threshold based on the position (e.g., position index) being above the threshold. x The context modeler 2746 may select a second probability model for the indication 2738 based on the position (e.g., position index) of the magnitude symbol 2716 in 2712 being less than a second threshold. x A third probability model for the indication 2738 may be selected based on the position (eg, position index) of the magnitude symbol 2716 in 2712 being above a second threshold.

[0168] A context modeler (e.g., context modeler 2746) may determine a probabilistic model (e.g., probabilistic model 2744) by fixed and / or adaptive selection from among two or more probabilistic models. The context modeler (e.g., context modeler 2746) may, for example, determine the magnitude of BVD 2710 (or the magnitude of BVD 2710) for incremental changes in the value of magnitude symbol 2716 of BVD 2710. x A probability model (e.g., probability model 2744) may be determined by fixed and / or adaptive selection from among two or more probability models based on a change in the value of the magnitude symbol 2716 of the BVD 2710. x The change in the value of 2712) is 2 (n-1) where n is the BVD of BVD2710 x 27 is the bit position of the magnitude symbol 2716 in BVD 2712. In the example shown in FIG. 27, n=4, so the magnitude symbol 2716 in BVD 2710 (or the BVD of BVD 2710) is x The change in the value of 2712) is 2 (4-1)or 8. The magnitude of BVD 2710 (or the magnitude of BVD 2710) for an incremental change in the value of the magnitude symbol 2716 of BVD 2710 may be determined as x 2712) may provide an indication of the distance 2753 (shown in FIG. 27B) between the two candidate BVDs. As described herein, the likelihood that the value of the magnitude symbol 2716 of the BVD predictor 2736 matches the value of the magnitude symbol 2716 of the BVD 2710 may be related to the distance 2753. The degree of difference between the respective templates of the candidate BVDs may be greater the greater the value of the distance 2753 between the candidate BVDs. The greater the difference between the respective templates of the BVD candidates, the more likely the cost of the BVD candidate may accurately reflect a BVD candidate with a magnitude symbol 2716 value that matches the value of the magnitude symbol 2716 of the BVD 2710. Thus, the likelihood of a change in the value of the magnitude symbol 2716 of the BVD 2710 (or the BVD of the BVD 2710) for an incremental change in the value of the magnitude symbol 2716 of the BVD 2710 may be greater. x Changes in the value of 2712) can be useful in selecting a probability model 2744 for an instruction 2738.

[0169] A context modeler (e.g., context modeler 2746) may calculate the magnitude of BVD 2710 (or the magnitude of BVD 2710) for incremental changes in the value of magnitude symbol 2716 of BVD 2710. x A context modeler (e.g., context modeler 2746) may compare the value of BVD 2710 (or the BVD of BVD 2710) to one or more thresholds, e.g., for adaptive selection among two or more probability models, for incremental changes in the value of magnitude symbol 2716 of BVD 2710. x The context modeler 2746 may compare the value of the BVD 2710 (or the BVD 2712) to one or more thresholds, for example, for incremental changes in the value of the magnitude symbol 2716 of the BVD 2710. x The context modeler 2746 may compare the value of the BVD 2710 (or the BVD 2712) to a first threshold, for example, for an incremental change in the value of the magnitude symbol 2716 of the BVD 2710. xThe context modeler 2746 may select the first probabilistic model for the instruction 2738 based on the value of the magnitude symbol 2716 of the BVD 2710 being less than a threshold. x The context modeler 2746 may select a second probabilistic model for the indication 2738 based on the value of the magnitude symbol 2716 of the BVD 2710 being above a threshold. x 2712) is greater than a threshold, the BVD 2710 (or the BVD of BVD 2710) for an incremental change in the value of the magnitude symbol 2716 of BVD 2710 x The context modeler 2746 may compare the value of the BVD 2710 (or the BVD 2712) to a second threshold, for example, for an incremental change in the value of the magnitude symbol 2716 of the BVD 2710. x The context modeler 2746 may select a second probability model for the indication 2738 based on the value of the magnitude symbol 2716 of the BVD 2710 being less than a second threshold. x A third probability model for instruction 2738 may be selected based on the value of 2712) being above a second threshold.

[0170] As shown in FIG. 27C, a probability model (e.g., probability model 2744) may be used to determine the probability P of the least likely symbol (LPS) for the instruction 2738. LPS , and the most likely symbol (MPS) value V for instruction 2738 MPS The probability model 2744 may include, for example, the probability P of an LPS for the instruction 2738. LPS In addition to or instead of, the probability P of MPS for instruction 2738 MPS Similarly, a probabilistic model (e.g., probabilistic model 2744) may include a value V of the MPS for instruction 2738. MPS In addition to or instead of, the value V of the LPS for instruction 2738LPS The computational encoder 2742 may provide one or more probability model update parameters 2750 to the context modeler 2746. The computational encoder 2742 may provide the one or more probability model update parameters 2750 to the context modeler 2746, for example, based on the computational encoder 2742 encoding the instruction 2738. The context modeler 2746 may adapt the probabilistic model 2744 based on the one or more probability model update parameters 2750. The one or more probability model update parameters 2750 may include an actual coded value of ... MPS If not equal to P for instruction 2738 LPS Otherwise, by increasing P LPS A probabilistic model (e.g., probabilistic model 2744) may be updated by decreasing

[0171] An arithmetic encoder (e.g., arithmetic encoder 2742) may determine the range values ​​of the final coding interval as an arithmetic codeword (e.g., arithmetic codeword 2752) for the binary symbols. The arithmetic encoder 2742 may determine the range values ​​of the final coding interval as an arithmetic codeword 2752 for the binary symbols, for example, based on processing several binary symbols (e.g., corresponding to one or more syntax elements). The arithmetic encoder 2742 may output the arithmetic codeword 2752. The arithmetic encoder 2742 may output the arithmetic codeword 2752 to a bitstream that may be received and processed by, for example, a video decoder.

[0172] 27D shows an example of a decoder (e.g., decoder 300 described with respect to FIG. 3) that may receive an opcode word (e.g., opcode word 2752), an opcode instruction (e.g., opcode instruction 2738) from opcode word 2752, and a use instruction 2738 to determine a magnitude symbol (e.g., magnitude symbol 2716) of BVD 2710.

[0173] The decoder may receive an op code word (e.g., op code word 2752) in the bitstream. The decoder may provide the op code word (e.g., op code word 2752) to an op decoder 2754. The instructions 2738 may have a non-uniform probability distribution, e.g., based on a method for determining the instructions 2738 as described herein. Accordingly, the op decoder 2754 may process the instructions 2738 in a normal op decoding mode. For example, the op decoder (e.g., op decoder 2754) may perform a recursive interval subdivision as described herein to decode a symbol encoded by the op code word 2752. The op decoder (e.g., op decoder 2754) may op eratively decode symbols that can take on values ​​from the m-ary source alphabet by dividing the initial coding interval into m disjoint subintervals. Each of the m disjoint subintervals may have a width proportional to the probability that the symbol has a different one of the values ​​of the m-ary source alphabet. The value m is equal to 2, and the initial coding interval may be subdivided into two disjoint intervals, each having a width proportional to the probability of a different one of two possible values ​​{0, 1} for a binary symbol such as designation 2738. The probability of symbols having different values ​​in the m-ary source alphabet may be referred to as a probability model for the symbol, as described herein. A symbol may be operationally decoded from an operation codeword 2752 by determining the symbol value corresponding to the subinterval in which the operation codeword falls. The decoder uses this interval subdivision scheme recursively N times to determine which subinterval the operation codeword 2752 falls within, thereby deriving the sequence s={s1, s2, ..., s N ) each symbol s i can be decoded sequentially.

[0174] A computation decoder (e.g., computation decoder 2754) may receive a probability model (e.g., probability model 2744) for instruction 2738 from context modeler 2746. The computation decoder (e.g., computation decoder 2754) may receive a probability model (e.g., probability model 2744) for instruction 2738 from context modeler 2746, for example, when decoding a symbol corresponding to instruction 2738. The context modeler (e.g., context modeler 2756) may determine probability model 2744 for instruction 2738 by a fixed selection or by adaptive selection from among two or more probability models, as described herein with respect to context modeler 2746, as shown in FIG.

[0175] As shown in FIG. 27D , a computation decoder (e.g., computation decoder 2754) may provide one or more probability model update parameters 2750 to a context modeler (e.g., context modeler 2756). The computation decoder 2754 may provide the one or more probability model update parameters 2750 to the context modeler 2756, for example, based on the computation decoder 2754 decoding the instruction 2738. The context modeler 2756 may adapt the probability model 2744. The context modeler 2756 may adapt the probability model 2744 based on the one or more probability model update parameters 2750. The one or more probability model update parameters 2750 may include an actual coded value of the instruction 2738. The context modeler 2756 may adapt the probability model 2744 based on the one or more probability model update parameters 2750, for example ... MPS If not equal to P for instruction 2738 LPS Otherwise, by increasing P LPS The probabilistic model 2744 may be updated by decreasing

[0176] The decoder may determine the value of the magnitude symbol 2716 of the BVD 2710 based on, for example, the value of the magnitude symbol 2716 of the BVD predictor 2736 and the value of the indication 2738. The decoder may determine the value of the magnitude symbol 2716 of the BVD 2710 based on, for example, the value of the magnitude symbol 2716 of the BVD predictor 2736 and the value of the indication 2738, after entropy decoding the indication 2738. The decoder may determine the value of the magnitude symbol 2716 of the BVD 2710 to be equal to the magnitude symbol of the BVD predictor 2736 based on, for example, the indication 2738 indicating that the value of the magnitude symbol 2716 of the BVD 2710 matches the value of the magnitude symbol 2716 of the BVD predictor 2736. The decoder may determine, for example, based on the indication 2738 indicating that the value of the magnitude symbol 2716 of BVD 2710 does not match the value of the magnitude symbol 2716 of the BVD predictor 2736, that the value of the magnitude symbol 2716 of BVD 2710 is not equal to or equal to the reciprocal of the magnitude symbol 2716 of the BVD predictor 2736. The magnitude symbol 2716 of the BVD predictor 2736 may have a value of “0,” which matches the value of the magnitude symbol 2716 of BVD 2710. The indication 2738 may indicate that the value of the magnitude symbol 2716 of BVD 2710 matches the value of the magnitude symbol 2716 of the BVD predictor 2736. The indication 2738 may be, for example, a single bit that may have a value of “0” if the value of the magnitude symbol 2716 of BVD 2710 matches the value of the magnitude symbol 2716 of the BVD predictor 2736. The indication 2738 may be, for example, a single bit that may have a value of “1” if the value of the magnitude symbol 2716 of the BVD 2710 does not match the value of the magnitude symbol 2716 of the BVD predictor 2736. Logic 2758 may be used to determine the magnitude symbol 2716 of the BVD 2710. The logic 2758 may implement a logical XOR function. The indication 2738 may, for example, indicate the first candidate among multiple candidates (e.g., sorted based on their respective costs) that has a magnitude symbol 2716 value that matches the magnitude symbol 2716 value of the BVD 2710 if the magnitude symbol 2716 is not binary.

[0177] The decoder may determine the value of the magnitude symbol 2716 of the BVD predictor 2736 described herein. More specifically, the decoder may select a BVD predictor (e.g., the BVD predictor 2736) from among multiple BVD candidates based on the costs of the multiple BVD candidates. The BVD candidates may include a BVD candidate for each possible value of the magnitude symbol of BVD 2710. The magnitude symbol of BVD 2710, represented in binary form, may have only two possible values. Thus, the BVD candidates may include at least two BVD candidates for this representation, one for each possible value of the magnitude symbol in BVD 2710 being encoded: a first BVD candidate that may be equal to BVD 2710 itself, and a second BVD candidate that is equal to BVD 2710 but may have the inverse (or other) value of the magnitude symbol of BVD 2710. The cost of each BVD candidate in the plurality of BVD candidates may be calculated as described herein with respect to the encoder. The cost of each BVD candidate in the plurality of BVD candidates may be calculated as described herein with respect to the encoder, for example, based on the difference between the template of the current block 2704 and the template of the candidate reference block. The candidate reference block may be displaced relative to the current block by the sum of the BVD candidate and the BVP 2708. The decoder may select the BVD candidate with the lowest cost as the BVD predictor 2736.

[0178] As discussed herein with respect to Figures 27A-27D, an exemplary approach of entropy coding an indication of whether the value of a BVD magnitude symbol matches the value of a BVD candidate magnitude symbol used as a predictor of BVD may be used with multiple magnitude symbols of BVD. This exemplary approach may be used with BVD candidates other than magnitude symbol 2716. x 27A-27D. x For each additional magnitude symbol of 2716, an additional candidate BVP can be determined. Using this approach, for example, xFor N magnitude symbols of 2716 (where N is an integer value), BVD x 2^N different BVP candidates may be determined, one for each possible combination of values ​​for the N magnitude symbols of 2716. A cost value is further determined for each of the BVP candidates, and the BVD x The 2716 N magnitude symbols may be sorted to determine the BVP predictor for encoding each of the N magnitude symbols.

[0179] As discussed herein with respect to FIGS. 27A through 27D, an exemplary approach for entropy coding an indication of whether the value of the magnitude symbol of a BVD matches the value of the magnitude symbol of a BVD candidate used as a predictor of BVD is x In addition to or in place of one or more of the magnitude symbols of 2716, BVD y It may be used with one or more magnitude symbols of 2714.

[0180] Ingredients of BVD2710BVD y 2714 and BVD x 2716, as well as the components of the BVD candidate, may use fixed-length binary, but the components of BVD 2710 y 2714 and BVD x 2716, and other binarizations of the components of the BVD candidate may be possible. For example, the component BVD of BVD 2710 y 2714 and BVD x 2716 can be represented using unary, truncated unary, k-th order truncated Rice, k-th order Exponential Golomb (Egk), or some combination of two or more of these binarization processes. For Egk, each codeword is of length L N +1 unary prefix and length L N +k suffix, where L N is expressed as follows:

number

[0181] 27A-27D may be used with IBC and inter prediction, for example, based on a translational motion model for the prediction block. The embodiments discussed herein with respect to Figures 27A-27D may be used with IBC and inter prediction, for example, based on an affine motion model for the prediction block.

[0182] 28 illustrates a method for entropy encoding. FIG. 28 shows a flowchart 2800. FIG. 28 illustrates an exemplary method for entropy encoding an indication of whether a value of a magnitude symbol of a BVD matches a value of a magnitude symbol of a BVD candidate used as a predictor of the BVD. One or more steps of the exemplary flowchart 2800 may be performed by an encoder (e.g., the encoder 200 described herein with reference to FIG. 2 or the encoder 2102 described herein with reference to FIG. 21).

[0183] In step 2802, the encoder may determine a BVD. The encoder may determine the BVD, for example, based on a difference between the BV and the BVP. The BV may indicate the displacement of a reference block relative to the current block, and the reference block may be used to predict the current block.

[0184] In step 2804, the encoder calculates a cost for each of the multiple BVD candidates. The multiple BVD candidates may include at least a first BVD candidate and a second BVD candidate, and the value of the magnitude symbol of the first BVD candidate may differ from the value of the magnitude symbol of the second BVD candidate. The encoder may calculate a cost for each BVD candidate in the multiple BVD candidates based on, for example, a difference between a template of the current block and a template of a candidate reference block displaced relative to the current block by the sum of the BVD candidate and the BVP. The BVD may be one of the first or second BVD candidates. The first BVD candidate may differ from the second BVD candidate by the value of the magnitude symbol. The magnitude symbol may be either the horizontal or vertical component of the first BVD candidate. The first and second BVD candidates may be represented in binary format. The first and second BVD candidates may be represented in binary form using unary, truncated unary, kth-order truncated Ricean, kth-order Exponential-Golomb (EGk), fixed length, or some combination of two or more of these binarization processes. The first BVD candidate may be represented in binary form using a Golomb codeword that includes the magnitude symbol of the first BVD candidate in a suffix of the Golomb codeword. The Golomb codeword may be an Exponential-Golomb codeword.

[0185] In step 2806, the encoder may select one of the multiple BVD candidates as the BVD predictor. The encoder may select one of the multiple BVD candidates as the BVD predictor based on, for example, cost. The encoder may select one of the multiple BVD candidates as the BVD predictor based on, for example, one of the multiple BVD candidates having a smallest cost among the costs. The BVD predictor may be the first BVD candidate or the second BVD candidate.

[0186] In step 2808, the encoder may entropy encode an indication of whether the magnitude symbol of the BVD matches the value of the magnitude symbol of the BVD predictor. The encoder may arithmetically encode the indication based on a probability model. The probability model may indicate the probability of the least probable symbol for the indication and the value of the most probable symbol for the indication. The encoder may select a probability model from multiple probability models based on, for example, the position of the magnitude symbol in the BVD. The encoder may select a probability model from multiple probability models based on, for example, a change in the value of the BVD relative to an incremental change in the value of the magnitude symbol of the BVD. The encoder may select a probability model from multiple probability models based on, for example, a comparison of the change in the value of the BVD to one or more thresholds.

[0187]

[0013] Figure 29 illustrates a method for entropy decoding. Figure 29 illustrates a flowchart 2900. Figure 29 illustrates an example method for entropy decoding an indication of whether the value of a magnitude symbol of BVD matches the value of the magnitude symbol of a BVD candidate used as a predictor of BVD, and using the indication to determine the magnitude symbol of BVD. One or more steps of the example flowchart 2900 may be performed by a decoder (e.g., the decoder 300 described herein with reference to Figure 3 or the decoder 2104 described herein with reference to Figure 21).

[0188] In step 2902, the decoder may calculate a cost for each of a plurality of BVD candidates. The BVD candidates may include at least a first and a second BVD candidate. The value of the magnitude symbol of the first BVD candidate may differ from the value of the magnitude symbol of the second BVD candidate. The first and second BVD candidates may be represented in binary form. The first and second BVD candidates may be represented in binary form using unary, truncated unary, kth-order truncated Rice, kth-order Exponential-Golomb (EGk), fixed length, or some combination of two or more of these binarization processes. The first BVD candidate may be represented in binary form using a Golomb codeword that includes the magnitude symbol of the first BVD candidate in a suffix of the Golomb codeword. The Golomb codeword may be an Exponential-Golomb codeword.

[0189] In step 2904, the decoder may select one of the multiple BVD candidates as the BVD predictor. The decoder may select one of the multiple BVD candidates as the BVD predictor based on, for example, cost. The decoder may select one of the multiple BVD candidates as the BVD predictor based on, for example, one of the multiple BVD candidates having a smallest cost. The BVD predictor may be the first BVD candidate or the second BVD candidate.

[0190] In step 2906, the decoder may entropy decode an indication of whether the magnitude symbol of the BVD matches the value of the magnitude symbol of the BVD predictor. The decoder may computationally decode the indication based on a probability model. The probability model may indicate the probability of the least probable symbol for the indication and the value of the most probable symbol for the indication. The decoder may select a probability model from multiple probability models, for example, based on the position of the magnitude symbol in the BVD. The decoder may select a probability model from multiple probability models, for example, based on a change in the value of the BVD relative to an incremental change in the value of the magnitude symbol of the BVD. The decoder may select a probability model from multiple probability models, for example, based on a comparison of the change in the value of the BVD to one or more thresholds.

[0191] In step 2908, the decoder may determine a value of a magnitude symbol of the BVD. The decoder may determine the value of the magnitude symbol of the BVD based on, for example, the value of the magnitude symbol of the BVD predictor and an indication. The decoder may determine the value of the magnitude symbol of the BVD to be equal to the magnitude symbol of the BVD predictor based on, for example, an indication indicating that the value of the magnitude symbol of the BVD matches the value of the magnitude symbol of the BVD predictor. The decoder may determine the value of the magnitude symbol of the BVD to be unequal to the magnitude symbol of the BVD predictor based on, for example, an indication indicating that the value of the magnitude symbol of the BVD does not match the value of the magnitude symbol of the BVD predictor.

[0192] The decoder may further determine a BV based on the sum of the BVD and the BVP. The BV may indicate the displacement of a reference block relative to the current block, and the reference block may be used to predict the current block. The decoder may further calculate a cost for each BVD candidate among the multiple BVD candidates. The decoder may calculate a cost for each BVD candidate among the multiple BVD candidates based on, for example, a difference between a template of the current block and a template of a candidate reference block that is displaced relative to the current block by the sum of the BVD candidate and the BVP. The BVD may be one of the first or second BVD candidates. The first BVD candidate may differ from the second BVD candidate by the value of a magnitude symbol. The magnitude symbol may be either the horizontal or vertical component of the first BVD candidate.

[0193] A decoder (e.g., the decoder 2104 described herein with respect to FIG. 21) may receive an indication in the bitstream of whether to use a local illumination compensation process, and may determine, based on the indication, a difference between a template of the current block and a respective template of each candidate reference block among a plurality of candidate reference blocks. The decoder may predict the current block based on the difference based on the reference block. The reference block may be one of the plurality of candidate reference blocks. The prediction may include using the local illumination compensation process for the reference block according to the indication. For LIC used with IBC, the current block and the plurality of candidate reference blocks may be within the current picture. For LIC used in inter-prediction mode, the plurality of candidate reference blocks may be from one or more pictures different from the current picture of the current block.

[0194] The determining by the decoder may include calculating the difference using a first difference calculation process that may be different from a second difference calculation process used to calculate the difference based on the instruction not to use local illumination compensation, for example, based on the instruction to use local illumination compensation. The second difference calculation process may be less sensitive to illumination differences than the first difference calculation process. The first difference calculation process may include a formula for sum of absolute differences (SAD), and the second difference calculation process may include one of mean-removed SAD (MR-SAD), Hadamard absolute difference (HAD), or sum of absolute transformed differences (SATD).

[0195] The decision at the decoder may include calculating, for each candidate reference block among the plurality of candidate reference blocks, local illumination compensation parameters based on a template of each candidate reference block, a template of the plurality of candidate reference blocks, and a template of the current block, for example, based on an indication to use local illumination compensation. The decoder may use, for each candidate reference block among the plurality of candidate reference blocks, a local illumination compensation process for the template of each candidate reference block among the plurality of candidate reference blocks using the calculated parameters. The decoder may calculate a difference after using the local illumination compensation process. Calculating the difference after use may include calculating the difference using a sum of absolute differences (SAD).

[0196] The determining at the decoder may include searching a predefined search area within the picture to identify each of a plurality of candidate reference blocks by comparing a template of each of the candidate reference blocks with a template of the current block based on the instructions. The determining at the decoder may include identifying a plurality of candidate reference blocks according to block vectors of a predetermined set of neighboring blocks of the current block. The decoder may determine an ordering sequence of the candidate reference blocks according to the differences. Predicting the current block may be based on the ordering sequence. The determining at the decoder may include determining a block vector predictor and a magnitude of the block vector difference for the current block. The decoder may identify (e.g., determine, obtain) a plurality of candidate reference blocks according to the block vector predictor and the magnitude of the block vector difference, and the decoder may determine a sign of the block vector difference, and reconstructing the current block may be further based on the determined sign and the block vector difference.

[0197] The decoder may calculate a cost for each of a plurality of block vector difference (BVD) candidates, including a first BVD candidate and a second BVD candidate. Each of the BVD candidates may correspond to a respective candidate reference block among a plurality of candidate reference blocks. The value of a magnitude symbol of the first BVD candidate may differ from the value of the magnitude symbol of the second BVD candidate. The decoder may select one of the plurality of BVD candidates as a BVD predictor based on the cost. The decoder may entropy decode a second indication indicating whether the value of the magnitude symbol of the BVD matches the value of the magnitude symbol of the BVD predictor. The decoder may determine the value of the magnitude symbol of the BVD based on the value of the magnitude symbol of the BVD predictor and the second indication. The decoder decoding the indication may include computationally decoding the indication based on a probability model indicating a probability of a least likely symbol for the indication and a value of a most likely symbol for the indication. The first BVD candidate may be represented in binary form using a Golomb codeword that includes the magnitude symbol of the first BVD candidate in a suffix of the Golomb codeword.

[0198] At the decoder, the local illumination compensation process may follow p'[x] = alpha * p[x] + beta, where p[x] may represent a sample within the template of the candidate reference block or within the candidate reference block, p'[x] may represent a sample resulting from the local illumination process, and alpha and beta may be determined based on the template of the current block and the template of the candidate reference block.

[0199] The indication of whether to use LIC may be a flag associated with the current block. The bitstream received by the decoder may include a second indication associated with the current block that indicates the use of intra block copy (IBC) mode. The bitstream may include a third indication associated with the current block that indicates the use of template match prediction (intra-TMP) mode.

[0200] An encoder (e.g., the encoder 2102 described herein with respect to FIG. 21) may determine whether the local illumination compensation process may use a difference between a template of the current block and a respective template of each candidate reference block from a plurality of candidate reference blocks. Based on the difference, the encoder may further predict the current block based on a reference block, which may be one of the plurality of candidate reference blocks. The encoder may send (e.g., transmit) in the bitstream a prediction error associated with the reference block and an indication of whether local illumination compensation is applied. The prediction may include using a local illumination process for the reference block according to the indication. For LIC used with IBC, the current block and the plurality of candidate reference blocks may be within the current picture. For LIC used in inter-prediction mode, the plurality of candidate reference blocks may be from one or more pictures different from the current picture of the current block.

[0201] Determining at the encoder may include calculating the difference using a first difference calculation process that may be different from a second difference calculation process used to calculate the difference based on the local illumination compensation process, for example, based on an indication that the local illumination compensation process is not used. The second difference calculation process may be less sensitive to illumination differences than the first difference calculation process. The first difference calculation process may include a formula for sum of absolute differences (SAD), and the second difference calculation process may include one of mean-removed SAD, Hadamard absolute difference (HAD), or sum of absolute transformed differences (SATD).

[0202] The determining at the encoder may include calculating, for each candidate reference block of the plurality of candidate reference blocks, local illumination compensation parameters based on a template of each candidate reference block and a template of the current block based on a local illumination compensation process used, using, for each candidate reference block of the plurality of candidate reference blocks, a local illumination compensation process on the template of each candidate reference block using the calculated parameters, and calculating a difference after using. The calculating a difference after using may include calculating the difference using a sum of absolute differences (SAD).

[0203] Determining at the encoder may include searching a predefined search area within the picture to identify each of the candidate reference blocks by comparing a template of each of the candidate reference blocks with a template of the current block based on whether local illumination compensation is used.

[0204] The determining in the encoder may include identifying a plurality of candidate reference blocks according to block vectors of a predetermined set of adjacent blocks of the current block, and determining an ordering sequence of the candidate reference blocks according to the differences, and predicting the current block is further based on the ordering sequence.

[0205] The determining in the encoder may include determining a block vector predictor and a magnitude of the block vector differential for the current block, identifying a plurality of candidate reference blocks according to the block vector predictor and the magnitude of the block vector differential, and determining a sign of the block vector differential, wherein the predicting is further based on the determined sign.

[0206] The encoder may further perform the following: determining a block vector difference (BVD) based on a difference between the block vector (BV) and the block vector predictor (BVP); calculating a cost for each of a plurality of BVD candidates including a first BVD candidate and a second BVD candidate, where each BVD candidate corresponds to a respective candidate reference block among a plurality of candidate reference blocks, and a magnitude symbol value of the first BVD candidate may differ from a magnitude symbol value of the second BVD candidate; selecting one of the plurality of BVD candidates as a BVD predictor based on the cost; and entropy encoding an indication of whether the magnitude symbol value of the BVD matches the magnitude symbol value of the BVD predictor. Encoding the indication may further include computationally encoding the indication based on a probability model indicating a probability of a least likely symbol for the indication and a value of a most likely symbol for the indication. The first BVD candidate may be represented in binary form using a Golomb codeword that includes the magnitude symbol of the first BVD candidate in a suffix of the Golomb codeword.

[0207] Various embodiments discussed herein with respect to Figures 27-29 may further be used with one or more magnitude symbols of MVD used in inter prediction (e.g., affine model / mode, translational model / mode, or multi-hypothesis prediction mode) in addition to or instead of one or more magnitude symbols of BVD used in IBC. With respect to inter prediction, the terms BV, BVP, BVD, and BVD candidate may be replaced with the terms MV, MVP, MVD, and MVD candidate, respectively. The embodiments discussed herein may be used with IBC and inter prediction. The embodiments discussed herein may be used for IBC and inter prediction, for example, based on a translational motion model of the prediction block. The embodiments discussed herein may be used for IBC and inter prediction, for example, based on an affine motion model of the prediction block. As used herein, the term "bin" may refer to a bit or binary symbol used to encode and decode a BVD or MVD symbol.

[0208] In the encoder, the local illumination compensation process may follow p'[x] = alpha * p[x] + beta, where p[x] may represent a sample within the template of the candidate reference block or within the candidate reference block, p'[x] may represent a sample resulting from the local illumination process, and alpha and beta may be determined based on the template of the current block and the template of the candidate reference block.

[0209] The indication of whether to use LIC may be a flag associated with the current block, i.e., a bitstream that may include a second indication associated with the current block that indicates the use of intra-block copy (IBC) mode. The bitstream may include a third indication associated with the current block. The third indication may indicate the use of template match prediction (intra-TMP) mode.

[0210] An encoder (e.g., the decoder 2102 described herein with respect to FIG. 21) may operate to adjust sample values ​​of a reference block according to local illumination compensation and to reconstruct a current block based on the adjusted reference block and a prediction error from the bitstream.

[0211] Figure 30 illustrates an example computer system. For example, the exemplary computer system 3000 illustrated in Figure 30 may implement one or more of the methods described herein. For example, various devices and / or systems described herein (e.g., Figures 1, 2, and 3) may be implemented in the form of one or more computer systems 3000. Furthermore, each of the steps of the flowcharts illustrated in this disclosure may be implemented on one or more computer systems 3000.

[0212] The computer system 3000 may include one or more processors, such as processor 3004. The processor 3004 may be a special purpose processor, a general purpose processor, a microprocessor, and / or a digital signal processor. The processor 3004 may be connected to a communications infrastructure 3002 (e.g., a bus or network). The computer system 3000 may also include a main memory 3006 (e.g., random access memory (RAM)) and / or a secondary memory 3008.

[0213] The secondary memory 3008 may include a hard disk drive 3010 and / or a removable storage drive 3012 (e.g., a magnetic tape drive, an optical disk drive, and / or the like). The removable storage drive 3012 may be read from and / or written to a removable storage unit 3016. The removable storage unit 3016 may include a magnetic tape, an optical disk, and / or the like. The removable storage unit 3016 may be read by and / or written to the removable storage drive 3012. The removable storage unit 3016 may comprise a computer-usable storage medium having computer software and / or data stored therein.

[0214] The secondary memory 3008 may include other similar means for allowing computer programs or other instructions to be loaded into the computer system 3000. Such means may include a removable storage unit 3018 and / or an interface 3014. Examples of such means may include a program cartridge and / or cartridge interface (such as a video game device), a removable memory chip (such as an erasable programmable read-only memory (EPROM) or a programmable read-only memory (PROM)), and associated sockets, thumb drives, and USB ports, and / or other removable storage units 3018 and interfaces 3014 that may allow software and / or data to be transferred from the removable storage unit 3018 to the computer system 3000.

[0215] Computer system 3000 may also include a communications interface 3020. Communications interface 3020 may allow software and data to be transferred between computer system 3000 and external devices. Examples of communications interface 3020 may include a modem, a network interface (e.g., an Ethernet card), a communications port, etc. Software and / or data transferred via communications interface 3020 may be in the form of signals, which may be electronic, electromagnetic, optical, and / or other signals capable of being received by communications interface 3020. Signals may be provided to communications interface 3020 via communications path 3022. Communications path 3022 may carry signals and may be implemented using wire or cable, fiber optics, a phone line, a cellular phone link, an RF link, and / or any other communications channel.

[0216] Computer program medium and / or computer-readable medium may be used to refer to tangible storage media, such as removable storage units 3016 and 3018, or a hard disk installed in hard disk drive 3010. A computer program product may be a means for providing software to computer system 3000. Computer programs (which may also be referred to as computer control logic) may be stored in main memory 3006 and / or secondary memory 3008. Computer programs may be received via communications interface 3020. Such computer programs, when executed, may enable computer system 3000 to implement the present disclosure as discussed herein. In particular, the computer programs, when executed, may enable processor 3004 to implement processes of the present disclosure, such as any of the methods described herein. Thus, such computer programs may represent controllers of computer system 3000.

[0217] 31 illustrates exemplary elements of a computing device that may be used to implement any of the various devices described herein, including, for example, a source device (e.g., 102), an encoder (e.g., 200), a destination device (e.g., 106), a decoder (e.g., 300), and / or any computing device described herein. The computing device 3130 may include one or more processors 3131 that may execute instructions stored on random access memory (RAM) 3133, removable media 3134 (such as a universal serial bus (USB) drive, a compact disc (CD) or digital versatile disc (DVD), or a floppy disk drive), or any other desired storage medium. Instructions may also be stored on an attached (or internal) hard drive 3135. Computing device 3130 may also include a security processor (not shown) that may execute instructions of one or more computer programs to monitor processes running on processor 3131 and any processes requesting access to any hardware and / or software components of computing device 3130 (e.g., ROM 3132, RAM 3133, removable media 3134, hard drive 3135, device controller 3137, network interface 3139, GPS 3141, Bluetooth interface 3142, WiFi interface 3143, etc.). Computing device 3130 may include one or more output devices such as a display 3136 (e.g., a screen, display device, monitor, television, etc.) and may include one or more output device controllers 3137 such as a video processor. There may also be one or more user input devices 3138 such as a remote control, keyboard, mouse, touch screen, microphone, etc. Computing device 3130 may also include one or more network interfaces such as network interface 3139, which may be a wired interface, a wireless interface, or a combination of the two.The network interface 3139 may provide an interface through which the computing device 3130 communicates with a network 3140 (e.g., a RAN, or any other network). The network interface 3139 may include a modem (e.g., a cable modem), and the external network 3140 may include a communications link, an external network, a home network, a provider's wireless, coaxial, fiber, or hybrid fiber / coaxial distribution system (e.g., a DOCSIS network), or any other desired network. Additionally, the computing device 3130 may include a location detection device such as a global positioning system (GPS) microprocessor 3141, which may be configured to receive and process global positioning signals and, with possible assistance from an external server and antenna, determine the geographic location of the computing device 3130.

[0218] While the example of FIG. 31 may be a hardware configuration, the components shown may equally be implemented as software. Changes may be made, as desired, to add, remove, combine, divide, etc., components of computing device 3130. Additionally, components may be implemented using basic computing devices and components, and the same components (e.g., processor 3131, ROM storage 3132, display 3136, etc.) may be used to implement any of the other computing devices and components described herein. For example, the various components described herein may be implemented using a computing device having components such as a processor that executes computer-executable instructions stored on a computer-readable medium, as shown in FIG. 31. Some or all of the entities described herein may be software-based and coexist on a common physical platform (e.g., a requesting entity may be a separate software process and program from a dependent entity, both of which may run as software on a common computing device).

[0219] A computing device may perform a method including multiple operations. The computing device may determine a reference region corresponding to a direction for flipping based on a reconstruction-reorder intra block copy (RRIBC) mode and a flip direction for flipping a reference block relative to a current block. The computing device may replace a BVP with an adjusted BVP that indicates a position within the reference region. The computing device may replace a BVP with the adjusted BVP based on determining that a block vector predictor (BVP) associated with the current block indicates a position outside the reference region. The computing device may use the adjusted BVP to determine a block vector (BV) of the reference block relative to the current block. The computing device may select a component of the BVP based on the RRIBC mode. The computing device may determine that a sum of the selected component and a dimension of the current block exceeds zero. The computing device may determine that a sum of the selected component and a dimension of the current block exceeds zero if the dimension aligns with the selected component in the direction for flipping. Based on the determination, the computing device may replace the selected component with the adjusted component to determine an adjusted BVP. The computing device may determine a component of the BVP based on the dimensions of the current block, the component being in the direction to flip. The computing device may use the components of the BVP to determine the BV of the reference block relative to the current block. The computing device may determine the components of the BVP based on the RRIBC mode and the direction to flip. The selected component of the BVP may be one of the horizontal component of the BVP if the direction to flip is horizontal, or one of the vertical components of the BVP if the direction to flip is vertical. The dimensions of the current block may be one of the width of the current block if the selected component is the horizontal component of the BVP, or one of the height of the current block if the selected component is the vertical component of the BVP. The BV of the reference block may be determined or predicted without using the unselected components of the BVP.The adjusted component of the adjusted BVP may correspond to a dimension of the current block. The reference region may be within an IBC reference region for the current block. The adjusted BVP may indicate a displacement from the current block to a position within the reference region. The reference region may include a rectangular reference region. Using the adjusted BVP may include determining a block vector difference (BVD) associated with decoding the current block as a difference between the BV and the adjusted BVP. Using the adjusted BVP may include determining an adjusted BVP candidate from a list of candidate BVPs as the adjusted BVP. Determining that the BVP indicates a position outside the reference region may include determining that the component indicates a position outside a range of values ​​that define a reference region for the direction to flip. Determining that the BVP is outside the reference region may include determining that a sum of a selected component and a dimension of the current block exceeds zero, where the dimension is aligned with the selected component in the direction to flip. The selected component may be aligned with the direction to flip. If the selected component is the horizontal component of the BVP, the dimension of the current block may be the width of the current block, and the adjusted component may be equal to the negative value of the width. If the selected component is the vertical component of the BVP, the dimension of the current block may be the height of the current block, and the adjusted component may be equal to the negative value of the height. If the direction for flipping is horizontal, the reference region may include top and bottom boundaries corresponding to the top and bottom boundaries of the current block, a right boundary defined by an offset of the width of the current block to the left boundary of the current block, and a left boundary corresponding to the left boundary of the IBC reference region. If the direction for flipping is vertical, the reference region may include left and right boundaries corresponding to the left and right boundaries of the current block, a bottom boundary defined by an offset of the height of the current block to the top boundary of the current block, and an top boundary corresponding to the top boundary of the IBC reference region. The reference block may be selected from within a rectangular reference region. The components of the block vector predictor may be selected based on the RRIBC mode and the direction for flipping. The reference region may include a reference width and a reference height.If the selected component is a horizontal component of the BVP, the horizontal component of the block vector difference (BVD) between the BV and the BVP may have a maximum possible value equal to the reference width. If the selected component is a vertical component of the BVP, the vertical component of the BVD may have a maximum possible value equal to the reference height. The components of the BVP may be selected based on the RRIBC mode and the direction for flipping. The BVD may include components (e.g., only components) in a direction aligned with the selected component. A residual between the current block and a reference block flipped with respect to the current block may be calculated. A reference block may be determined based on the calculated residual. An adjusted BVP candidate may be selected based on the adjusted component and an index of an adjusted BVP candidate list. The adjusted BVP may be selected as the BVP based on the BVD calculated for each of the candidate BVPs in the list and index of the candidate BVPs. The adjusted BVP may be selected as the one having the smallest cost among the costs calculated for the candidate BVPs, where the cost of the candidate BVP is calculated based on combining the cost of encoding an index corresponding to the candidate BVP and the cost of encoding a BVD corresponding to the candidate BVP. The selecting may further include determining, based on the adjusted component, a component of the BVD indicating a displacement from the current block to the reference block, where the displacement is along a direction aligned with the selected component. The adjusted BVP candidate may be selected without using one or more other components of the adjusted BVP candidate that are not aligned with the selected component. The adjusted BVP candidate may be selected based on one or more other components of the adjusted BVP candidate that are not aligned with the selected component being set to zero. The list of candidate BVPs may include a list of components of the BVP that are aligned with the selected component. Multiple BVP candidates, including the BVP candidate, may be determined. Based on the substitution, the adjusted BVP candidate may be added to the list of candidate BVPs to determine or predict the BV of the current block.A computing device may include one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements. A system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to encode or decode the current block. A computer-readable medium may store instructions that, when executed, cause the computing device to perform the described methods, additional operations, and / or include additional elements.

[0220] A computing device may perform a method including multiple operations. The computing device may select a component of a block vector predictor (BVP) based on a reconstruction-reordering intra block copy (RRIBC) mode. The computing device may determine that a sum of the selected component and a dimension of the current block exceeds zero if the dimension aligns with the selected component in the direction for flipping. Based on the determination, the computing device may replace the selected component with the adjusted component to generate an adjusted BVP. The computing device may use the adjusted BVP to determine a block vector (BV) of a reference block relative to the current block. The computing device may use the adjusted BVP to determine a BV of the reference block relative to the current block. The selecting may be based on the RRIBC mode and a direction for flipping the reference block relative to the current block, and the selected component may be aligned with the flip direction. The dimension may be aligned with the selected component and aligned with the flip direction. The computing device may include one or more processors and memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements. The system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to encode or decode the current block. A computer-readable medium may store instructions that, when executed, cause the described methods, additional operations, and / or include additional elements.

[0221] A computing device may perform a method including multiple operations. The computing device may determine dimensions of a current block aligned with a direction for flipping based on a reconstruction-reorder intra block copy (RRIBC) mode and a flip direction for flipping a reference block relative to the current block. The computing device may determine components of a block vector predictor (BVP) based on the determined dimensions of the current block, the components being in the direction for flipping. The computing device may use the components of the BVP to determine a block vector (BV) of a reference block relative to the current block. The computing device may determine that the BVP is outside the reference region. The selected components of the BVP may be one of a horizontal component of the BVP if the direction for flipping is horizontal, or a vertical component of the BVP if the direction for flipping is vertical. The computing device may include one or more processors and memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements. The system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to encode or decode the current block. A computer-readable medium may store instructions that, when executed, cause the described methods, additional operations, and / or include additional elements.

[0222] A computing device may perform a method including multiple operations. The computing device may determine a reference region corresponding to a direction for flipping based on a reconstruction-reordering intra block copy (RRIBC) mode and a direction for flipping a reference block relative to a current block. The computing device may determine that a block vector predictor (BVP) points outside the reference region. The computing device may replace the BVP with an adjusted BVP that is within the reference region. The computing device may use the adjusted BVP to determine or predict a block vector (BV) of the reference block relative to the current block. A component of the block vector predictor (BVP) may be selected. The component of the block vector predictor may be selected based on the RRIBC mode and the direction for flipping. Determining that a BVP is outside the reference region may include determining that the component is outside a range of values ​​defining the reference region in that direction. Determining that a BVP is outside the reference region may include determining that a sum of the selected component and a dimension (cbDimension) of the current block exceeds zero, where the dimension is aligned with the selected component. The selected component may be aligned with the direction to flip. The selected component of the BVP may be the horizontal component of the BVP if its direction is horizontal, or one of the vertical components of the BVP if its direction is vertical. The dimension of the current block may be the width (cbWidth) of the current block if the selected component is the horizontal component of the BVP, or the height (cbHeight) of the current block if the selected component is the vertical component of the BVP. The BV of the reference block may be determined or predicted without using the non-selected components of the BVP. The adjusted component may be equal to -cbDimension. If the selected component is the horizontal component of the BVP, the dimension of the current block may be the width (cbWidth) of the current block, and the adjusted component may be equal to -cbWidth. If the selected component is the vertical component of the BVP, the dimension of the current block may be the height (cbHeight) of the current block, and the adjusted component may be equal to -cbHeight.The reference region may be within the IBC reference region for the current block. The adjusted BVP may indicate a displacement from the current block to a position within the reference region. The reference region may include a rectangular reference region. If the direction for flipping is horizontal, the reference region may include top and bottom boundaries corresponding to the top and bottom boundaries of the current block, a right boundary defined by an offset of the width (cbWidth) of the current block to the left boundary of the current block, and a left boundary corresponding to the left boundary of the IBC reference region. If the direction for flipping is vertical, the reference region may include left and right boundaries corresponding to the left and right boundaries of the current block, a bottom boundary defined by an offset of the height (cbHeight) of the current block to the top boundary of the current block, and an top boundary corresponding to the top boundary of the IBC reference region. Components of a block vector predictor (BVP) may be selected. The components of the block vector predictor (BVP) may be selected based on the RRIBC mode and the direction for flipping. The reference region may include a reference width and a reference height. If the selected component is a horizontal component of the BVP, the horizontal component of the block vector difference (BVD) between the BV and the BVP may have a maximum possible value equal to the reference width. If the selected component is a vertical component of the BVP, the vertical component of the BVD may have a maximum possible value equal to the reference height. Using the adjusted BVP may include calculating the block vector difference (BVD) as the difference between the BV and the adjusted BVP. Components of the block vector predictor (BVP) may be selected. The components of the block vector predictor (BVP) may be selected based on the RRIBC mode and the direction to flip. The BVD may include components in a direction aligned with the selected component. The BVD may include only components in a direction aligned with the selected component. A residual between the current block and a reference block flipped with respect to the current block may be calculated. The reference block may be determined based on the calculated residual. Using the adjusted BVP may include selecting an adjusted BVP candidate from a list of candidate BVPs as the adjusted BVP. The adjusted BVP candidate may be selected based on the adjusted component and the index of the adjusted BVP candidate in the list.The adjusted BVP may be selected as a BVP based on the BVD calculated for each of the candidate BVPs in the list of candidate BVPs and the index. The adjusted BVP may be selected as the one having the smallest cost among the costs calculated for the candidate BVPs, and the cost of the candidate BVP may be calculated based on combining the cost of encoding the index corresponding to the candidate BVP and the cost of encoding the BVD corresponding to the candidate BVP. Selecting based on the adjusted component may further include determining a component of a block vector difference (BVD) indicating a displacement from the current block to the reference block, where the displacement may be along a direction aligned with the selected component. The adjusted BVP candidate may be selected without using one or more other components of the adjusted BVP candidate that are not aligned with the selected component. The adjusted BVP candidate may further be selected based on setting one or more other components of the adjusted BVP candidate that are not aligned with the selected component to zero. The list of candidate BVPs may include a list of components of the BVP that are aligned with the selected component. Multiple BVP candidates, including the BVP candidate, may be determined. The adjusted BVP candidate may be added to a list of candidate BVPs to determine or predict the BV of a reference block. The adjusted BVP candidate may be added to a list of candidate BVPs to determine or predict the BV of a current block based on the substitution. A computing device may include one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements. A system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to encode or decode the current block. A computer-readable medium may store instructions that, when executed, cause the computing device to perform the described methods, additional operations, and / or include additional elements.

[0223] A computing device may perform a method including multiple operations. A component of a block vector predictor (BVP) may be selected. The component of the BVP may be selected based on a reconstruction-reordering intra block copy (RRIBC) mode. A sum of the selected component and a dimension (cbDimension) of the current block may be determined to be greater than zero, and the dimension may be aligned with the selected component. The selected component may be replaced with the adjusted component to generate an adjusted BVP. The selected component may be replaced with the adjusted component to generate an adjusted BVP based on the determination. The adjusted BVP may be used to determine or predict a block vector (BV) of a reference block relative to the current block. The selecting may be based on the RRIBC mode and a direction for flipping the reference block relative to the current block, and the selected component may be aligned with the flip direction. The dimension may be aligned with the selected component and aligned with the flip direction. The computing device may include one or more processors and memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements. The system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to encode or decode the current block. A computer-readable medium may store instructions that, when executed, cause the described methods, additional operations, and / or include additional elements.

[0224] A computing device may perform a method including multiple operations. Dimensions of a current block may be determined. The dimensions of the current block may be determined based on a reconstruction-reordering intra block copy (RRIBC) mode and a direction for flipping a reference block relative to the current block, aligned with a direction for flipping. A component block vector predictor (BVP) may be determined based on the determined dimensions of the current block, the components being in the direction for flipping. The components of the BVP may be used to determine or predict a block vector (BV) of a current reference block relative to the current block. The computing device may include one or more processors and a memory that stores instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements. A system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to encode or decode the current block. A computer-readable medium may store instructions that, when executed, cause the computing device to perform the described methods, additional operations, and / or include additional elements.

[0225] Various features are highlighted below in sets of numbered clauses or paragraphs. These features are not to be construed as limiting the invention or inventive concept, but are provided merely as highlighting some of the features described herein, without implying the importance or relevance of any particular order of such features.

[0226] Clause 1. A method comprising receiving, in a bitstream, an indication of whether to use local illumination compensation.

[0227] Clause 2. The method of clause 1, further comprising: determining, based on the instructions, a difference between a corresponding template of each candidate reference block of the plurality of candidate reference blocks and a template of the current block.

[0228] Clause 3. The method of clause 1 or 2, further comprising predicting the current block using a reference block from the plurality of candidate reference blocks based on the determined difference, wherein predicting includes using a local illumination compensation process on the reference block in accordance with the instructions.

[0229] Clause 4. The method of any one of clauses 1-3, wherein determining the difference includes determining the difference using a first difference calculation based on instructions instructing to use local illumination compensation, and the first difference calculation is different from a second difference calculation for calculating the difference based on instructions instructing not to use local illumination compensation.

[0230] Clause 5. The method of any one of clauses 1-4, wherein a first difference calculation used to calculate the difference is less sensitive to illumination differences than a second difference calculation.

[0231] Clause 6. The method of any one of clauses 1-5, wherein the first difference calculation includes using a formula for sum of absolute differences (SAD) and the second difference calculation includes using one of mean-removed SAD (MR-SAD), Hadamard absolute difference (HAD), or sum of absolute transformed differences (SATD).

[0232] Clause 7. The method of any one of clauses 1 to 6, wherein determining the difference includes: determining, based on instructions to use a local illumination compensation process, and for each candidate reference block of the plurality of candidate reference blocks, a local illumination compensation parameter based on a template of each candidate reference block and a template of the current block; applying, for each candidate reference block of the plurality of candidate reference blocks, local illumination compensation to the template of the candidate reference block based on the calculated local illumination compensation parameter; and after application, determining the difference using a sum of absolute differences (SAD).

[0233] Clause 8. The method of any one of clauses 1 to 7, wherein the current block and a plurality of candidate reference blocks are in a picture, and determining the difference includes determining each candidate reference block of the plurality of candidate reference blocks within a predefined search area of ​​the picture by comparing a template of each candidate reference block with a template of the current block based on the instructions.

[0234] Clause 9. The method of any one of clauses 1 to 8, wherein determining the difference includes determining a plurality of candidate reference blocks based on block vectors of a predetermined set of neighboring blocks of the current block, and determining an ordering sequence of the candidate reference blocks based on the difference, and predicting the current block is further based on the determined sequence.

[0235] Clause 10. The method of any one of clauses 1-9, further comprising: determining a cost for each of a plurality of block vector difference (BVD) candidates, including a first BVD candidate and a second BVD candidate, each of the BVD candidates corresponding to a respective candidate reference block of a plurality of candidate reference blocks, and a value of a magnitude symbol of the first BVD candidate being different from a value of a magnitude symbol of the second BVD candidate; selecting one of the plurality of BVD candidates as a BVD predictor based on the determined cost; decoding a second instruction indicating whether the value of the magnitude symbol of the BVD matches the value of the magnitude symbol of the BVD predictor; and determining the value of the magnitude symbol of the BVD based on the value of the magnitude symbol of the BVD predictor and the second instruction.

[0236] Clause 11. The method of any one of clauses 1 to 10, wherein the indication is a flag associated with the current block.

[0237] Clause 12. The method of any one of clauses 1 to 11, wherein the bitstream includes a second instruction associated with the current block that indicates use of an intra block copy (IBC) mode.

[0238] Clause 13. The method of any one of clauses 1-12, wherein the bitstream includes a second instruction associated with the current block that indicates use of template match prediction (intra-TMP) mode.

[0239] Clause 14. The method of any one of clauses 1 to 13, wherein predicting the current block is further based on a prediction error received from the bitstream.

[0240] Clause 15. A computing device comprising one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the method of any one of clauses 1 to 14.

[0241] Clause 16. A system comprising: a first computing device configured to implement the method of any one of clauses 1 to 14; and a second computing device configured to send (e.g., transmit) in a bitstream an indication of whether to use a local illumination compensation process.

[0242] Clause 17. A computer-readable medium storing instructions that, when executed, cause the method of any one of clauses 1 to 14 to be performed.

[0243] Clause 18. A method comprising: determining a difference between a corresponding template of each candidate reference block of a plurality of candidate reference blocks and a template of a current block based on whether local illumination compensation is used, wherein the current block and the plurality of candidate reference blocks are in a picture.

[0244] Clause 19. The method of clause 18, further comprising predicting the current block based on a reference block based on the difference, the reference block being one of a plurality of candidate reference blocks, and the predicting comprising using a local illumination process on the reference block according to the instructions.

[0245] Clause 20. The method of clause 18 or 19, wherein the method transmits (eg, transmits) in the bitstream a prediction error associated with the reference block and an indication of whether local illumination compensation is applied.

[0246] Clause 21. The method of any one of clauses 18-20, wherein determining the difference includes determining the difference using a first difference calculation based on instructions indicating that local illumination compensation is to be applied, and the first difference calculation is different from a second difference calculation used to calculate the difference based on instructions indicating that local illumination compensation is not to be used.

[0247] Clause 22. The method of any one of clauses 18 to 21, wherein determining the difference includes determining a block vector predictor and a magnitude of the block vector difference for the current block, determining a plurality of candidate reference blocks based on the block vector predictor and the magnitude of the block vector difference, and determining a sign of the block vector difference, and predicting the current block is further based on the determined sign.

[0248] Clause 23. The method of any one of clauses 18-22, further comprising: determining a block vector difference (BVD) based on a difference between a block vector (BV) and a block vector predictor (BVP); determining a cost for each of a plurality of BVD candidates, including a first BVD candidate and a second BVD candidate, each of the BVD candidates corresponding to a corresponding candidate reference block of a plurality of candidate reference blocks, and a value of a magnitude symbol of the first BVD candidate being different from a value of a magnitude symbol of the second BVD candidate; selecting one of the plurality of BVD candidates as a BVD predictor based on the cost; and entropy encoding an indication of whether the value of the magnitude symbol of the BVD matches the value of the magnitude symbol of the BVD predictor.

[0249] Clause 24. A computing device comprising one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the method of any one of clauses 18 to 23.

[0250] Clause 25. A system comprising: a first computing device configured to implement the method of any one of clauses 18 to 23; and a second computing device configured to receive, in a bitstream, a prediction error associated with a reference block and an indication of whether local illumination compensation is applied.

[0251] Clause 26. A computer-readable medium storing instructions that, when executed, cause the method of any one of clauses 18 to 23 to be performed.

[0252] Clause 27. A method, comprising: determining a plurality of candidate reference blocks according to block vectors of a predetermined set of neighboring blocks of a current block.

[0253] Clause 28. The method of clause 27, further comprising determining a difference between the template of the current block and a respective template of each candidate reference block from a plurality of candidate reference blocks based on application of a local illumination compensation process, wherein the current block and the plurality of candidate reference blocks are within the picture.

[0254] Clause 29. The method of any one of clauses 27 or 28, further comprising determining an ordering sequence of the candidate reference blocks according to the differences.

[0255] Clause 30. The method of any one of clauses 27-29, further comprising predicting the current block based on a reference block based on the difference, the reference block being one of a plurality of candidate reference blocks, predicting the current block based on an ordering sequence, and predicting including using a local illumination process to the reference block according to the instructions.

[0256] Clause 31. The method of any one of clauses 27-30, further comprising transmitting (e.g., transmitting) in the bitstream a prediction error associated with the reference block and an indication of whether a local illumination compensation process is applied.

[0257] Clause 32. The method of any one of clauses 27 to 31, wherein the indication is a flag associated with the current block.

[0258] Clause 33. The method of any one of clauses 27 to 32, wherein the bitstream includes a second instruction associated with the current block that indicates use of an intra block copy (IBC) mode.

[0259] Clause 34. The method of any one of clauses 27-33, wherein the bitstream includes a second instruction associated with the current block that indicates use of template match prediction (intra-TMP) mode.

[0260] Clause 35. A computing device comprising one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the method of any one of clauses 27 to 34.

[0261] Clause 36. A system comprising: a first computing device configured to implement the method of any one of clauses 27 to 34; and a second computing device configured to receive, in the bitstream, a prediction error associated with a reference block and an indication of whether local illumination compensation is applied.

[0262] Clause 37. A computer-readable medium storing instructions that, when executed, cause the method of any one of clauses 27-34 to be performed.

[0263] Clause 38. A method comprising: determining a difference between a template of a current block and a respective template of each candidate reference block from a plurality of candidate reference blocks based on whether a local illumination compensation process is used, the current block and the plurality of candidate reference blocks being in a picture.

[0264] Clause 39. The method of clause 38, further comprising predicting the current block based on a reference block based on the difference, the reference block being one of a plurality of candidate reference blocks, and the predicting comprising using a local illumination process on the reference block according to the instructions.

[0265] Clause 40. The method of clause 38 or 39, further comprising transmitting in the bitstream a prediction error associated with the reference block and an indication of whether a local illumination compensation process is applied.

[0266] Clause 41. The method of any one of clauses 38-40, wherein the second difference calculation is less sensitive to illumination differences than the first difference calculation.

[0267] Clause 42. The method of any one of clauses 38-41, wherein the first difference calculation includes a formula for sum of absolute differences (SAD) and the second difference calculation includes one of mean-removed SAD, Hadamard absolute differences (HAD), or sum of absolute transformed differences (SATD).

[0268] Clause 43. The method of any one of clauses 38 to 42, wherein determining includes calculating, for each candidate reference block of the plurality of candidate reference blocks, a local illumination compensation parameter based on a template of each candidate reference block and a template of the current block based on a local illumination compensation process used; using, for each candidate reference block of the plurality of candidate reference blocks, a local illumination compensation process for the template of each candidate reference block using the calculated parameter; and calculating a difference after using.

[0269] Clause 44. The method of any one of clauses 38 to 43, wherein calculating the difference after use includes calculating the difference using a sum of absolute differences (SAD).

[0270] Clause 45. The method of any one of clauses 38 to 44, wherein determining includes searching a predefined search area within the picture to identify each of the candidate reference blocks by comparing a template of each of the candidate reference blocks with a template of the current block based on whether local illumination compensation is used.

[0271] Clause 46. The method of any one of clauses 38 to 45, wherein determining includes identifying a plurality of candidate reference blocks according to block vectors of a predetermined set of neighboring blocks of the current block, and determining an ordering sequence of the candidate reference blocks according to the differences, and predicting the current block is further based on the ordering sequence.

[0272] Clause 47. The method of any one of clauses 38-46, wherein encoding the indication further comprises computationally encoding the indication based on a probability model that indicates a probability of a least probable symbol for the indication and a value of a most probable symbol for the indication.

[0273] Clause 48. The method of any one of clauses 38-47, wherein the first BVD candidate is represented in binary form using a Golomb codeword that includes a magnitude symbol of the first BVD candidate in a suffix of the Golomb codeword.

[0274] Clause 49. The method of any one of clauses 38 to 48, wherein the local illumination compensation process is according to p'[x] = alpha * p[x] + beta, where p[x] represents a sample in the template of the candidate reference block or in the candidate reference block, and p'[x] represents a sample resulting from the local illumination process, and alpha and beta are determined based on the template of the current block and the template of the candidate reference block.

[0275] Clause 50. The method of any one of clauses 38-49, wherein a difference between a template of the current block after using the local illumination compensation process and a respective template of each candidate reference block after using the local illumination compensation process is determined.

[0276] Clause 51. The method of any one of clauses 38 to 50, wherein determining the difference comprises applying a formula for sum of absolute differences (SAD).

[0277] Clause 52. The method of any one of clauses 38 to 51, wherein the indication is a flag associated with the current block.

[0278] Clause 53. The method of any one of clauses 38-52, wherein the bitstream includes a second instruction associated with the current block, the second instruction indicating use of an intra block copy (IBC) mode.

[0279] Clause 54. The method of any one of clauses 38-53, wherein the bitstream includes a third instruction associated with the current block, the third instruction indicating use of template match prediction (intra-TMP) mode.

[0280] Clause 55. A computing device comprising one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the method of any one of clauses 38 to 54.

[0281] Clause 56. A system comprising: a first computing device configured to implement the method of any one of clauses 38 to 54; and a second computing device configured to receive, in the bitstream, a prediction error associated with a reference block and an indication of whether local illumination compensation is applied.

[0282] Clause 57. A computer-readable medium storing instructions that, when executed, cause the method of any one of clauses 38 to 54 to be performed.

[0283] Clause 58. A method comprising: determining a difference between a corresponding template of each candidate reference block of a plurality of candidate reference blocks and a template of a current block based on whether local illumination compensation is used, wherein the current block and the plurality of candidate reference blocks are in a picture.

[0284] Clause 59. The method of clause 58, further comprising predicting the current block based on a reference block based on the difference, the reference block being one of a plurality of candidate reference blocks, and the predicting comprising using a local illumination process on the reference block according to the instructions.

[0285] Clause 60. The method of clause 58 or 59, further comprising transmitting (eg, transmitting) in the bitstream a prediction error associated with the reference block and an indication of whether local illumination compensation is applied.

[0286] Clause 61. The method of any one of clauses 58 to 60, wherein calculating the difference after use includes calculating the difference using a sum of absolute differences (SAD).

[0287] Clause 62. The method of any one of clauses 58 to 61, wherein determining includes determining a block vector predictor and a magnitude of the block vector difference for the current block, identifying a plurality of candidate reference blocks according to the block vector predictor and the magnitude of the block vector difference, and determining a sign of the block vector difference, and reconstructing the current block is further based on the determined sign and the block vector difference.

[0288] Clause 63. The method of any one of clauses 58-62, wherein decoding the instruction further comprises computationally decoding the instruction based on a probability model that indicates a probability of a least probable symbol for the instruction and a value of a most probable symbol for the instruction.

[0289] Clause 64. The method of any one of clauses 58-63, wherein the first BVD candidate is represented in binary form using a Golomb codeword that includes a magnitude symbol of the first BVD candidate in a suffix of the Golomb codeword.

[0290] Clause 65. The method of any one of clauses 58 to 64, wherein the local illumination compensation process is according to p'[x] = alpha * p[x] + beta, where p[x] represents a sample in the template of the candidate reference block or in the candidate reference block, and p'[x] represents a sample resulting from the local illumination process, and alpha and beta are determined based on the template of the current block and the template of the candidate reference block.

[0291] Clause 66. The method of any one of clauses 58 to 65, wherein a difference between a template of the current block after using the local illumination compensation process and a respective template of each candidate reference block after using the local illumination compensation process is determined.

[0292] Clause 67. The method of any one of clauses 58 to 66, wherein determining the difference comprises applying a formula for sum of absolute differences (SAD).

[0293] Clause 68. A computing device comprising one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the method of any one of clauses 58 to 67.

[0294] Clause 69. A system comprising: a first computing device configured to implement the method of any one of clauses 58 to 67; and a second computing device configured to receive, in the bitstream, a prediction error associated with a reference block and an indication of whether local illumination compensation is applied.

[0295] Clause 70. A computer-readable medium storing instructions that, when executed, cause the method of any one of clauses 58 to 67 to be performed.

[0296] A computing device may perform a method including a number of operations. The computing device may receive an indication in a bitstream of whether to use a local illumination compensation process. Based on the indication, a difference may be determined between a template for each candidate reference block of a plurality of candidate reference blocks and a template of a current block. The current block and a plurality of candidate reference blocks may be in a picture. Based on the determined difference, the current block may be predicted based on the reference block. The reference block may be one of a plurality of candidate reference blocks, and predicting may include using a local illumination compensation process on the reference block in accordance with the indication. Based on the indication, a difference may be determined between the current block and the respective template. Determining the difference between the current block and the respective template may include instructing application of local illumination compensation and calculating the difference using a first difference calculation. A second difference calculation may be used to calculate the difference based on the absence of the indication to use local illumination compensation. The second difference calculation may be less sensitive to illumination differences than the first difference calculation. The first difference calculation may include application of a formula for sum of absolute differences (SAD), and the second difference calculation process may include application of one of mean-removed SAD (MR-SAD), Hadamard absolute differences (HAD), and / or sum of absolute transformed differences (SATD). Determining the difference may be based on an indication indicating application of a local illumination compensation process. Determining the difference may include calculating local illumination compensation parameters based on a template of each candidate reference block and a template of the current block. The calculating may be for each candidate reference block of the multiple candidate reference blocks. A local illumination compensation process may be used with the template of each candidate reference block using the calculated parameters for each candidate reference block of the multiple candidate reference blocks. The difference may be calculated based on the illumination process used. Calculating the difference after application may include calculating the difference using sum of absolute differences (SAD).Determining the difference may include searching a predefined search area within the picture to identify each candidate reference block by comparing a template of each of the candidate reference blocks with a template of the current block based on the instruction. Determining the difference may include identifying multiple candidate reference blocks according to block vectors of a predetermined set of neighboring blocks of the current block and determining an ordering sequence to order the candidate reference blocks according to the difference. Predicting the current block may be further based on the ordering sequence. Determining the difference may include determining a block vector predictor and / or a magnitude of the block vector difference for the current block. Multiple candidate reference blocks may be identified according to the block vector predictor and / or the magnitude of the block vector difference. A sign of the block vector difference may be determined, and reconstructing the current block may be further based on the determined sign and / or block vector difference. Calculating a cost for each of multiple block vector difference (BVD) candidates may be performed. Calculating the cost may include a first BVD candidate and / or a second BVD candidate. Each of the BVD candidates may correspond to one of a plurality of candidate reference blocks, and the value of the magnitude symbol of the first BVD candidate may differ from the value of the magnitude symbol of the second BVD candidate. BVD predictors of the plurality of BVD candidates may be selected based on cost. A second instruction may be entropy decoded to determine whether the value of the magnitude symbol of the BVD matches the value of the magnitude symbol of the BVD predictor. The value of the magnitude symbol of the BVD may be determined based on the value of the magnitude symbol of the BVD predictor and the second instruction. Decoding the instruction may include computationally decoding the instruction based on a probability model that may indicate a probability of the least likely symbol for the instruction and / or a value of the most likely symbol for the instruction. The first BVD candidate may be represented in binary form using a Golomb codeword that may include the magnitude symbol of the first BVD candidate in a suffix of the Golomb codeword. A local illumination compensation process may follow p'[x] = alpha * p[x] + beta.p[x] may represent a sample in the template of the candidate reference block and / or in the candidate reference block, p'[x] may represent a sample resulting from the local illumination process, and alpha and / or beta may be determined based on the template of the current block and / or the template of the candidate reference block. A difference between the template of the current block after using the local illumination compensation process and each template of each candidate reference block after using the local illumination compensation process may be determined. Determining the difference may include applying a formula to sum of absolute differences (SAD). The instruction may be a flag associated with the current block. The bitstream may include a second instruction associated with the current block. The second instruction may indicate the use of intra block copy (IBC) mode. The bitstream may include a third instruction associated with the current block. The third instruction may indicate the use of template match prediction (intra-TMP) mode. Predicting the current block may be further based on a prediction error from the bitstream. The computing device may comprise one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements. The system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to send (e.g., transmit) an indication of whether to use a local illumination compensation process in a bitstream. A computer-readable medium may store instructions that, when executed, cause the computing device to perform the described methods, additional operations, and / or include additional elements.

[0297] A computing device may perform a method including multiple operations. The computing device may determine a difference between a template of each candidate reference block from a plurality of candidate reference blocks and a template of a current block, and it may be determined whether a local illumination compensation process is used. The current block and / or the plurality of candidate reference blocks may be in a picture. Based on the reference block, the current block may be predicted based on the difference. The reference block may be one of the plurality of candidate reference blocks, and predicting may include using a local illumination process on the reference block according to the instruction. A prediction error associated with the reference block and / or an instruction of whether a local illumination compensation process is applied may be transmitted (e.g., transmitted) in a bitstream. Determining the difference may include calculating a difference using a first difference calculation if the local illumination compensation process is used, which may be different from a second difference calculation that may be used to calculate the difference if the local illumination compensation process is not used. The second difference calculation may be less sensitive to illumination differences than the first difference calculation. The first difference calculation process may include an expression for sum of absolute differences (SAD) and / or the second difference calculation may include one of mean-removed SAD, Hadamard absolute difference (HAD), and / or sum of absolute transformed difference (SATD). Determining the difference may include, if a local illumination compensation process is used, calculating local illumination compensation parameters for each candidate reference block of the plurality of candidate reference blocks based on a template for each candidate reference block and / or a template for the current block. For each candidate reference block of the plurality of candidate reference blocks, a local illumination compensation process for the template for each candidate reference block using the calculated parameters may be used. The difference may be determined after the local illumination compensation process may be used. Calculating the difference after using the local illumination compensation process may include calculating the difference using sum of absolute differences (SAD).Determining the difference may include searching a predefined search area in the picture to identify each of the candidate reference blocks by comparing a template of each of the candidate reference blocks with a template of the current block based on whether local illumination compensation may be used. Determining the difference may include identifying multiple candidate reference blocks according to block vectors of a predetermined set of neighboring blocks of the current block. An ordering sequence of the candidate reference blocks may be determined based on the difference. Predicting the current block may be further based on the ordering sequence. Determining the difference may include determining a block vector predictor and / or a magnitude of the block vector difference relative to the current block. Multiple candidate reference blocks may be identified according to the block vector predictor and / or the magnitude of the block vector difference. A sign of the block vector difference may be determined. Predicting may be further based on the determined sign. A block vector difference (BVD) may be determined based on a difference between a block vector (BV) and a block vector predictor (BVP). The difference may be determined based on calculating a cost of each of multiple BVD candidates, which may include a first BVD candidate and / or a second BVD candidate. Each of the BVD candidates may correspond to one of a plurality of candidate reference blocks. The value of the magnitude symbol of a first BVD candidate may differ from the value of the magnitude symbol of a second BVD candidate. The predictor BVD may be selected from the plurality of BVD candidates based on cost. An indication of whether the value of the magnitude symbol of the BVD matches the value of the magnitude symbol of the BVD predictor may be entropy coded. Encoding the indication may further include arithmetically coding the indication based on a probability model that may indicate a probability of the least likely symbol for the indication and / or a value of the most likely symbol for the indication. The first BVD candidate may be represented in binary form using a Golomb codeword that includes the magnitude symbol of the first BVD candidate in a suffix of the Golomb codeword. The local illumination compensation process may follow p'[x] = alpha * p[x] + beta.p[x] may represent a sample within the template of the candidate reference block and / or within the candidate reference block, p'[x] may represent a sample resulting from the local illumination process, and alpha and / or beta may be determined based on the template of the current block and the template of the candidate reference block. A difference between the template of the current block after using the local illumination compensation process and each template of each candidate reference block after using the local illumination compensation process may be determined. Determining the difference may include applying a formula to sum of absolute differences (SAD). The instruction may be a flag associated with the current block. The bitstream may include a second instruction associated with the current block. The second instruction may indicate the use of intra block copy (IBC) mode. The bitstream may include a third instruction associated with the current block. The third instruction may indicate the use of template match prediction (intra-TMP) mode. A computing device may include one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements. The system may comprise a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to receive, in the bitstream, a prediction error associated with the reference block and an indication of whether a local illumination compensation process is applied. A computer-readable medium may store instructions that, when executed, cause the described methods, additional operations, and / or include additional elements.

[0298] A computing device may perform a method including a plurality of operations. The computing device may determine a plurality of candidate reference blocks according to block vectors of a predetermined set of neighboring blocks of a current block. The computing device may determine a difference between a template of each candidate reference block from the plurality of candidate reference blocks and a template of the current block, which may be based on application of a local illumination compensation process. The current block and a plurality of candidate reference blocks may be in a picture. The computing device may determine an ordering sequence of the candidate reference blocks based on the difference. The computing device may predict the current block based on the reference block. The reference block may be one of the plurality of candidate reference blocks based on the difference. Predicting the current block may be based on the ordering sequence. Predicting the current block may include using a local illumination process for the reference block based on the instruction. The computing device may transmit (e.g., transmit) in a bitstream a prediction error associated with the reference block and an instruction of whether the local illumination compensation process is applied. The instruction may be a flag associated with the current block. The bitstream may include a second instruction associated with the current block, which may indicate use of an intra block copy (IBC) mode. The bitstream may include a second instruction associated with the current block that may indicate the use of template match prediction (intra-TMP) mode. The computing device may include one or more processors and a memory that stores instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements. The system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to receive, in the bitstream, a prediction error associated with the reference block and an indication of whether a local illumination compensation process is applied.The computer-readable medium may store instructions that, when executed, cause the described methods, additional operations, and / or include additional elements.

[0299] One or more embodiments herein may be described as a process, which may be depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, and / or a block diagram. A flowchart may describe operations as a sequential process, but one or more of the operations may be performed in parallel or concurrently. The order of operations shown may be rearranged. A process may terminate when its operations are completed, but may have additional steps not shown in the figures. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to the calling function or the main function.

[0300] The operations described herein may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, program code or code segments (e.g., a computer program product) to perform the necessary tasks may be stored on a computer-readable or machine-readable medium. A processor may perform the necessary tasks. Features of the present disclosure may be implemented in hardware using, for example, hardware components such as application-specific integrated circuits (ASICs) and gate arrays. Implementation of hardware state machines to perform the functions described herein will also be apparent to those skilled in the art.

[0301] One or more features described herein may be implemented in computer-usable data and / or computer-executable instructions, such as one or more program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. that, when executed by a processor in a computer or other data processing device, perform particular tasks or implement particular abstract data types. Computer-executable instructions may be stored on one or more computer-readable media, such as hard disks, optical disks, removable storage media, solid-state memory, RAM, etc. The functionality of the program modules may be combined or distributed as desired. Functionality may be implemented in whole or in part in firmware or hardware equivalents, e.g., integrated circuits, field programmable gate arrays (FPGAs), etc. Certain data structures may be used to more effectively implement one or more features described herein, and such data structures are contemplated within the scope of the computer-executable instructions and computer-usable data described herein. Computer-readable media may include, but are not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or carrying instructions and / or data. Computer-readable media may also include non-transitory media that can store data and do not include carrier waves and / or transitory electronic signals propagated via wireless or wired connections. Examples of non-transitory media include, but are not limited to, magnetic disks or tapes, optical storage media such as compact discs (CDs) or digital versatile discs (DVDs), flash memory, memory, or memory devices. Computer-readable media may store code and / or machine-executable instructions, which may represent procedures, functions, subprograms, programs, routines, subroutines, modules, software packages, classes, or any combination of instructions, data structures, or program statements.A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. can be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, or the like.

[0302] A non-transitory tangible computer-readable medium may include instructions executable by one or more processors configured to cause the operations described herein. An article of manufacture may include a non-transitory tangible computer-readable machine-accessible medium encoded with instructions for enabling programmable hardware to cause a device (e.g., an encoder, decoder, transmitter, receiver, and the like) to perform the operations described herein. A device, or one or more devices, such as in a system, may include one or more processors, memory, interfaces, and / or the like.

[0303] Communications described herein may be determined, generated, sent, and / or received using any number of messages, information elements, fields, parameters, values, instructions, information, bits, and / or the like. While one or more embodiments may be described herein using any of the terms / phrases message, information element, field, parameter, value, instruction, information, bit, and / or the like, those skilled in the art will understand that such communications may be implemented using any one or more of these terms, including other such terms. For example, one or more parameters, fields, and / or information elements (IEs) may include one or more information objects, values, and / or any other information. An information object may include one or more other objects. At least some (or all) parameters, fields, IEs, and / or the like may be used and may be interchangeable depending on the context. Where meanings or definitions are given, such meanings or definitions are controlling.

[0304] One or more elements of the examples described herein may be implemented as a module. A module may be an element that performs a defined function and / or has a defined interface to other elements. A module may be implemented in hardware, software combined with hardware, firmware, wetware (e.g., hardware with biological components), or a combination thereof, all of which may be behaviorally equivalent. For example, a module may be implemented as a software routine written in a computer language configured to run on a hardware machine (e.g., C, C++, Fortran, Java, Basic, Matlab, etc.) or Simulink, Stateflow, GNU Octave, or LabVIEW MathScript. Additionally or alternatively, it may be possible to implement a module using physical hardware incorporating discrete or programmable analog, digital, and / or quantum hardware. Examples of programmable hardware include computers, microcontrollers, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and / or complex programmable logic devices (CPLDs). Computers, microcontrollers, and / or microprocessors may be programmed using languages ​​such as assembly, C, C++, etc. FPGAs, ASICs, and CPLDs are often programmed using hardware description languages ​​(HDLs) such as Verilog or VHSIC Hardware Description Language (VHDL), which allow for the construction of connections between the less functional internal hardware modules of the programmable device. The techniques described above may also be used in combination to achieve a functionally modular result.

[0305] One or more of the operations described herein may be conditional. For example, one or more operations may be performed if certain criteria are met, such as the computing device, communication device, encoder, decoder, network, combinations of the above, and / or the like. Exemplary criteria may be based on one or more conditions, such as device configuration, traffic load, initial system setup, packet size, traffic characteristics, combinations of the above, and / or the like. If one or more criteria are met, various embodiments may be used. It may be possible to implement any part of the embodiments described herein in any order and based on any condition.

[0306] Although embodiments are described above, features and / or steps of these embodiments may be combined, divided, omitted, rearranged, modified, and / or extended in any desired manner. Various changes, modifications, and improvements will readily occur to those skilled in the art. Such changes, modifications, and improvements, although not explicitly described herein, are intended to be a part of this specification and are intended to be within the spirit and scope of the description herein. Accordingly, the foregoing description is by way of example only and not by way of limitation.

Claims

1. 1. A method comprising: receiving an indication in the bitstream of whether to use local illumination compensation; determining a difference between a corresponding template of each candidate reference block of a plurality of candidate reference blocks and a template of the current block based on the indication; and predicting the current block by applying the local illumination compensation to a reference block from the plurality of candidate reference blocks in accordance with the instructions based on the determined difference.

2. determining the difference 2. The method of claim 1, comprising determining the difference using a first difference calculation based on the instruction to use the local lighting compensation, the first difference calculation being different from a second difference calculation for calculating the difference based on the instruction not to use the local lighting compensation.

3. The method of claim 1 or 2, wherein the first difference calculation is less sensitive to illumination differences than the second difference calculation.

4. 4. The method of claim 1, wherein the first difference calculation comprises using a formula for sum of absolute differences (SAD), and the second difference calculation comprises using one of mean removed SAD (MR-SAD), Hadamard absolute difference (HAD), or sum of absolute transformed differences (SATD).

5. determining the difference determining, for each candidate reference block among the plurality of candidate reference blocks, local illumination compensation parameters based on the instruction to use a local illumination compensation process and based on the corresponding template of each candidate reference block and the template of the current block; for each candidate reference block of the plurality of candidate reference blocks, applying the local illumination compensation to the template of the candidate reference block based on the determined local illumination compensation parameters; After said applying, determining said difference using a sum of absolute differences (SAD).

6. the current block and the plurality of candidate reference blocks are in a picture, and determining the difference includes:

6. The method of claim 1, further comprising: determining each candidate reference block of the plurality of candidate reference blocks by comparing the template of each candidate reference block with the template of the current block based on the instruction within a predefined search area in the picture.

7. determining the difference determining the plurality of candidate reference blocks based on block vectors of a predetermined set of neighboring blocks of the current block; determining an ordering sequence of the candidate reference blocks based on the differences; The method according to any one of claims 1 to 6, wherein the predicting of the current block is further based on the determined sequence.

8. determining a cost for each of a plurality of block vector difference (BVD) candidates, including a first BVD candidate and a second BVD candidate, each of the BVD candidates corresponding to a respective candidate reference block of the plurality of candidate reference blocks, and a value of a magnitude symbol of the first BVD candidate being different from a value of a magnitude symbol of the second BVD candidate; selecting one of the plurality of BVD candidates as a BVD predictor based on the cost; and decoding a second indication indicating whether a value of a magnitude symbol of a BVD matches a value of a magnitude symbol of the BVD predictor; and determining a value of the magnitude symbol of the BVD based on the value of the magnitude symbol of the BVD predictor and the second indication.

9. The method of any one of claims 1 to 8, wherein the indication is a flag associated with the current block.

10. The method of any one of claims 1 to 9, wherein the bitstream includes a second instruction associated with the current block that indicates the use of an intra block copy (IBC) mode.

11. The method of any one of claims 1 to 10, wherein the bitstream includes a second instruction associated with the current block that indicates the use of a template match prediction (intra-TMP) mode.

12. The method of any one of claims 1 to 11, wherein the predicting of the current block is further based on a prediction error from the bitstream.

13. 1. A computing device comprising: one or more processors; A computing device comprising: a memory storing instructions that, when executed, cause the computing device to perform the method of any one of claims 1 to 12.

14. 1. A system comprising: a first computing device configured to perform the method of any one of claims 1 to 12; a second computing device configured to transmit, in a bitstream, an indication of whether the local illumination compensation is used.

15. A computer readable medium storing instructions that, when executed, cause the method of any one of claims 1 to 12 to be performed.

Images