Video compression using block vector predictor refinement
By employing block vector predictors based on block dimensions and borders, the signaling overhead in video encoding is reduced, enhancing the efficiency of video compression and decoding.
Patent Information
- Application Number
- JP2024519694
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2022-03-03
- Filing Date
- 2022-09-30
- Publication Date
- 2026-01-15
- Estimated Expiration
- 2042-09-30
AI Technical Summary
Existing video encoding and decoding technologies face challenges in efficiently reducing the data size of video sequences due to high signaling overhead in indicating block vectors, which affects storage and transmission efficiency.
The use of block vector predictors (BVPs) that are determined based on the dimensions of the current block and its border within a decoded region, providing a close prediction of the actual block vector to reduce signaling overhead.
This approach reduces the signaling overhead required to indicate block vectors, leading to more efficient video compression and decoding processes.
Smart Images

Figure 0007799820000011 
Figure 0007799820000012 
Figure 0007799820000013
Abstract
Description
[Technical Field]
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Patent Application No. 63 / 250,336, filed September 30, 2021, and U.S. Provisional Patent Application No. 63 / 316,312, filed March 3, 2022. The above-referenced applications are incorporated herein by reference in their entireties. [Background technology]
[0002] Computing devices process video for storage, transmission, reception, and / or display, including encoding and decoding, for example, to reduce the data size associated with the video. Summary of the Invention
[0003] The following summary provides a simplified overview of certain features. It is not an extensive overview and is not intended to identify key or critical elements.
[0004] Video may include a series of frames displayed consecutively. Predictive encoding and decoding may involve using information associated with a block in a frame to encode and / or decode other blocks within the same frame. For example, information associated with a block (e.g., the block's luma and / or chroma components) may be encoded using previously decoded information associated with a reference block within the same frame. The reference block may be indicated in the form of a block vector (BV) that represents the position of the reference block relative to the current block being coded or decoded. The BV itself may be indicated as a function of a block vector predictor (BVP) to reduce the signaling overhead required to directly indicate the BV. An encoder and / or decoder may determine one or more BVPs within a decoded region of a frame. For example, the encoder and / or decoder may determine one or more BVPs based on dimensions of the current block and / or border associated with the reference region. Determining a BVP in the decoded region may help ensure that the BVP is a close prediction of the BV. A BVP, being a close prediction of a BV, may offer additional advantages such as reduced signaling overhead required to indicate a BV.
[0005] These and other features and advantages are described in more detail below. [Brief explanation of the drawings]
[0006] Certain features are illustrated by way of example, and not by way of limitation, in the accompanying drawings in which like numerals refer to like elements and in which:
[0007] [Figure 1] FIG. 1 shows an exemplary video coding / decoding system. [Figure 2] FIG. 2 shows an exemplary encoder. [Figure 3] FIG. 3 shows an exemplary decoder. [Figure 4]FIG. 4 shows an exemplary quadtree division of a coding tree block (CTB). [Figure 5] FIG. 5 shows an exemplary quadtree corresponding to the exemplary quadtree division of the CTB of FIG. [Figure 6] FIG. 6 shows exemplary binary and ternary tree partitioning. [Figure 7] Figure 7 shows an example of a combination of quadtree and multitype tree partitioning of a CTB. [Figure 8] FIG. 8 shows the tree corresponding to the combination of the CTB quadtree and multitype tree partitioning shown in FIG. [Figure 9] FIG. 9 shows an exemplary set of reference samples determined for intra prediction of a current block. [Figure 10A] FIG. 10A shows exemplary intra-prediction modes. [Figure 10B] FIG. 10B shows exemplary intra-prediction modes. [Figure 11] FIG. 11 shows the current block and the corresponding reference samples. [Figure 12] FIG. 12 illustrates the application of intra prediction modes for the prediction of the current block. [Figure 13A] FIG. 13A shows an example of inter prediction. [Figure 13B] FIG. 13B shows an example motion vector. [Figure 14] FIG. 14 shows an example of bi-prediction. [Figure 15A] FIG. 15A shows exemplary spatial candidate neighboring blocks for a current block. [Figure 15B] FIG. 15B shows an example of a block that is temporally co-located with the current block. [Figure 16] FIG. 16 shows an example of intra block copy (IBC) for coding. [Figure 17A] FIG. 17A shows an example of an ICB for encoding. [Figure 17B] FIG. 17B shows an example location of a reference block that is not in the IBC reference region. [Figure 17C] FIG. 17C shows an example location of a reference block within the IBC reference region. [Figure 17D] FIG. 17D illustrates an exemplary method for an adjusted block vector predictor (BVP). [Figure 18] FIG. 18 shows an example of an IBC with an adjusted BVP. [Figure 19] FIG. 19 shows an example of an IBC with an adjusted BVP. [Figure 20] FIG. 20 shows an example of an IBC with an adjusted BVP. [Figure 21A] FIG. 21A shows exemplary IBC reference regions. [Figure 21B] FIG. 21B shows exemplary IBC reference regions. [Figure 22] FIG. 22 illustrates an exemplary method for replacing a BVP with an adjusted BVP. [Figure 23A] FIG. 23A shows an example of candidate BVP adjustment. [Figure 23B] FIG. 23B shows an example of candidate BVP adjustment. [Figure 23C] FIG. 23C shows an example of candidate BVP adjustment. [Figure 24] FIG. 24 shows an example of a distance check that may be performed before adjusting the BVP. [Figure 25] FIG. 25 illustrates an exemplary method for replacing a BVP with an adjusted BVP. [Figure 26] FIG. 26 shows an example of a computer system that may use any of the embodiments described herein. [Figure 27] FIG. 27 illustrates exemplary elements of a computing device that may be used to implement any of the various devices described herein. DETAILED DESCRIPTION OF THE INVENTION
[0008] The accompanying drawings and description provide examples. It should be understood that the examples shown in the drawings and / or description are non-exclusive, and that the features shown and described may be practiced in other embodiments. Examples are provided for the operation of video encoding and decoding systems that may be used in the field of video data storage and / or transmission / reception. More specifically, the techniques disclosed herein may relate to video compression used in encoding and / or decoding devices and / or systems.
[0009] A video sequence containing multiple pictures / frames may be represented in a digital format for storage and / or transmission. Representing a video sequence in a digital format may require a large number of bits. The large data size that may be associated with a video sequence may require significant resources for storage and / or transmission. Video encoding may be used to compress the size of a video sequence for more efficient storage and / or transmission. Video decoding may be used to restore the compressed video sequence for display and / or other forms of consumption.
[0010] 1 shows an exemplary video coding / decoding system. The video coding / decoding system 100 may include a source device 102, a transmission medium 104, and a destination device 106. The source device 102 may encode a video sequence 108 into a bitstream 110 for more efficient storage and / or transmission. The source device 102 may store and / or transmit / transmit the bitstream 110 to the destination device 106 via the transmission medium 104. The destination device 106 may decode the bitstream 110 to display the video sequence 108. The destination device 106 may receive the bitstream 110 from the source device 102 via the transmission medium 104. The source device 102 and / or the destination device 106 may be any of a number of different devices (e.g., a desktop computer, a laptop computer, a tablet computer, a smartphone, a wearable device, a television, a camera, a video game console, a set-top box, a video streaming device, etc.).
[0011] Source device 102 may include one or more of video source 112, encoder 114, and / or output interface 116 (e.g., for encoding video sequence 108 into bitstream 110). Video source 112 may provide and / or generate video sequence 108 based on the capture of natural and / or synthetically generated scenes. Synthetically generated scenes may be scenes including computer-generated graphics and / or screen content. Video source 112 may include a video capture device (e.g., a video camera), a video archive containing previously captured natural and / or synthetically generated scenes, a video feed interface for receiving captured natural and / or synthetically generated scenes from a video content provider, and / or a processor for generating synthetic scenes.
[0012] A video sequence, such as video sequence 108, may include a series of pictures (also called frames). A video sequence may achieve the impression of motion based on the sequential presentation of the pictures of the video sequence using fixed or variable time intervals between pictures. A picture may include one or more sample arrays of intensity values. The intensity values may be obtained (e.g., measured, determined, provided) at a series of regularly spaced locations within the picture. A color picture may (e.g., typically) include a luma sample array and two chroma sample arrays. The luma sample array may include intensity values representing the brightness of the picture (e.g., luminance component, Y). The chroma sample array may include intensity values representing the blue and red components of the picture (e.g., chroma components, Cb and Cr), respectively, separate from the brightness. Other color picture sample arrays are possible based on different color schemes (e.g., red, green, blue (RGB) color schemes). A pixel in a color picture may point to, contain, or associate all intensity values (e.g., luma component, chroma component) for a given location in the sample arrays used to represent the color picture. A monochrome picture may include a single luminance sample array, and a pixel in a monochrome picture may point to / contain / associate an intensity value (e.g., luminance component) at a given location in the single luminance sample array used to represent the monochrome picture.
[0013] The encoder 114 may encode the video sequence 108 into the bitstream 110. The encoder 114 may apply / use one or more prediction techniques (e.g., to encode the video sequence 108) to reduce redundant information in the video sequence 108. The redundant information may include information that can be predicted at a decoder and does not need to be transmitted to the decoder for accurate decoding of the video sequence. For example, the encoder 114 may apply spatial prediction (e.g., intra-frame or intra-prediction), temporal prediction (e.g., inter-frame or inter-prediction), inter-layer prediction, and / or other prediction techniques to reduce redundant information in the video sequence 108. The encoder 114 may, for example, divide a picture including the video sequence 108 into rectangular regions called blocks before applying one or more prediction techniques. The encoder 114 may then encode the blocks using one or more of the prediction techniques.
[0014] For example, for temporal prediction, the encoder 114 may search for a block similar to a block to be coded in another picture (e.g., a reference picture) of the video sequence 108. The encoder 114 may then predict the block to be coded using a block (e.g., a predictive block) determined during the search. For example, for spatial prediction, the encoder 114 may form a predictive block based on data from reconstructed neighboring samples of a block to be coded within the same picture of the video sequence 108. The reconstructed samples may be coded and then decoded samples. The encoder 114 may determine a prediction error (e.g., a residual) based on the difference between the block to be coded and the predictive block. The prediction error may represent non-redundant information that can be transmitted / communicated to a decoder for accurate decoding of the video sequence.
[0015] Encoder 114 may apply a transform to the prediction errors (e.g., using a discrete cosine transform (DCT) or any other transform) to generate transform coefficients. Encoder 114 may form bitstream 110 based on the transform coefficients and other information used to determine the prediction blocks (e.g., prediction type, motion vectors, and prediction mode). Encoder 114 may perform one or more of quantization and entropy coding of the transform coefficients and / or other information used to determine the prediction blocks before forming bitstream 110. The quantization and / or entropy coding may further reduce the amount of bits required to store and / or transmit video sequence 108.
[0016] The output interface 116 may be configured to write and / or store the bitstream 110 on the transmission medium 104 for transmission to the destination device 106. The output interface 116 may be configured to transmit / transmit, upload, and / or stream the bitstream 110 to the destination device 106 via the transmission medium 104. The output interface 116 may include a wired and / or wireless transmitter configured to transmit / transmit, upload, and / or stream the bitstream 110 according to one or more proprietary, open source, and / or standardized communication protocols (e.g., Digital Video Broadcasting (DVB) standards, Advanced Television Systems Committee (ATSC) standards, Integrated Services Digital Broadcasting (ISDB) standards, Data Over Cable Service Interface Specification (DOCSIS) standards, 3rd Generation Partnership Project (3GPP®) standards, Institute of Electrical and Electronics Engineers (IEEE) standards, Internet Protocol (IP) standards, Wireless Application Protocol (WAP) standards, and / or other communication protocols).
[0017] The transmission medium 104 may include wireless, wired, and / or computer-readable media. For example, the transmission medium 104 may include one or more wires, cables, air interfaces, optical disks, flash memory, and / or magnetic memory. The transmission medium 104 may also include one or more networks (e.g., the Internet) or file servers configured to store and / or transmit / transmit encoded video data.
[0018] Destination device 108 may decode bitstream 110 into video sequence 108 for display. Destination device 106 may include one or more of input interface 118, decoder 120, and / or video display 122. Input interface 118 may be configured to read bitstream 110 stored on transmission medium 104 by source device 102. Input interface 118 may be configured to receive, download, and / or stream bitstream 110 from source device 102 via transmission medium 104. Input interface 118 may include a wired and / or wireless receiver configured to receive, download, and / or stream bitstream 110 according to one or more proprietary, open source, standardized communication protocols, and / or any other communication protocols (e.g., as referenced herein).
[0019] The decoder 120 may decode the video sequence 108 from the encoded bitstream 110. The decoder 120 may generate prediction blocks for pictures of the video sequence 108 in a manner similar to the encoder 114, and may, for example, determine prediction errors for blocks to decode the video sequence. The decoder 120 may generate the prediction blocks using / based on the prediction type, prediction mode, and / or motion vectors received in the bitstream 110. The decoder 120 may determine the prediction errors using transform coefficients received in the bitstream 110. The decoder 120 may determine the prediction errors by weighting the transform basis functions using the transform coefficients. The decoder 120 may combine the prediction blocks and the prediction errors to decode the video sequence 108. The decoded video sequence at the destination device may, or may not necessarily, be the same video sequence as transmitted, such as the video sequence 108 transmitted by the source device 102. For example, the decoder 120 may decode a video sequence that approximates the video sequence 108 due to, for example, lossy compression of the video sequence 108 by the encoder 114 and / or errors introduced into the encoded bitstream 110 during transmission to the destination device 106.
[0020] The video display 122 may display the video sequence 108 to a user. The video display 122 may include a cathode ray tube (CRT) display, a liquid crystal display (LCD), a plasma display, a light emitting diode (LED) display, and / or any other display device suitable for displaying the video sequence 108.
[0021] Video encoding / decoding system 100 is merely an example, and video encoding / decoding systems other than video encoding / decoding system 100 and / or modified versions of video encoding / decoding system 100 may perform the methods and processes described herein. For example, video encoding / decoding system 100 may include other components and / or arrangements. Video source 112 may be external to source device 102. Video display device 122 may be external to destination device 106 or may be omitted entirely (e.g., if video sequence 108 is intended for consumption by a machine and / or storage device). Source device 102 may further include a video decoder, and destination device 104 may further include a video encoder. For example, source device 102 may be configured to further receive an encoded bitstream from destination device 106 to support bidirectional video transmission between the devices.
[0022] The encoder 114 and / or the decoder 120 may operate according to one or more proprietary or industry video coding standards. For example, the encoder 114 and / or the decoder 120 may operate according to one or more proprietary, open-source, and / or standardized protocols (e.g., International Telecommunication Union Telecommunication Standardization Sector (ITU-T) H.263, ITU-T H.264, and Motion Picture Experts Group (MPEG)-4 Visual (also known as Advanced Video Coding (AVC)), ITU-T H.265 and MPEG-H Part 2 (also known as High Efficiency Video Coding (HEVC)), ITU-T H.265 and MPEG-I Part 3 (also known as Versatile Video Coding (VVC)), WebM VP8 and VP9 codecs, and / or AOMedia Video 1 (AV1)), and / or other communication protocols.
[0023] FIG. 2 shows an example encoder. The encoder 200 shown in FIG. 2 may implement one or more processes described herein. The encoder 200 may encode a video sequence 202 into a bitstream 204 for more efficient storage and / or transmission. The encoder 200 may be implemented in the video coding / decoding system 100 (e.g., as encoder 114) as shown in FIG. 1 or in any computing, communication, or electronic device (e.g., a desktop computer, a laptop computer, a tablet computer, a smartphone, a wearable device, a television, a camera, a video game console, a set-top box, a video streaming device, etc.). The encoder 200 may include one or more of an inter-prediction unit 206, an intra-prediction unit 208, combiners 210 and 212, a transform and quantization unit (TR+Q) unit 214, an inverse transform and quantization unit (iTR+iQ) 216, an entropy coding unit 218, one or more filters 220, and / or a buffer 222.
[0024] The encoder 200 may divide (e.g., include) a picture (e.g., a frame) of the video sequence 202 into blocks and encode the video sequence 202 block by block. The encoder 200 may perform / apply prediction techniques on the blocks to be encoded using either an inter prediction unit 206 or an intra prediction unit 208. The inter prediction unit 206 may perform inter prediction by searching for a block similar to a block to be encoded in another reconstructed picture (e.g., a reference picture) of the video sequence 202. The reconstructed picture may be an encoded and then decoded picture. The block (e.g., a predictive block) determined during the search may then be used to predict the block to be encoded to remove redundant information. The inter prediction unit 206 may determine the predictive block by exploiting temporal redundancy or similarity in scene content from picture to picture of the video sequence 202. For example, the scene content between pictures of the video sequence 202 may be similar over time except for differences due to motion or affine transformation of screen content.
[0025] The intra prediction unit 208 may perform intra prediction by forming a predictive block based on data from reconstructed neighboring samples of a block encoded within the same picture of the video sequence 202. The reconstructed samples may refer to samples that have been encoded and then decoded. The intra prediction unit 208 may determine the predictive block by exploiting spatial redundancy or similarity in scene content within a picture of the video sequence 202. For example, the texture of a region of scene content in a picture may be similar to the texture of the region immediately surrounding the region of scene content in the same picture.
[0026] The combiner 210 may determine a prediction error (e.g., a residual) based on the difference between the block to be coded and the prediction block. The prediction error may represent non-redundant information that can be transmitted / communicated to a decoder for accurate decoding of the video sequence.
[0027] The transform and quantization unit 214 may transform and quantize the prediction errors. The transform and quantization unit 214 may convert the prediction errors into transform coefficients, for example, by applying a DCT to reduce correlation information of the prediction errors. The transform and quantization unit 214 may quantize the coefficients by mapping the data of the transform coefficients to a set of predetermined representative values. The transform and quantization unit 214 may quantize the coefficients to reduce irrelevant information in the bitstream 204. The irrelevant information may be information that can be removed from the coefficients without producing visible and / or perceptible distortion in the video sequence 202 after decoding (e.g., at a receiving device).
[0028] The entropy coding unit 218 may apply one or more entropy coding methods to the quantized transform coefficients to further reduce the bit rate. For example, the entropy coding unit 218 may apply context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), and / or context-based binary arithmetic coding (SBAC). The entropy-coded coefficients may be packed to form the bitstream 204.
[0029] The inverse transform and quantization unit 216 may inverse quantize and inverse transform the quantized transform coefficients to determine a reconstructed prediction error. The combiner 212 may combine the reconstructed prediction error with the prediction block to form a reconstructed block. The filter 220 may filter the reconstructed block using, for example, a deblocking filter and / or a sample adaptive offset (SAO) filter. The buffer 222 may store the reconstructed block for prediction of one or more other blocks in the same and / or different pictures of the video sequence 202.
[0030] The encoder 200 may further include an encoder control unit. The encoder control unit may be configured to control one or more units of the encoder 200 shown in FIG. 2. The encoder control unit may control one or more units of the encoder 200 so that the bitstream 204 may be generated in accordance with the requirements of one or more proprietary coding protocols, industry video coding standards, and / or any other communication protocol. For example, the encoder control unit may control one or more units of the encoder 200 so that the bitstream 204 is generated in compliance with one or more of ITU-T H.263, AVC, HEVC, VVC, VP8, VP9, AV1, and / or any other video coding standards / formats.
[0031] The encoder control unit may attempt to minimize (or reduce) the bitrate of bitstream 204 and / or maximize (or increase) the reconstructed video quality (e.g., within the constraints of a proprietary coding protocol, an industry video coding standard, and / or any other communication protocol). For example, the encoder control unit may attempt to minimize or reduce the bitrate of bitstream 204 so that the reconstructed video quality does not fall below a particular level / threshold, and / or may attempt to maximize or increase the reconstructed video quality so that the bitrate of bitstream 204 does not exceed a particular level / threshold. The encoder control unit may determine / control one or more of: the division of a picture of the video sequence 202 into blocks, whether the block is inter predicted by the inter prediction unit 206 or intra predicted by the intra prediction unit 208; the motion vector for inter prediction of the block; the intra prediction mode among multiple intra prediction modes for intra prediction of the block; the filtering performed by the filter 220; and / or one or more transform types and / or quantization parameters applied by the transform and quantization unit 214. The encoder control unit may determine / control one or more of the above based on a rate-distortion measurement for the block or picture being coded. The encoder control unit may determine / control one or more of the above to reduce the rate-distortion measurement for the block or picture being coded.
[0032] The prediction type (intra- or inter-prediction) used to encode the block, the prediction information for the block (intra-prediction, intra-prediction mode, motion vector, etc., in the case of intra-prediction), and / or transform and / or quantization parameters may be transmitted to entropy coding unit 218 for further compression (e.g., to reduce bitrate). The prediction type, prediction information, and transform and / or quantization parameters may be packed with the prediction error to form bitstream 204.
[0033] Encoder 200 is merely an example, and encoders other than encoder 200 and / or modified versions of encoder 200 may implement the methods and processes described herein. For example, encoder 200 may have other components and / or arrangements. One or more of the components shown in FIG. 2 may optionally be included in encoder 200 (e.g., entropy coding unit 218 and / or filter 220).
[0034] FIG. 3 shows an exemplary decoder. The decoder 300 shown in FIG. 3 may implement one or more processes described herein. The decoder 300 may decode a bitstream 302 into a decoded video sequence for display and / or some other form of consumption. The decoder 300 may be implemented in the video coding / decoding system 100 of FIG. 1 and / or in a computing, communication, or electronic device (e.g., a desktop computer, a laptop computer, a tablet computer, a smartphone, a wearable device, a television, a camera, a video game console, a set-top box, and / or a video streaming device). The decoder 300 may include an entropy decoding unit 306, an inverse transform and quantization (iTR+iQ) unit 308, a combiner 310, one or more filters 312, a buffer 314, an inter prediction unit 316, and / or an intra prediction unit 318.
[0035] The decoder 300 may include a decoder control unit configured to control one or more units of the decoder 300. The decoder control unit may control one or more units of the decoder 300 such that the bitstream 302 is decoded in accordance with the requirements of one or more proprietary coding protocols, industry video coding standards, and / or any other communication protocol. For example, the decoder control unit may control one or more units of the decoder 300 such that the bitstream 302 is decoded in accordance with one or more of ITU-T H.263, AVC, HEVC, VVC, VP8, VP9, AV1, and / or any other video coding standards / formats.
[0036] The decoder control unit may determine / control one or more of the following, regardless of whether the block is inter predicted by inter prediction unit 316 or intra predicted by intra prediction unit 318: a motion vector for inter prediction of the block, an intra prediction mode among multiple intra prediction modes for intra prediction of the block, filtering performed by filter 312, and / or one or more inverse transform types and / or inverse quantization parameters applied by inverse transform and quantization unit 308. One or more of the control parameters used by the decoder control unit may be packed into the bitstream 302.
[0037] The entropy decoding unit 306 may entropy decode the bitstream 302. The inverse transform and quantization unit 308 may inverse quantize and / or inverse transform the quantized transform coefficients to determine a decoded prediction error. The combiner 310 may combine the decoded prediction error with a prediction block to form a decoded block. The prediction block may be generated by the inter prediction unit 318 or the inter prediction unit 316 (e.g., as described above with respect to the encoder 200 of FIG. 2). The filter 312 may filter the decoded block using, for example, a deblocking filter and / or a sample adaptive offset (SAO) filter. The buffer 314 may store the decoded block for prediction of one or more other blocks in the same and / or different pictures of the video sequence in the bitstream 302. As shown in FIG. 3, the decoded video sequence 304 may be output from the filter 312.
[0038] The decoder 300 is merely an example, and decoders other than the decoder 300 and / or modified versions of the decoder 300 may implement the methods and processes described herein. For example, the decoder 300 may have other components and / or arrangements. One or more of the components shown in FIG. 3 may optionally be included in the decoder 300 (e.g., the entropy decoding unit 306 and / or the filter 312).
[0039] Although not shown in Figures 2 and 3, each of the encoder 200 and the decoder 300 may further include an intra block copy unit in addition to the inter prediction and intra prediction units. The intra block copy unit may perform / operate similarly to the inter prediction unit, but may predict blocks within the same picture. For example, the intra block copy unit may take advantage of repetitive patterns that appear in screen content. The screen content may include computer-generated text, graphics, animation, etc.
[0040] Video encoding and / or decoding may be performed on a block-by-block basis. The process of dividing a picture into blocks may be adaptive based on the content of the picture. For example, to improve coding efficiency, larger block divisions may be used in areas of a picture that have a higher level of homogeneity.
[0041] A picture (e.g., HEVC, or any other coding standard / format) may be divided into non-overlapping square blocks, which may be referred to as coding tree blocks (CTBs). A CTB may contain samples of a sample array. A CTB may have a size of 2n x 2n samples, where n may be specified by parameters of the coding system. For example, n may be 4, 5, 6, or any other value. A CTB may have any other size. A CTB may be further divided by a recursive quadtree division into coding blocks (CBs) of half-vertical and half-horizontal size. A CTB may form the root of the quadtree. CBs that are not further divided as part of the recursive quadtree division may be referred to as leaf CBs of the quadtree, or otherwise may be referred to as non-leaf CBs of the quadtree. A CB may have a minimum size specified by parameters of the coding system. For example, a CB may have a minimum size of 4 x 4, 8 x 8, 16 x 16, 32 x 32, 64 x 64 samples, or any other minimum size. The CB may be further divided into one or more prediction blocks (PBs) to perform inter-prediction and intra-prediction. A PB may be a rectangular block of samples to which the same prediction type / mode may be applied. For transformation, the CB may be divided into one or more transform blocks (TBs). A TB may be a rectangular block of samples that may determine / indicate the applied transform size.
[0042] FIG. 4 shows an example of a quadtree division of a CTB. FIG. 5 shows a quadtree corresponding to the exemplary quadtree division of the CTB 400 of FIG. 4. As shown in FIGS. 4 and 5, the CTB 400 may first be divided into four CBs of semi-vertical and semi-horizontal size. Three of the CBs resulting from the first level division of the CTB 400 may be leaf CBs. The three leaf CBs of the first level division of the CTB 400 are labeled 7, 8, and 9, respectively, in FIGS. 4 and 5. The non-leaf CBs of the first level division of the CTB 400 may be divided into four sub-CBs of semi-vertical and semi-horizontal size. Three of the sub-CBs resulting from the second level division of the CTB 400 may be leaf CBs. The three leaf CBs of the second level division of the CTB 400 are labeled 0, 5, and 6, respectively, in FIGS. 4 and 5. The non-leaf CBs of the second level division of the CTB 400 may be divided into four leaf CBs of semi-vertical and semi-horizontal size. The four lobes CB can be labeled 1, 2, 3, and 4 in Figures 4 and 5, respectively.
[0043] The CTB 400 of FIG. 4 may be divided into ten leaf CBs, labeled 0 through 9, and / or any other number of leaf CBs. The ten leaf CBs may correspond to ten CB leaf nodes (e.g., FIG. 5). In other embodiments, the CTB may be divided into a different number of leaf CBs. The resulting quadtree division of the CTB 400 may be scanned using a z-scan (e.g., left to right, top to bottom) to form a sequence order for encoding / decoding the CB leaf nodes. The numeric label (e.g., indicator, index) of each CB leaf node in FIGS. 4 and 5 may correspond to a sequence order for encoding / decoding. For example, CB leaf node 0 may be encoded / decoded first, and CB leaf node 9 may be encoded / decoded last. Although not shown in FIGS. 4 and 5, each CB leaf node may include one or more PBs and / or TBs.
[0044] Pictures in VVC (or any other coding standard / format) can be partitioned in a similar manner (such as HEVC). Pictures can first be partitioned into non-overlapping square CTBs. The CTBs can then be partitioned into half-vertical and half-horizontal sized CBs using recursive quadtree partitioning. Quadtree leaf nodes (e.g., in VVC) can be further partitioned into unequal sized CBs by binary or ternary tree partitioning (or any other partitioning).
[0045] FIG. 6 illustrates exemplary binary tree and ternary tree partitioning. Binary tree partitioning may divide a parent block in half, either vertically 602 or horizontally 604. The resulting partitions may be half the size compared to the parent block. The resulting partitions may correspond to sizes less than and / or more than half the parent block size. Ternary tree partitioning may divide a parent block into three parts, either vertically 606 or horizontally 608. FIG. 6 illustrates an example in which a middle partition may be twice the size of the other two end partitions in the ternary tree partitioning. In other examples, the partitions may be other sizes relative to each other and the parent block. Binary tree partitioning and ternary tree partitioning are examples of multi-type tree partitioning. Multi-type tree partitioning may include dividing a parent block into other amounts of smaller blocks. A block partitioning strategy (e.g., in VVC) may be referred to as quad-tree + multi-type tree partitioning due to the addition of binary and / or ternary tree partitioning to quad-tree partitioning.
[0046] FIG. 7 shows an example of a combined quadtree and multitype tree partitioning of a CTB. FIG. 8 shows a tree corresponding to the combined quadtree and multitype tree partitioning of CTB 700 shown in FIG. 7. In both FIG. 7 and FIG. 8, the quadtree partitioning is shown with solid lines, and the multitype tree partitioning is shown with dashed lines. CTB 700 is shown with the same quadtree partitioning as CTB 400 described in FIG. 4, and a description of the quadtree partitioning of CTB 700 is omitted. The quadtree partitioning of CTB 700 is merely an example, and the CTB may be quadtree partitioned in a manner different from CTB 700. Additional multitype tree partitioning of CTB 700 may be performed on the three leaf CBs shown in FIG. 4. The three leaf CBs of FIG. 4 shown in FIG. 7 as being further partitioned may be leaves CBs 5, 8, and 9. The three leaf CBs may be further partitioned using one or more binary and ternary tree partitionings.
[0047] Leaf CB5 in Figure 4 can be split into two CBs based on a vertical binary tree partition. The resulting two CBs may be leaf CBs labeled 5 and 6 in Figures 7 and 8, respectively. Leaf CB8 in Figure 4 can be split into three CBs based on a vertical ternary tree partition. Two of the three resulting CBs may be leaf CBs labeled 9 and 14 in Figures 7 and 8, respectively. The remaining non-leaf CBs may be initially split into two CBs based on a horizontal binary tree partition. One of the two CBs may be the leaf CB labeled 10. The other of the two CBs may be further split into three CBs based on a vertical ternary tree partition. The resulting three CBs may be leaf CBs labeled 11, 12, and 13 in Figures 7 and 8, respectively. Leaf CB9 in Figure 4 can be split into three CBs based on a horizontal ternary tree partition. Two of the three CBs may be the leaf CBs labeled 15 and 19 in Figures 7 and 8, respectively. The remaining non-leaf CBs can be split into three CBs based on another horizontal ternary tree split. All three resulting CBs can be leaf CBs, labeled 16, 17, and 18 in Figures 7 and 8, respectively.
[0048] Overall, CTB 700 may be divided into 20 leaf CBs, labeled 0 through 19, respectively. The resulting quadtree + multitype tree division of CTB 700 may be scanned using a z-scan (left to right, top to bottom) to form a sequence order for encoding / decoding the CB leaf nodes. The numeric labels of each CB leaf node in FIGS. 7 and 8 may correspond to the sequence order for encoding / decoding, with CB leaf node 0 being encoded / decoded first and CB leaf node 19 being encoded / decoded last. Note that, although not shown in FIGS. 7 and 8, each CB leaf node may contain one or more PBs and / or TBs.
[0049] A coding standard / format (e.g., HEVC, VVC, or any other of the coding standards / formats) may define various units (e.g., in addition to specifying various blocks (e.g., CTB, CB, PB, TB)). A block may include a rectangular region of samples within a sample array. A unit may include co-located blocks of samples from different sample arrays (e.g., luma and chroma sample arrays) that form a picture and syntax elements and prediction data for the block. A coding tree unit (CTU) may include co-located CTBs of different sample arrays and may form a complete entity in the coded bitstream. A coding unit (CU) may include co-located CBs of different sample arrays and syntax structures used to code samples of the CBs. A prediction unit (PU) may include co-located PBs of different sample arrays and syntax elements used to predict the PBs. A transform unit (TU) may include TBs of different sample arrays and syntax elements used to transform the TBs.
[0050] A block may refer to any of a CTB, CB, PB, TB, CTU, CU, PU, and / or TU (e.g., in the context of HEVC, VVC, or any other coding format / standard). A block may be used to refer to a similar data structure in the context of any video coding format / standard / protocol. For example, a block may refer to a macroblock in the AVC standard, a macroblock or sub-block in the VP8 coding format, a superblock or sub-block in the VP9 coding format, or a superblock or sub-block in the AV1 coding format.
[0051] Samples of a block to be coded (e.g., a current block) may be predicted from samples in columns adjacent to the leftmost column of the current block and samples in rows adjacent to the top row of the current block, such as in intra-prediction. Samples from adjacent columns and rows may be collectively referred to as reference samples. Each sample of the current block may be predicted by projecting the sample's position in the current block in a given direction onto a point along the reference sample (e.g., in intra-prediction mode). If the projection does not fall directly onto the reference sample, the sample may be predicted by interpolating between the two closest reference samples of the projected point. A prediction error (e.g., a residual) may be determined for the current block based on the difference between the predicted sample value and the original sample value of the current block.
[0052] Predicting samples and determining a prediction error based on a difference between the predicted sample and the original sample may be performed (e.g., in an encoder) for multiple different intra-prediction modes (e.g., including a non-directional intra-prediction mode). The encoder may select one of the multiple intra-prediction modes and its corresponding prediction error to encode the current block. The encoder may send an indication of the selected prediction mode and its corresponding prediction error to a decoder for decoding the current block. The decoder may decode the current block by predicting samples of the current block using the intra-prediction mode indicated by the encoder and / or combining the predicted samples with the prediction error.
[0053] 9 illustrates an exemplary set of reference samples determined for intra-prediction of a current block. The current block 904 may correspond to a block to be encoded and / or decoded. The current block 904 may correspond to block 3 of the split CTB 700, as shown in FIG. 7. As described herein, the numeric labels 0-19 of the blocks of the split CTB 700 may correspond to a sequence order for encoding / decoding the blocks and may be used as in the example of FIG. 9.
[0054] The current block 904 may be w × h samples in size. The reference samples 902 may include 2 w samples (or any other number of samples) in a row adjacent to the top row of the current block 904, 2 h samples (or any other number of samples) in a column adjacent to the leftmost column of the current block 904, and an upper-left adjacent corner sample for the current block 904. The current block 904 may be square, such that w = h = s. In other embodiments, the current block need not be square, such that w ≠ h. Available samples from neighboring blocks of the current block 904 may be used to construct the set of reference samples 902. A sample may not be available to construct the set of reference samples 902, for example, if the sample is outside the picture of the current block, if the sample is part of a different slice of the current block (e.g., if the concept of slices is used), and / or if the sample belongs to an inter-coded block and constrained intra prediction is indicated. For example, if constrained intra prediction is indicated, the intra prediction may not depend on the inter-predicted block.
[0055] Samples that are not available for constructing the set of reference samples 902 may include samples within blocks that have not yet been coded and reconstructed at the encoder and / or decoded at the decoder based on the sequence order for encoding / decoding. Restricting such samples from inclusion in the set of reference samples may enable both the encoder and the decoder to determine the same prediction result. Samples from neighboring blocks 0, 1, and 2 may be available for constructing reference samples 902, given that these blocks are coded and reconstructed at the encoder and decoded at the decoder before coding of the current block 904. Samples from neighboring blocks 0, 1, and 2 may be available for constructing reference samples 902, for example, if there are no other issues (e.g., as described above) that prevent the availability of samples from neighboring blocks 0, 1, and 2. A portion of reference samples 902 from neighboring block 6 may not be available due to the sequence order for encoding / decoding (e.g., because block 6 may not yet be coded and reconstructed at the encoder and / or decoded at the decoder based on the sequence order for encoding / decoding).
[0056] Unavailable samples from the reference samples 902 may be filled with one or more of the available reference samples 902. For example, an unavailable reference sample may be filled with the nearest available reference sample. The nearest available reference sample may be determined by moving clockwise from the position of the unavailable reference through the reference samples 902. The reference samples 902 may be filled with, for example, an intermediate value of the dynamic range in which the picture is encoded when a reference sample is unavailable.
[0057] The reference sample 902 may be filtered based on the size of the current block 904 to be coded and the applied intra-prediction mode. Figure 9 shows an example determination of a reference sample for intra-prediction of a block. The reference sample may be determined in a manner different from that described above. For example, multiple reference lines may be used in other instances (e.g., VVC).
[0058] The samples of the current block 904 may be intra predicted based on the reference sample 902, e.g., based on determining the reference sample and (optionally) filtering. At least some (e.g., most) encoders / decoders may support multiple intra prediction modes according to one or more video coding standards. For example, HEVC supports 35 intra prediction modes, including planar mode, direct current (DC) mode, and 33 angular modes. VVC supports 67 intra prediction modes, including planar mode, DC mode, and 65 angular modes. Planar and DC modes may be used to predict smooth and gradually changing regions of a picture. Angular modes may be used to predict directional structure in regions of a picture. Any amount of intra prediction modes may be supported.
[0059] 10A and 10B show exemplary intra prediction modes. FIG. 10A shows 35 intra prediction modes such as those supported by HEVC. The 35 intra prediction modes may be indicated / identified by indexes 0 through 34. Prediction mode 0 may correspond to planar mode. Prediction mode 1 may correspond to DC mode. Prediction modes 2 through 34 may correspond to angular modes. Prediction modes 2 through 18 may be referred to as horizontal prediction modes because the primary prediction source is horizontal. Prediction modes 19 through 34 may be referred to as vertical prediction modes because the primary prediction source is vertical.
[0060] FIG. 10B illustrates 67 intra prediction modes, such as those supported by VVC. The 67 intra prediction modes may be indicated / identified by indexes 0 through 66. Prediction mode 0 may correspond to planar mode. Prediction mode 1 may correspond to DC mode. Prediction modes 2 through 66 may correspond to angular modes. Prediction modes 2 through 34 may be referred to as horizontal prediction modes because the primary prediction source is in the horizontal direction. Prediction modes 35 through 66 may be referred to as vertical prediction modes because the primary prediction source is in the vertical direction. Some of the intra prediction modes illustrated in FIG. 10B may be adaptively replaced by wide-angle directions because VVC blocks need not be square.
[0061] 11 shows a current block and corresponding reference samples. In FIG. 11, the current block 904 and reference samples 902 from FIG. 9 are shown in a two-dimensional x, y plane, and the samples may be referenced as p[x][y]. To simplify the prediction process, the reference samples 902 may be arranged in two one-dimensional arrays. The reference samples 902 above the current block 904 may be arranged in a one-dimensional array ref1[x]. ref1[x]=p[-1+x][-1],(x≧0) (1)
[0062] The reference samples 902 to the left of the current block 904 may be arranged in a one-dimensional array ref2[y]. ref2[y]:=p[-1][-1+y],(y≧0) (2)
[0063] The prediction process may include determining a predicted sample p[x][y] (e.g., a predicted value) at a position [x][y] within the current block 904. In planar mode, the sample at a position [x][y] within the current block 904 may be predicted by determining / calculating the average of two interpolated values. The first of the two interpolated values may be based on a horizontal linear interpolation at the position [x][y] within the current block 904. The second of the two interpolated values may be based on a vertical linear interpolation at the position [x][y] within the current block 904. The predicted sample p[x][y] in the current block 904 may be determined / calculated as follows:
number
[0064] The sample at position [x][y] in the current block 904 may be predicted by the average of the reference samples 902, such as in DC mode. The predicted sample p[x][y] in the current block 904 may be determined / calculated as follows:
number
[0065] The sample at position [x][y] within the current block 904 may be predicted by projecting position [x][y] onto a point on a horizontal or vertical line of samples that includes the reference sample 902, for example, for a given angular mode, in a direction specified by the given angular mode. The sample at position [x][y] may be predicted by interpolating between the two nearest reference samples to the projection point if the projection does not fall directly on the reference sample. The direction specified by the angular mode may be given by an angle φ defined with respect to the y-axis for vertical prediction modes (e.g., modes 19-34 of HEVC and modes 35-66 of VVC). The direction specified by the angular mode may be given by an angle φ defined with respect to the x-axis for horizontal prediction modes (e.g., modes 2-18 of HEVC and modes 2-34 of VVC).
[0066] FIG. 12 illustrates the application of an intra prediction mode for predicting a current block. FIG. 12 specifically illustrates the prediction of a sample at a position [x][y] within a current block 904 for a vertical prediction mode 906. The vertical prediction mode 906 may be given by an angle φ relative to the vertical axis. In the vertical projection mode, the position [x][y] within the current block 904 may be projected to a point (e.g., a projection point) on a horizontal line ref1[x] of the reference sample 902. The reference sample 902 is only partially illustrated in FIG. 12 for ease of illustration. As can be seen in FIG. 12, the projection point on the horizontal line of the reference sample ref1[x] may not be exactly on the reference sample. The prediction sample p[x][y] of the current block 904 may be determined / calculated by linearly interpolating between two reference samples, for example, if the projection point falls on a fractional sample position between the two reference samples. The prediction sample p[x][y] may be determined as follows: p[x][y]=(1-i f )·ref1[x+i i +1]+i f ref1[x+i i +2] (7) i i can be the integer part of the horizontal displacement of the projected point relative to the position [x][y]. imay be determined / calculated as a function of the tangent of the angle φ for vertical prediction mode 906 as follows:
number
number
number
[0067] The position [x][y] of the sample in the current block 904 may be projected onto a vertical line of the reference sample ref2[y], such as for a horizontal prediction mode. The predicted sample p[x][y] for the horizontal prediction mode may be determined / calculated as follows: p[x][y]=(1-i f )·ref2[y+i i +1]+i f ref2[y+i i +2] (10) i i may be the integer part of the vertical displacement of the projected point relative to the position [x][y], and i i may be determined / calculated as a function of the tangent of the angle φ for the horizontal prediction mode as follows:
number
number
number
[0068] The interpolation functions given by equations (7) and (10) may be implemented by an encoder and / or a decoder (e.g., the encoder 200 of FIG. 2 and / or the decoder 300 of FIG. 3). The interpolation functions may be implemented by finite impulse response (FIR) filters. For example, the interpolation functions may be implemented as a set of 2-tap FIR filters. The coefficients of the 2-tap FIR filters are (1-i f ) and i f The prediction samples p[x][y] in angular intra prediction may be calculated at some predetermined level of sample accuracy (e.g., 1 / 32 sample accuracy, or accuracy defined by any other metric). For 1 / 32 sample accuracy, the set of 2-tap FIR interpolation filters may include up to 32 different 2-tap FIR interpolation filters, which can be used to calculate the projection displacement i f In other embodiments, different levels of sample precision may be used.
[0069] An FIR filter may be used to predict the chroma samples and / or the luma samples. For example, a 2-tap interpolating FIR filter may be used to predict the chroma samples, and the same or a different interpolation technique / filter may be used for the luma samples. For example, a 4-tap FIR filter may be used to determine the predicted values for the luma samples. The coefficients of the 4-tap FIR filter are f (e.g., similar to a 2-tap FIR filter). For 1 / 32 sample accuracy, the set of 32 different 4-tap FIR filters may include up to 32 different 4-tap FIR filters, which is determined based on the projection displacement i f One for each of the 32 possible values of the fractional part of i. In other embodiments, different levels of sample precision may be used. The set of 4-tap FIR filters is stored in a look-up table (LUT) and fThe prediction samples p[x][y] for the vertical prediction mode may be determined based on a 4-tap FIR filter as follows:
number
number
[0070] If the position [x][y] of a sample in the current block 904 to be predicted is projected to a negative x-coordinate, a supplemental reference sample may be determined / constructed. The position [x][y] of the sample may be projected to a negative x-coordinate, for example, if a negative vertical prediction angle φ is used. The supplemental reference sample may be determined / constructed by projecting the reference sample of ref2[y] within the vertical line of reference samples 902 onto the horizontal line of reference samples 902 using the negative vertical prediction angle φ. For example, if the position [x][y] of a sample in the current block 904 to be predicted is projected to a negative y-coordinate, a supplemental reference sample may be determined similarly. The position [x][y] of the sample may be projected to a negative y-coordinate, for example, if a negative horizontal prediction angle φ is used. The supplemental reference sample may be determined / constructed by projecting the reference sample of ref2[y] on the horizontal line of reference samples 902 onto the vertical line of reference samples 902 using the negative horizontal prediction angle φ.
[0071] The encoder may determine / predict samples of a current block (e.g., current block 904) to be coded for multiple intra prediction modes (e.g., using one or more of the functions described herein). For example, the encoder may predict samples of the current block for each of the 35 intra prediction modes in HEVC or the 67 intra prediction modes in VVC. For each applied intra prediction mode, the encoder may determine a corresponding prediction error for the current block based on the difference (e.g., sum of squared differences (SSD), sum of absolute differences (SAD), or sum of absolute transformed differences (SATD)) between the predicted samples determined for the intra prediction mode and the original samples of the current block. The encoder may determine / select one of the intra prediction modes to code the current block based on the determined prediction error. For example, the encoder may select the intra prediction mode that results in the smallest prediction error for the current block. The encoder may select the intra prediction mode to code the current block based on a rate-distortion measure (e.g., a Lagrangian rate-distortion cost) determined using the prediction error. The encoder may send an indication of the selected intra-prediction mode and its corresponding prediction error (e.g., residual) to the decoder for decoding of the current block.
[0072] The decoder may determine / predict samples of a current block (e.g., current block 904) to be decoded for an intra-prediction mode. For example, the decoder may receive an indication of a prediction mode (e.g., an angular intra-prediction mode) from the encoder of the block. The decoder may construct a set of reference samples and perform intra-prediction based on the prediction mode indicated by the encoder for the block in a similar manner (e.g., as described above for the encoder). The decoder adds predicted values of the samples of the block (e.g., determined based on intra-prediction) to the residual of the block to reconstruct the block. It is not necessary for the decoder to receive an indication of an angular intra-prediction mode from the encoder of the block. The decoder may determine the intra-prediction mode based on, for example, other criteria. Although various embodiments herein correspond to intra-prediction modes of HEVC and VVC, the methods, devices, and systems described herein may be applied / used for other intra-prediction modes (e.g., as used in other video coding standards / formats such as VP8, VP9, AV1, etc.).
[0073] Intra prediction may utilize correlation between spatially adjacent samples of the same picture of a video sequence to perform video compression. Inter prediction is another coding tool that can be used to perform video compression. Inter prediction may utilize time-domain correlation between sample blocks of different pictures of a video sequence. For example, an object may be visible across multiple pictures of a video sequence. The object may move (e.g., with some translational and / or affine motion) or remain stationary across multiple pictures. A current block of samples of a current picture being encoded may have / be associated with a corresponding block of samples of a previously decoded picture. The corresponding block of samples may accurately predict the current block of samples. The corresponding block of samples may be displaced from the current block of samples due, for example, to an object represented in both blocks moving across the respective pictures of the block. The previously decoded picture may be a reference picture. The corresponding block of samples in the reference picture may be a reference block for motion-compensated prediction. An encoder may use block matching techniques to estimate the displacement (or motion) of an object and / or determine a reference block in a reference picture.
[0074] The encoder may determine a difference between the current block and a prediction for the current block. The encoder may determine the difference, for example, based on / after determining / generating a prediction for the current block (e.g., using inter-prediction). The difference may be referred to as a prediction error and / or a residual. The encoder may then store and / or transmit (e.g., signal) a bitstream, the prediction error, and / or other related prediction information. The prediction error and / or other related prediction information may be used for decoding or consumption in other formats. The decoder may decode the current block by predicting samples of the current block (e.g., by using the related prediction information) and combining the predicted samples with the prediction error.
[0075] FIG. 13A shows an example of inter prediction. Inter prediction may be performed on a current block 1300 of a current picture 1302 to be encoded. An encoder (e.g., the encoder 200 shown in FIG. 2) may perform inter prediction to determine and / or generate a reference block 1304 in a reference picture 1306. The reference block 1304 may be used to predict the current block 1300. The reference picture (e.g., the reference picture 1306) may be a pre-decoded picture available to the encoder and decoder. The availability of a previous decoded picture may depend / be based on whether a previous decoded picture is available in the decoded picture buffer when the current block 1300 is encoded or decoded. The encoder may search one or more reference pictures for a reference block that is similar (or substantially similar) to the current block 1300. The encoder may determine a best-matching reference block from the blocks tested during the search process. The best-matching reference block may be the reference block 1304. The encoder may determine that the reference block 1304 is the best matching reference block based on one or more cost criteria. The one or more cost criteria may include a rate-distortion criterion (e.g., a Lagrangian rate-distortion cost). The one or more cost criteria may be based on the difference (e.g., SSD, SAD, and / or SATD) between the predicted samples of the reference block 1304 and the original samples of the current block 1300.
[0076] The encoder may search for the reference block 1304 within a reference region 1308. The reference region 1308 may be positioned around a collocated position (or block) 1310 of the current block 1300 in the reference picture 1306. The collocated block 1310 may have the same position in the reference picture 1306 as the current block 1300 in the current picture 1302. The reference region 1308 may also be referred to as a search range. The reference region 1308 may extend at least partially outside the reference picture 1306. A certain boundary extension may be used, for example, when the reference region 1308 extends outside the reference picture 1306. A certain boundary extension may be used so that values of samples in a row or column of the reference picture 1306 adjacent to a portion of the reference region 1308 that extends outside the reference picture 1306 may be used for sample positions outside the reference picture 1306. A subset of the potential locations within the reference region 1308, or all potential locations, may be searched for the reference block 1304. The encoder may utilize one or more search implementations to determine and / or generate the reference block 1304. For example, the encoder may determine a set of candidate search locations based on motion information of neighboring blocks relative to the current block 1300.
[0077] One or more reference pictures may be searched by the encoder during inter-prediction to determine and / or generate a best-matching reference block. The reference pictures searched by the encoder may be included in (e.g., added to) one or more reference picture lists. For example, in HEVC and VVC (and / or in one or more other communication protocols), two reference picture lists (e.g., Reference Picture List 0 and Reference Picture List 1) may be used. A reference picture list may contain one or more pictures. A reference picture 1306 of a reference block 1304 may be indicated by a reference index that points to the reference picture list that contains the reference picture 1306.
[0078] 13B shows an example motion vector. The displacement between the reference block 1304 and the current block 1300 may be interpreted as an estimate of the motion between the reference block 1304 and the current block 1300 across the respective pictures. The displacement may be represented by a motion vector 1312. For example, the motion vector 1312 may have a horizontal component (MV) relative to the position of the current block 1300. x ) and vertical component (MV y ) Motion vectors (e.g., motion vector 1312) may have fractional or integer resolution. A motion vector with fractional resolution may point between two samples in a reference picture to provide a better estimate of the motion of the current block 1300. For example, a motion vector may have ½, ¼, ⅛, ⅙, ⅛, ⅙, ⅛, or any other fractional sample resolution. Interpolation between samples at integer positions may be used to generate a reference block and its corresponding samples at fractional positions, for example, if the motion vector points to a non-integer sample value in the reference picture. The interpolation may be performed by a filter with two or more taps.
[0079] The encoder may determine a difference (e.g., a corresponding sample-by-sample difference) between the reference block 1304 and the current block 1300. The encoder may determine the difference between the reference block 1304 and the current block 1300, for example, based on / after the reference block 1304 was determined and / or generated using inter-prediction with respect to the current block 1300. The difference may be referred to as a prediction error and / or a residual. The encoder may store and / or transmit (e.g., a signal) within / via a bitstream, the prediction error and / or associated motion information. The prediction error and / or associated motion information may be used for decoding (e.g., decoding the current block 1300) and / or other forms of consumption. The motion information may include a motion vector 1312 and / or a reference indicator / index. The reference indicator may indicate a reference picture 1306 in a reference picture list. The motion information may include an indication of the motion vector 1312 and / or an indication of the reference index. The reference index may indicate a reference picture 1306 in a reference picture list. A decoder may decode the current block 1300 by determining and / or generating a reference block 1304. The decoder may determine and / or generate the reference block 1304, for example, based on motion information. The reference block 1304 may correspond to / form (e.g., take into account) a prediction of the current block 1300. The decoder may decode the current block 1300 based on combining the prediction with a prediction error.
[0080] As shown in Figure 13A, inter-prediction may be performed using one reference picture 1306 as the source of prediction for a current block 1300. Inter-prediction based on predicting a current block using a single picture may be referred to as uni-prediction.
[0081] 14 shows an example of bi-prediction. Prediction for a current block 1400 may be based on two pictures using bi-prediction. Bi-prediction may be useful, for example, when a video sequence includes fast motion, camera panning, zooming, and / or scene changes. Bi-prediction may be useful for imaging a fade-out from one scene or a fade-out from one scene to another, where two pictures may be effectively displayed simultaneously at different levels of intensity.
[0082] One or both of uni-prediction and bi-prediction may be available / used to perform inter-prediction (e.g., an encoder and / or a decoder). Performing a particular type of inter-prediction (e.g., uni-prediction and / or bi-prediction) may depend on the slice type of the current block 1400. For example, for a P slice, only uni-prediction may be available / used to perform inter-prediction. For a B slice, either uni-prediction or bi-prediction may be used to perform inter-prediction. The encoder may determine and / or generate a reference block for predicting the current block 1400, for example, from reference picture list 0 if the encoder uses uni-prediction. The encoder may determine and / or generate a first reference block for predicting the current block 1400 from reference picture list 0, and may determine and / or generate a second reference block for predicting the current block 1400, for example, from reference picture list 1 if the encoder uses bi-prediction.
[0083] FIG. 14 shows an example of inter-prediction performed using bi-prediction. Two reference blocks 1402 and 1404 may be used to predict a current block 1400. Reference block 1402 may be in one reference picture of reference picture list 0 or reference picture list 1. Reference block 1404 may be in another reference picture of reference picture list 0 or reference picture list 1. As shown in FIG. 14, reference block 1402 may be in a first picture preceding (e.g., temporally) the current picture of current block 1400, and reference block 1402 may be in a second picture following (e.g., temporally) the current picture of current block 1400. The first picture may precede the current picture in terms of a picture order count (POC). The second picture may follow the current picture in terms of a POC. The reference picture may precede or follow the current picture in terms of a POC. The POC may indicate the order in which pictures are output (e.g., from a decoded picture buffer). The POC may be / indicate the order in which pictures are generally intended to be displayed. The output picture may not necessarily be displayed, but may undergo different processing and / or consumption (e.g., transcoding). Two reference blocks determined and / or generated using / for bi-prediction may correspond to (e.g., be included in) the same reference picture. A reference picture may be included in both Reference Picture List 0 and Reference Picture List 1, for example, if the two reference blocks correspond to the same reference picture.
[0084] Configurable weights and / or offset values may be applied to one or more inter-prediction reference blocks. The encoder may enable the use of weighted prediction using a flag in a picture parameter set (PPS). The encoder may transmit / signal the weighting and / or offset parameters in the slice segment header of the current block 1400. Different weights and / or offset parameters may be signaled for the luma and chroma components.
[0085] The encoder may use inter-prediction to determine and / or generate reference blocks 1402 and 1404 for the current block 1400. The encoder may determine a difference between the current block 1400 and each of the reference blocks 1402 and 1404. The difference may be referred to as a prediction error or residual. The encoder may store and / or transmit / signal the prediction errors and their respective associated motion information within / via a bitstream. The prediction errors and their respective associated motion information may be used for decoding or other forms of consumption. The motion information for the reference block 1402 may include a motion vector 1406 and a reference indicator / index. The reference indicator may indicate a reference picture for the reference block 1402 in a reference picture list. The motion information for the reference block 1402 may include an indication of the motion vector 1406 and / or an indication of the reference index. The reference index may indicate a reference picture for the reference block 1402 in a reference picture list.
[0086] The motion information for the reference block 1404 may include a motion vector 1408 and / or a reference index / indicator. The reference indicator may indicate a reference picture for the reference block 1408 in a reference picture list. The motion information for the reference block 1404 may include an indication of the motion vector 1408 and / or an indication of a reference index. The reference index may indicate a reference picture for the reference block 1404 in a reference picture list.
[0087] A decoder may decode the current block 1400 by determining and / or generating reference blocks 1402 and 1404. The decoder may determine and / or generate reference blocks 1402 and 1404, for example, based on respective motion information for reference blocks 1402 and 1404. Reference blocks 1402 and 1404 may correspond to / form (e.g., take into account) a prediction of the current block 1400. The decoder may decode the current block based on combining the prediction with a prediction error.
[0088] The motion information may be predictively coded, for example, before being stored and / or transmitted / signaled in / via a bitstream (e.g., HEVC, VVC, and / or other video coding standards / formats / protocols). The motion information for a current block may be predictively coded based on the motion information of one or more blocks neighboring the current block. The motion information of neighboring blocks may often be correlated with the motion information of the current block because the motion of an object represented in the current block is often the same as (or similar to) the motion of an object in the neighboring block. Motion information prediction techniques may include advanced motion vector prediction (AMVP) and inter-prediction block merging.
[0089] An encoder (e.g., the encoder 200 shown in FIG. 2) may encode a motion vector. The encoder may encode the motion vector (e.g., using AMVP) as the difference between the motion vector of the current block being encoded and a motion vector predictor (MVP). The encoder may determine / select an MVP from a list of candidate MVPs. The candidate MVP may be / correspond to previously decoded motion vectors of neighboring blocks of the current picture of the current block or blocks collocated or near the current block in other reference pictures. The encoder and / or decoder may generate and / or determine the list of candidate MVPs.
[0090] The encoder may determine / select an MVP from a list of candidate MVPs. The encoder may transmit / signal an indication of the selected MVP and the motion vector difference (MVD) in / via the bitstream. The encoder may indicate the selected MVP in the bitstream using an index / indicator. The index may indicate the selected MVP in the list of candidate MVPs. The MVD may be determined / calculated based on the difference between the motion vector of the current block and the selected MVP. For example, the position (e.g., horizontal component (MVD)) relative to the position of the current block being coded may be used. x ) and vertical component (MV y For a motion vector denoted by x and MVD y MVDx and MVDy may be determined / calculated as follows: MVD x =MV x -MVP x (15) MVD y =MV y -MVP y (16) MVD x and MVD y may represent the horizontal and vertical components of the MVD, respectively. x and MVP y may represent the horizontal and vertical components of the MVP, respectively. A decoder (e.g., decoder 300 shown in FIG. 3) may decode a motion vector by adding the MVD to the MVP shown in the bitstream. The decoder may decode a current block by determining and / or generating a reference block. The decoder may determine and / or generate a reference block, for example, based on the decoded motion vector. The reference block may correspond to / form (e.g., take into account) a prediction of the current block. The decoder may decode the current block by combining the prediction with a prediction error.
[0091] A list of candidate MVPs for AMVP (e.g., HEVC, VVC, and / or one or more other communication protocols) may include two or more candidates (e.g., candidate A and candidate B). Candidates A and B may include up to two spatial candidate MVPs determined / derived from five spatial neighboring blocks of the current block being coded, one temporal candidate MVP determined / derived from two temporally co-located blocks (e.g., when both spatial candidate MVPs are unavailable or identical), or a zero motion vector candidate MVP (e.g., when one or both of the spatial candidate MVP or the temporal candidate MVP are unavailable). Other quantities of spatial candidate MVPs, spatial neighboring blocks, temporal candidate MVPs, and / or temporally co-located blocks may be used in the list of candidate MVPs.
[0092] Figure 15A shows spatial candidate neighboring blocks relative to a current block. For example, five (or any other number) spatial candidate neighboring blocks may be located relative to a current block 1500 to be coded. The five spatial candidate neighboring blocks may be A0, A1, B0, B1, and B2. Figure 15B shows temporally co-located blocks relative to the current block. For example, two (or any other number) temporally co-located blocks may be located relative to the current block 1500. The two temporally co-located blocks may be C0 and C1. The two temporally co-located blocks may be in one or more reference pictures that may be different from the current picture of the current block 1500.
[0093] An encoder (e.g., encoder 200 shown in FIG. 2) may encode motion vectors using inter-prediction block merging (e.g., merge mode). The encoder (e.g., using merge mode) may reuse the same motion information of a neighboring block (e.g., one of neighboring blocks A0, A1, B0, B1, and B2) for inter prediction of the current block. The encoder (e.g., using merge mode) may reuse the same motion information of a temporally co-located block (e.g., one of temporally co-located blocks C0 and C1) for inter prediction of the current block. MVD does not need to be transmitted (e.g., indicated or signaled) for the current block because the same motion information as that of the neighboring block or temporally co-located block can be used for the current block (e.g., at the encoder and / or decoder). Because MVD does not need to be indicated for the current block, signaling overhead for transmitting / signaling motion information of the current block may be reduced. The encoder and / or decoder (e.g., both the encoder and the decoder) may generate a candidate list of motion information from neighboring blocks or temporally co-located blocks of a current block (e.g., in a manner similar to AMVP). The encoder may decide to use (e.g., inherit) the motion information of one neighboring block or one temporally co-located block in the candidate list to predict the motion information of the current block being encoded. The encoder may signal / transmit an indication of the motion information determined from the candidate list within / via the bitstream. For example, the encoder may signal / transmit / send an indicator / index. The index may indicate the determined motion information in the list of candidate motion information. The encoder may signal / transmit an index to indicate the determined motion information.
[0094] A list of candidate motion information for a merge mode (e.g., in HEVC, VVC, or any other coding format / standard / protocol) may include up to four (or any other number) spatial merge candidates derived / determined from five (or any other number) spatial neighboring blocks (e.g., as shown in Figure 15A), one (or any other number) temporal merge candidates derived from two (or any other number) temporally co-located blocks (e.g., as shown in Figure 15B), and / or additional merge candidates including both prediction candidates and zero motion vector candidates. The spatial neighboring blocks and temporally co-located blocks used for the merge mode may be the same as the spatial neighboring blocks and temporally co-located blocks used for AMVP.
[0095] Inter prediction may be performed in other ways and variants than those described herein. For example, motion information prediction techniques other than AMVP and merge mode may be used. While various examples herein correspond to inter prediction modes such as those used in HEVC and VVC, the methods, devices, and systems described herein may be applied / used for other inter prediction modes (e.g., as used in other video coding standards / formats such as VP8, VP9, AV1, etc.). History-based motion vector prediction (HMVP), combined intra / inter prediction mode (CIIP), and / or merge mode with motion vector difference (MMVD) (e.g., as described in VVC) may be performed / used and are within the scope of the present disclosure.
[0096] Block matching may be used (e.g., in inter prediction) to determine a reference block of a picture different from the current block being coded. Block matching may also be used to determine a reference block of the same picture as that of the current block being coded. When a reference block is determined using block matching in the same picture as the current block, it often cannot accurately predict the current block (e.g., in the case of video captured by a camera). Prediction accuracy for screen video content may not be similarly affected, for example, when a reference block in the same picture as the current block is used for coding. Screen content video may include, for example, computer-generated text, graphics, animation, etc. Screen video content may (e.g., often does) include repetitive patterns (e.g., repetitive patterns of text and graphics) within the same picture. Using a reference block (e.g., determined using block matching) in the same picture as the current block being coded may provide efficient compression for screen content video.
[0097] Prediction techniques may be used (e.g., in HEVC, VVC, and / or any other coding standard / format / protocol) to exploit correlation between blocks of samples within the same picture (e.g., of screen content video). Prediction techniques may be referred to as intra block copy (IBC) or current picture reference (CPR). The encoder may apply / use a block matching technique (e.g., similar to inter prediction) to determine a displacement vector (e.g., block vector (BV)). The BV may indicate the relative position of a reference block that best matches the current block from the position of the current block (e.g., according to intra block compensated prediction). For example, the relative position of the reference block may be the relative position of the upper left corner (or any other point / sample) of the reference block. The BV may indicate the relative displacement from the current block to the reference block that best matches the current block. The encoder may determine the best matching reference block from the blocks tested during the search process (e.g., in a manner similar to that used for inter prediction). The encoder may determine that the reference block is the best matching reference block based on one or more cost criteria. The one or more cost criteria may include a rate-distortion criterion (e.g., a Lagrangian rate-distortion cost). The one or more cost criteria may be based, for example, on one or more differences (e.g., differences determined based on SSD, SAD, SATD, and / or a hash function) between predicted samples of the reference block and original samples of the current block. The reference block may correspond to / include a previously decoded block of samples of the current picture. The reference block may include a decoded block of samples of the current picture before being processed by an in-loop filtering operation (e.g., deblocking and / or SAW filtering).
[0098] Figure 16 shows an example of an IBC for coding. The exemplary IBC shown in Figure 16 may correspond to screen content. The rectangular portion / section with the arrow starting at their boundary may be the current block to be coded. The rectangular portion / section to which the arrow points may be a reference block for predicting the current block.
[0099] A reference block may be determined and / or generated for the current block for IBC. The encoder may determine a difference (e.g., corresponding sample-by-sample difference) between the reference block and the current block. The difference may be referred to as a prediction error or residual. The encoder may store and / or transmit / signal the prediction error and / or associated prediction information within / via a bitstream. The prediction error and / or associated prediction information may be used for decoding and / or other forms of consumption. The prediction information may include a BV. The prediction information may include an indication of the BV. A decoder (e.g., decoder 300 shown in FIG. 3) may decode the current block by determining and / or generating a reference block. The decoder may determine and / or generate the current block, for example, based on the prediction information (e.g., a BV). The reference block may correspond to / form (e.g., take into account) a prediction of the current block. The decoder may decode the current block by combining the prediction with the prediction error.
[0100] The BVs may be predictively encoded (e.g., in HEVC, VVC, and / or any other coding standard / format / protocol) before being stored and / or transmitted / signaled in / via a bitstream. The BVs for a current block may be predictively encoded based on BV blocks neighboring the current block. For example, the encoder may predictively encode the BVs using merge mode (e.g., in a manner similar to that described herein for inter prediction), AMVP (e.g., as described herein for inter prediction), or an AMVP-like technique. The AMVP-like technique may be BV prediction and differential coding (or AMVP for IBC).
[0101] An encoder that performs BV prediction and encoding (e.g., the encoder 200 shown in FIG. 2) may encode the BV as the difference between the BV of the current block being encoded and a BV predictor (BVP). The encoder may select / determine a BVP from a list of candidate BVPs. The candidate BVPs may include / correspond to previously decoded BVs of blocks neighboring the current block in the current picture. The encoder and / or decoder may generate or determine the list of candidate BVPs.
[0102] The encoder may transmit / signal an indication of the selected BVP and BV difference (BVD) in / via the bitstream. The encoder may indicate the selected BVP in the bitstream using an index / indicator. The index may indicate the selected BVP in a list of candidate BVPs. The BVD may be determined / calculated based on the difference between the BV of the current block and the selected BVP. For example, the horizontal component (BV) relative to the position of the current block being coded may be used to determine the difference between the BV of the current block and the selected BVP. x ) and vertical component (BVy), BVD is a function of two components BVD x and BVD y It can be expressed as: BVD x and BVD y can be determined / calculated as follows: BVD x =BV x -BVP x (17) BVD y =BV y -BVP y (18) BVD x and BVD y may represent the horizontal and vertical components of the BVD, respectively. x and BVP ymay represent the horizontal and vertical components of the BVP, respectively. A decoder (e.g., decoder 300 shown in FIG. 3) may decode the BV by appending the BVD to the BVP indicated in / via the bitstream. The decoder may decode the current block by determining and / or generating a reference block. The decoder may determine and / or generate a reference block, for example, based on the decoded BV. The reference block may correspond to / form (e.g., take into account) a prediction of the current block. The decoder may decode the current block by combining the prediction with the prediction error.
[0103] The same BV as the neighboring block may be used for the current block, and there is no need to separately signal / transmit the BVD for the current block, such as in merge mode. The BVP (in the candidate BVP) that may correspond to the decoded BV of the neighboring block may itself be used as the BV of the current block. Not transmitting the BVD may reduce signaling overhead.
[0104] A list of candidate BVPs (e.g., in HEVC, VVC, and / or any other coding standard / format / protocol) may include two (or more) candidates. The candidates may include candidates A and B. Candidates A and B may include up to two (or any other number) spatial candidate BVPs determined / derived from five (or any other number) spatial neighboring blocks of the current block to be coded, and / or one or more of the last two (or any other number) coded BVs (e.g., when spatial neighboring candidates are unavailable). Spatial neighboring candidates may not be available, for example, when neighboring blocks are coded using intra prediction or inter prediction. The positions of spatial candidate neighboring blocks relative to a current block coded using IBC may be illustrated in a manner similar to the spatial candidate neighboring blocks used to code motion vectors in inter prediction (e.g., as shown in FIG. 15A). For example, the five spatial candidate neighboring blocks of IBC may be denoted as A0, A1, B0, B1, and B2, respectively.
[0105] In at least some encoders and / or decoders, a BV for a current block (e.g., coded using IBC) may be constrained to indicate a relative displacement from the current block to a reference block within a reference region (e.g., an IBC reference region). The reference region may correspond to at least a portion of a region in the current picture of the current block that was previously coded (e.g., in an encoder) and subsequently decoded (e.g., in a decoder). Constraining the reference block within the reference region may ensure that encoding and decoding systems can produce identical results when processing the current block. A BVP used to predictively code a BV may not be similarly constrained within the reference region. The BVP may not be similarly constrained because it may be determined / derived based on BVs of spatially neighboring blocks of the current block and / or previously coded (e.g., history-based) BVs. The BVP may not accurately predict the BV because it may not be constrained (e.g., to indicate a relative displacement from the current block to the reference block) within the IBC reference region. For example, the BVP may not be an approximate estimate of the BV. Inaccurate prediction of the BV using an unconstrained BVP may cause one or more problems, such as a possible increase in signaling overhead (e.g., the amount of bits required) to indicate the BVD between the BV and the BVP.
[0106] Various examples described herein provide improvements to a BVP, for example, by adjusting the BVP to provide a more accurate prediction of the BV (e.g., determined using an IBC procedure). A device (e.g., an encoder, a source device, a computing device, etc.) may adjust the BVP based on one or more criteria. The one or more criteria may be based on a position relative to a position of a current block being encoded, as indicated by the BVP. The one or more criteria may include determining whether the position is within a reference region (e.g., an IBC reference region). The one or more criteria may include determining whether the position is at least one or more offsets away from the current block in one or more predetermined directions. The one or more criteria may be based on dimensions of the current block being encoded. The one or more criteria may be based on a position indicated by the BVP relative to one or more boundaries of the reference region. The encoder may use an adjusted BVP based on one or more criteria being met or not met. For example, the encoder may replace a BVP (e.g., as present in a list) with the adjusted BVP. Using an adjusted BVP and / or replacing a BVP with an adjusted BVP may provide advantages such as reducing the signaling overhead required for transmitting coded video. For example, the amount of bits required to represent the BVD of a block may be reduced if the BVP is replaced by an adjusted BVP.
[0107] The encoder may determine that the BVP points to a position that is neither at least a first offset to the left of the current block nor at least a second offset above the current block relative to the position of the current block. The first offset may be equal to the width of the current block. The second offset may be equal to the height of the current block. The encoder may determine that the BVP points to a position outside the reference region (e.g., the IBC reference region) of the current block, for example, if the BVP points to a position that is neither at least a first offset distance to the left of the current block nor at least a second offset distance above the current block relative to the position of the current block. The encoder may, for example, use an adjusted BVP based on the BVP pointing outside the reference region. The encoder may, for example, replace the BVP with the adjusted BVP based on the BVP pointing outside the reference region. The adjusted BVP may be used to determine or predict (e.g., code and / or decode) the BV for the current block. These and other features that result in improved BVPs are further described below.
[0108] FIG. 17A shows an example of IBC for encoding. An encoder (e.g., the encoder 200 shown in FIG. 2) may encode a current block 1700 in a current picture (or a portion of a current picture) 1702 using IBC mode. The current block 1700 may be a coding block (CB) in a coding tree unit (CTU) 1704. An encoder using IBC may search for a reference block in the current picture that is the same as that of the current block. Only a portion of the current picture may be available for searching for the reference block. For example, only a portion of the current picture decoded before encoding the current block may be available for searching for the reference block. The portion of the current picture available for searching for the reference block may be an IBC reference region. Searching only a portion of the current picture decoded before encoding the current block may ensure that the encoding and decoding systems may produce identical results but may limit the IBC reference region.
[0109] Blocks may be scanned in a particular order. Blocks may be scanned from left to right and top to bottom using z-scan (e.g., in HEVC, VVC, and / or other coding standards / formats / protocols) to form a sequence order for encoding / decoding. CTUs (represented by large square tiles, as shown in FIG. 17A ) to the left and above the current CTU 1704 may be encoded / decoded before the current CTU 1704 and the current block 1700, for example, based on z-scan. Samples of these CTUs (e.g., CTUs 1703-1 through 1703-6) may form an exemplary IBC reference region 1706 for determining a reference block for predicting the current block 1700. A different sequence order for encoding / decoding may be used in other video encoders / decoders. Using a different sequence order may modify the IBC reference region 1706 accordingly.
[0110] The IBC reference region 1706 may represent the location of a valid reference block. The IBC reference region 1706 (e.g., as indicated by shading) may be defined / represented in the form of a valid location / position of a reference block that can be used for encoding / decoding / predicting the current block 1700. The location of the reference block may be defined as the location / position of the top-left corner of the reference block. A reference block whose top-left corner is outside or coincides with a boundary (e.g., a bottom-most or right-most boundary) of the IBC reference region 1706 may be at least partially outside the IBC reference region and / or may coincide with the current block 1700. A reference block whose top-left corner is outside or coincides with a boundary (e.g., a bottom-most or right-most boundary) of the IBC reference region 1706 may be considered to be located outside the IBC reference region 1706. The location of the reference block may be defined relative to the current block 1700 using a BV.
[0111] The IBC reference region 1706 may not include an L-shaped region surrounding the left and top of the current block 1700. The L-shaped region may be defined in terms of the width (e.g., cbWidth) of the current block 1700 and the height (e.g., cbHeight) of the current block. For example, the left of the L-shaped region may have a width equal to (cbWidth-1), and the top of the L-shaped region may have a height equal to (cbHeight-1). The IBC reference region 1706 may not include the L-shaped region to prevent the BV from pointing to a reference block that may overlap (at least partially) with the current block 1700 or an unreconstructed region of the current picture 1702.
[0112] 17B shows an example location of a reference block that is not in the IBC reference region. For example, the upper left corner 1707 of the reference block 1709 may coincide with a boundary (e.g., the right edge or bottom edge) of the IBC reference region 1706 and / or may be outside the IBC reference region 1706. The reference block 1709 may be considered to be outside the IBC reference region 1706.
[0113] 17C shows example locations of reference blocks within the IBC reference region. For example, the upper left corner 1711 of reference block 1713 may be within IBC reference region 1706. Reference block 1713 may be considered to be outside IBC reference region 1706, for example, based on the fact that the upper left corner 1711 of reference block 1713 is within IBC reference region 1706.
[0114] One or more reference region constraints (e.g., in addition to the encoding / decoding sequence order) may be placed on the IBC reference region 1706. For example, the IBC reference region 1706 may be constrained based on limited memory (e.g., in an encoder) for storing reference samples. The IBC reference region 1706 may be constrained to CTUs determined based on a parallel processing approach (e.g., tiled or wavefront parallel processing (WPP)). Tiles may be used as part of a picture partitioning process to flexibly subdivide a picture into rectangular regions of CTUs such that coding dependencies between CTUs of different tiles are not allowed. WPP may similarly be used as part of a picture partitioning process to partition a picture into CTU rows. The partitioning into CTU rows may be such that dependencies between CTUs of different partitions are not allowed. The use of tiles or WPP may enable parallel processing of the picture partitioning.
[0115] The encoder may use / apply a block matching technique to determine a block vector (BV) 1708. The BV may indicate the relative displacement from the current block 1700 to a reference block 1710 within the IBC reference region 1706. The reference block 1710 may be a block that best matches the current block 1700 (e.g., according to intra-block compensated prediction). The IBC reference region 1706 may be a constraint that may be applied to the BV 1708. The BV 1708 may be constrained by the IBC reference region 1706 to indicate the displacement from the current block 1700 to a reference block that is within the IBC reference region 1706. The encoder may determine the best-matching reference block 1710 from among the blocks tested during the search process (e.g., within the IBC reference region 1706). The encoder may determine that the reference block 1710 may be the best matching reference block based on one or more cost criteria, where the one or more cost criteria may include a rate-distortion criterion (e.g., a Lagrangian rate-distortion cost). The one or more cost criteria may be based, for example, on one or more differences (e.g., one or more of SSD, SAD, SATD, and / or differences determined based on a hash function) between predicted samples of the reference block 1710 and original samples of the current block 1700. The reference block 1710 may include decoded (and / or reconstructed) samples of the current picture 1702 before being processed by in-loop filtering operations (e.g., deblocking and / or SAO filtering).
[0116] The encoder may determine and / or use the difference (e.g., corresponding sample-by-sample difference) between the current block 1700 and the reference block 1710. The difference may be referred to as a prediction error or residual. The encoder may store and / or transmit / signal the prediction error and associated prediction information for decoding in / via the bitstream.
[0117] The prediction information may include BV 1708. The prediction information may include an indication of BV 1708. BV 1708 may be predictively encoded (e.g., in HEVC, VVC, and / or other video compression formats / standards / protocols) before being stored and / or transmitted / signaled within / via a bitstream. BV 1708 of current block 1700 may be predictively encoded using techniques similar to AMVP applied to inter prediction. Techniques similar to AMVP (applied to inter prediction) may be referred to as BV prediction and differential coding for IBC, or AMVP. An encoder may encode BV 1708 as the difference between BV 1708 and BVP 1712 using BV prediction and differential coding. The encoder may select / determine BVP 1712 from a list of candidate BVPs. The candidate BVPs may correspond to / include previously decoded BVs of neighboring blocks of current block 1700 or may originate from other sources. Invalid BVP candidates (e.g., having x and y components with magnitudes of zero) may be added to the list of candidate BVPs, for example, if BVs from neighboring blocks of the current block 1700 are unavailable. The encoder and / or decoder may generate and / or determine the list of candidate BVPs.
[0118] The encoder may determine the BVD 1714. The BVD 1714 may be determined / calculated based on the difference between the BV 1708 and the BVP 1712. For example, the BVD 1714 may be calculated based on the difference between the two directional components BVD x and BVD y It can be expressed as: BVD x and BVD y can be determined / calculated as follows: BVD x =BV x -BVP x (17) BVD y =BV y -BVP y (18) BVD x and BVD ymay represent the horizontal and vertical components of the BVD 1714, respectively. x and BV y may represent the horizontal and vertical components of the BV 1708, respectively. x and BVP y may represent the horizontal and vertical components of BVP 1712, respectively. The horizontal x-axis and vertical y-axis are shown in the lower right corner of current picture 1702 for reference purposes. As shown in FIG. 17A, the x-axis (e.g., x-axis values) may increase from left to right, and the y-axis (e.g., y-axis values) may increase from top to bottom.
[0119] The encoder may transmit / signal within / via the bitstream the prediction error, an indication of the selected BVP 1712, and another component of the BVD 1714 determined based on equations (17) and (18). The indication of the BVP 1712 may be an index indicating the BVP 1712 in a list of candidate BVPs. A decoder (e.g., the decoder 300 shown in FIG. 3) may decode the BV 1708 by adding the corresponding component of the BVD 1714 to the corresponding component of the BVP 1712. The decoder may decode the current block 1700 by determining and / or generating a reference block 1710. The decoder may use the decoded BV to determine and / or generate the reference block 1710. The reference block 1710 may correspond to / form (e.g., take into account) a prediction of the current block 1700. The decoder may decode the current block based on combining the prediction with the prediction error received within / via the bitstream.
[0120] The BVP 1712 may indicate a relative displacement from the position of the current block 1700 to a position (e.g., sample position) 1716. The position of a block (e.g., current block 1700, reference block 1710) may be determined by / corresponding to the position of the top-left sample of the block (e.g., current block 1700, reference block 1710). The position of a block may be determined by / corresponding to any other sample of the block. The position 1716 may be an integer sample position or a fractional sample position between two integer sample positions. The sample position 1716 indicated by the BVP 1712 may be outside the IBC reference region 1706 (as shown in FIG. 17A ) because the BVP 1712 may be derived from spatially neighboring blocks of the current block 1700 and / or previously coded BVs. The BV 1708 may be constrained to indicate a relative displacement from the current block 1700 to the position of the reference block 1710 within the IBC reference region 1706. The position of the reference block 1710 may be determined by the position of the top-left sample (or any other sample) of the reference block 1710. The BVP 1712 may not accurately predict the BV 1708 because the BVP 1712 may indicate a relative displacement from the current block 1700 to a sample position 1716 that may be outside the IBC reference region 1706, while the BV 1708 may indicate a relative displacement from the current block 1700 to a position of the reference block 1710 that is within the IBC reference region 1706. Inaccurate prediction of the BV 1708 may require higher signaling overhead to indicate the BVD 1714. Various embodiments herein describe adjusting the BVP (e.g., the BVP 1716) to more accurately predict the BV.
[0121] BVP 1712 being a more accurate prediction of BV 1708 may correspond to the BVP indicating a location (e.g., location 1716) that is within the IBC reference region. The encoder and / or decoder may determine whether BVP 1712 points / indicates a location (e.g., location 1716) that is within the IBC reference region. BVP 1712 pointing / indicating a location that is within the IBC reference region may result in the BVP being a more accurate prediction of BV 1708. BVP 1712 being a more accurate prediction of BV 1708 may result in reduced signaling overhead for signaling / transmitting BVD 1714. The encoder and / or decoder may replace / adjust BVP 1712 with an adjusted BVP, for example, based on determining that BVP 1712 points / indicates a location that is not within the IBC reference region.
[0122] Determining whether the BVP 1712 points / indicates a position (e.g., position 1716) within the IBC reference region may include determining whether the position is away from the current block 1700 by at least one or more offsets in one or more predetermined directions. The one or more predetermined directions may be based on a scan type used for video coding. For example, in the case of video coding using z-scanning (e.g., scanning from left to right or top to bottom), the one or more directions may include a first direction pointing above the current block 1700 and a second direction pointing to the left of the current block 1700. Determining whether the BVP 1712 points / indicates a position within the IBC reference region may include determining whether the position is away from the left of the current block 1700 by at least a first offset distance and / or away from the current block by at least a second offset distance. The first offset distance and the second offset distance may be defined as a function of the dimensions of the current block 1700 (e.g., the width and / or height of the current block 1700).
[0123] 17A , the encoder and / or decoder may determine that the BVP 1712 indicates a position 1716 that is neither to the left of the current block 1700 by at least a first offset distance (e.g., a distance at least equal to the width (cbWidth) of the current block 1700, or by some additional or other offset amount) nor above the current block 1700 by at least a second offset distance (e.g., a distance at least equal to the height (cbHeight) of the current block 1700, or by some additional or other offset amount). The BVP 1712 may point to a position in a non-shaded region of the current picture 1702 (e.g., as shown in FIG. 17A ), which may be outside the IBC reference region 1706 of the current block 1700, for example, if the position 1716 is neither to the left of the current block 1700 by at least the first offset distance nor above the current block 1700 by at least the second offset distance.
[0124] As mentioned above, the IBC reference region 1706 may not include an L-shaped region to prevent the BV from pointing to a reference block that may overlap (at least partially) with the current block 1700 or an unreconstructed region of the current picture 1702. The encoder or decoder may determine whether the BVP 1712 points / indicates a location 1716 that is within or outside the IBC reference region 1706. The encoder or decoder may determine whether the horizontal component BVP of the BVP 1712 points / indicates a location 1716 that is within or outside the IBC reference region 1706. x and the first offset distance is greater than zero, and the vertical component of BVP1712 y and the second offset distance is greater than zero, it may be determined that the BVP 1712 points to / indicates a location 1716 that is neither to the left of the current block 1700 by at least the first offset distance (e.g., cbWidth, or any other distance) nor above the current block 1700 by at least the second offset distance (e.g., cbHeight, or any other distance). xis greater than -cbWidth, the vertical component of BVP1712 y may equivalently be described as determining that -cbWidth is greater than -cbHeight (eg, where the first offset distance is cbWidth and the second offset distance is cbHeight).
[0125] The encoder and / or decoder may replace the BVP 1712 with an adjusted BVP, for example, based on determining the position 1716 indicated by the BVP 1712. The encoder and / or decoder may replace the BVP 1712 with an adjusted BVP, for example, based on determining that the position 1716 is not within (e.g., outside) the IBC reference region 1706. The encoder and / or decoder may replace the BVP 1712 with an adjusted BVP, for example, based on determining that the position 1716 is neither to the left of the current block 1700 by at least a first offset distance (e.g., a distance equal to the width (cbWidth) of the current block 1700) nor above the current block 1700 by at least a second offset distance (e.g., a distance equal to the height (cbHeight) of the current block 1700). The encoder and / or decoder may replace the BVP 1712 with an adjusted BVP that satisfies one or more of the above conditions.
[0126] The encoder and / or decoder may replace the BVP 1712 with an adjusted BVP that may indicate a displacement from the position of the current block 1700 to a position within the IBC reference region 1706. For example, the encoder and / or decoder may replace the BVP 1712 with an adjusted BVP that indicates a displacement from the position of the current block 1700 to a position within the IBC reference region 1706. For example, the encoder and / or decoder may replace the BVP 1712 with an adjusted BVP that indicates a displacement from the position of the current block 1700 to a position within the IBC reference region 1706. y The encoder and / or decoder may replace BVP1712 with a scaled BVP having a vertical component equal to the horizontal component BVP xThe encoder and / or decoder may replace BVP 1712 with an adjusted BVP that includes a horizontal component equal to -cbWidth and a vertical component equal to -cbHeight. The encoder and / or decoder may replace BVP 1712 with an adjusted BVP that includes a horizontal component equal to -cbWidth and a vertical component equal to -cbHeight. The encoder and / or decoder may replace BVP 1712 with an adjusted BVP that is to the left of current block 1700 by at least a first offset distance (e.g., cbWidth) and / or above current block 1700 by at least a second offset distance (e.g., cbHeight). The encoder and / or decoder may replace BVP 1712 with an adjusted BVP that indicates a displacement from the position of the current block 1700 to a position within the IBC reference region 1706, for example, based on determining that position 1716 is neither to the left of the current block 1700 by at least a first offset distance (e.g., a distance equal to the width (cbWidth) of the current block 1700) nor above the current block 1700 by at least a second offset distance (e.g., a distance equal to the height (cbHeight) of the current block 1700). The encoder and / or decoder may replace BVP 1712 with an adjusted BVP that points to / indicates a position within the IBC reference region 1706 relative to the position of the current block 1700. The encoder and / or decoder may replace BVP 1712 with an adjusted BVP based on determining one or more adjusted BVP candidates.
[0127] FIG. 17D shows an example method for adjusting a BVP. The method 1750 may include determining an adjusted BVP candidate. The example method 1750 may be implemented in an encoder and / or decoder. At step 1752, the encoder and / or decoder may determine multiple adjusted BVP candidates. The multiple adjusted BVP candidates may include a horizontal component equal to the negative value of the first offset distance (e.g., −cbWidth) and a vertical component BVP of BVP 1712. yThe horizontal component of the first adjusted BVP candidate, BVP1712, has a vertical component equal to x and a vertical component equal to the negative of the second offset distance (e.g., −cbHeight), and a third adjusted BVP candidate having a horizontal component equal to the negative of the first offset distance (e.g., −cbWidth) and a vertical component equal to the negative of the second offset distance (e.g., −cbHeight). The multiple adjusted BVP candidates may include at least two of: a first adjusted BVP candidate having a horizontal component based on the width (cbWidth) of the current block 1700, a second adjusted BVP candidate having a vertical component based on the height (cbHeight) of the current block 1700, and a third adjusted BVP candidate having a horizontal component based on the width (cbWidth) of the current block 1700 and a vertical component equal to the height (cbHeight) of the current block.
[0128] In step 1754, the encoder and / or decoder may select an adjusted BVP from the multiple adjusted BVP candidates. For example, the encoder and / or decoder may determine / select one of the adjusted BVP candidates as the adjusted BVP to replace BVP 1712. In step 1758, the encoder and / or decoder may add the adjusted BVP to a list of candidate BVPs. The encoder and / or decoder may select one of the adjusted BVP candidates (e.g., in the list of candidate BVPs) based on, for example, the BVD determined / calculated for each adjusted BVP candidate. The BVP may be determined / calculated as the difference between BV 1708 and the adjusted BVP candidate for each adjusted BVP candidate. The encoder and / or decoder may select the adjusted BVP candidate that results in the smallest BVD among the BVDs calculated for the adjusted BVP candidates as the adjusted BVP.
[0129] The encoder and / or decoder may select the adjusted BVP candidate based on a predetermined order. The predetermined order may include the first, second, and third adjusted BVP candidates. The encoder and / or decoder may select one of the adjusted BVP candidates from only the valid adjusted BVP candidates. A valid BVP candidate may be a candidate that indicates a displacement from the position of the current block 1700 to a position within the IBC reference region 1706. The encoder may send / signal an indication of the selected adjusted BVP candidate to the decoder in / via the bitstream. The decoder may receive the indication from the bitstream and replace the BVP with the selected adjusted BVP candidate. The adjusted BVP may be used to predict (e.g., in the encoder) and / or determine (e.g., in the decoder) the BV 1708 in the same manner as described for the BVP 1712.
[0130] FIG. 18 shows an example of an IBC with an adjusted BVP. An encoder and / or decoder may replace BVP 1712 (e.g., as described with reference to FIG. 17A) with adjusted BVP 1812. The replacement of BVP 1712 with adjusted BVP 1812 may be based on one or more considerations, as described with reference to FIGS. 17A-17D. The adjusted BVP may be determined / selected from one or more adjusted BVP candidates. The adjusted BVP candidate may be determined based on one or more considerations, as described with reference to FIGS. 17A-17D. For example, adjusted BVP 1812 may have a horizontal component equal to -cbWidth and a vertical component equal to -cbWidth of BVP 1712. y As shown in FIG. 18, the adjusted BVP 1812 may include a vertical component equal to the BVD 1714 of FIG. 17A. x The horizontal component of BVD1814 is higher than that of BVD x Reducing the length of may provide a better prediction of BV1708.
[0131] FIG. 19 shows an example of an IBC with an adjusted BVP. An encoder and / or decoder may replace BVP 1712 with adjusted BVP 1912. The replacement of BVP 1712 with adjusted BVP 1912 may be based on one or more considerations, as described with reference to FIGS. 17A-D. The adjusted BVP may be determined / selected from one or more adjusted BVP candidates. The adjusted BVP candidate may be determined based on one or more considerations, as described with reference to FIGS. 17A-D. The adjusted BVP 1912 is a horizontal component BVP of BVP 1712. x 19, the adjusted BVP 1912 may have a horizontal component equal to β and a vertical component equal to −cbHeight. y Increasing the length of the nucleotide sequence may provide a worse prediction of BV1708.
[0132] FIG. 20 illustrates an example of an IBC with an adjusted BVP. An encoder and / or decoder may replace BVP 1712 with adjusted BVP 2012. The replacement of BVP 1712 with adjusted BVP 1912 may be based on one or more considerations, as described with reference to FIGS. 17A-D. The adjusted BVP may be determined / selected from one or more adjusted BVP candidates. The adjusted BVP candidate may be determined based on one or more considerations, as described with reference to FIGS. 17A-D. The adjusted BVP 2012 may include a horizontal component equal to -cbWidth and a vertical component equal to -cbHeight. As shown in FIG. 20, adjusted BVP 2012 may provide a better prediction of BV 1708 at one point by reducing the length of the horizontal component of BVD 2014 compared to the horizontal component of BVD 1714 of FIG. 17, but may not be based on the vertical component of BVD 1714 of FIG. 17. y Compared to the vertical component of BVD2014, yIncreasing the length of may provide a worse prediction of BV1708 in other respects.
[0133] An encoder and / or decoder may use the adjusted BVP in a manner similar to the BVP described above. For example, the encoder may determine a BVD based on an adjusted BVP according to Equations 17 and 18. The encoder may transmit / signal a prediction error, an indication of the adjusted BVP, and another component of the BVD within / via the bitstream. The indication of the adjusted BVP may be an index indicating the adjusted BVP in a list of candidate BVPs. A decoder (e.g., decoder 300 shown in FIG. 3) may decode the BV by adding a corresponding component of the BVD to a corresponding component of the adjusted BVP. The decoder may decode the current block by determining and / or generating a reference block. The decoder may use the decoded BV to determine and / or generate a reference block. The reference block may correspond to / form (e.g., take into account) a prediction of the current block. The decoder may decode the current block based on combining the prediction with a prediction error received within / via the bitstream.
[0134] 17A-17D, 18, 19, and 20, the IBC reference region 1706 is merely exemplary, and the IBC reference region may be different from the IBC reference region 1706. With reference to FIGS. 17A-17D, 18, 19, and 20, the methods, apparatuses, and systems described herein may be used / applied to an IBC reference region different from the IBC reference region 1706. For example, the IBC reference region 1706 may be replaced by an IBC reference region determined based on a different set of IBC reference region constraints. For example, in addition to being constrained to a reconstructed portion of the current picture 1702 and / or one or more WPP or tile divisions, the IBC reference region 1706 may be further constrained to include the number / quantity of decoded or reconstructed samples that can be stored in a limited memory size (e.g., an IBC reference sample memory). The size of the IBC reference sample memory may be limited based on its on-chip implementation with an encoder or decoder. The IBC reference area may be increased in size by using a larger sized IBC reference sample memory off-chip from the encoder or decoder. Using off-chip memory may require higher memory bandwidth requirements and increased latency in writing and / or reading samples (e.g., in the IBC reference area 1706) to and / or from the IBC reference sample memory.
[0135] An IBC reference region (e.g., IBC reference region 1706) may be constrained to the reconstructed portion of the current CTU and / or one or more reconstructed CTUs to the left of the current CTU. The one or more reconstructed CTUs to the left of the current CTU may not include a leftmost portion of the one or more reconstructed CTUs that is collocated with either the reconstructed portion of the current CTU or the virtual pipeline data unit (VPDU) in which the current block to be coded is located. Blocks of samples in different CTUs may be collocated based on having the same size and / or CTU offset. The CTU offset of a block may be the offset of the block's upper left corner relative to the upper left corner of the CTU in which the block is located.
[0136] The IBC reference area may not include the leftmost portion of a more reconstructed CTU that is collocated with the reconstructed portion of the current CTU. For example, the IBC reference area may not include the leftmost portion of a more reconstructed CTU that is collocated with the reconstructed portion of the current CTU. This is because the IBC reference sample memory may be implemented in a manner similar to a circular buffer. For example, the IBC reference sample memory may store reconstructed reference samples corresponding to one or more CTUs. For example, the reconstructed reference samples of the current CTU may replace the reconstructed reference samples stored in the IBC reference sample memory of the CTU located furthest to the left of the current CTU (e.g., in a picture or frame) when the IBC reference sample memory is filled. The samples of the CTU stored in the IBC reference sample memory located furthest to the left of the current CTU in a picture or frame may correspond to the oldest data in the IBC reference sample memory. As described herein, updating the samples in the IBC reference sample memory may allow at least a portion of the reconstructed reference samples from the leftmost CTU to remain stored in the IBC reference sample memory when processing the current CTU. The remaining reference samples of the leftmost CTU stored in the IBC reference sample memory may be used to predict the current block of the current CTU.
[0137] CTUs may or may not be processed at once. For example, in a typical hardware implementation of an encoder and / or decoder, CTUs may not be processed at once. CTUs may be divided into VPDUs for processing by pipeline stages. A VPDU may include a 4x4 sample region, a 16x16 sample region, a 32x32 sample region, a 64x64 sample region, a 128x128 sample region, or any other sample region size. The size of a VPDU may be determined based on the smaller of the maximum VPDU size (e.g., a 64x64 sample region) and the size (e.g., width or height) of the current CTU. The leftmost portion of one or more reconstructed CTUs collocated with the VPDU in which the block to be coded is located may further be excluded from the IBC reference region. Excluding this leftmost portion of one or more reconstructed CTUs from the IBC reference region may enable a portion of the IBC reference sample memory (e.g., used to store reference samples reconstructed from this portion) to store only samples within the region of the current CTU that corresponds to the VPDU. Storing only samples within the region of the current CTU that corresponds to the VPDU may reduce and / or avoid certain complexities in encoder and / or decoder design.
[0138] The amount of reconstructed CTUs to the left of the current CTU included in the IBC reference area may be determined based on the amount of reconstructed reference samples that the IBC reference sample memory can store and / or the size of the CTUs in the current picture. The amount of reconstructed CTUs to the left of the current CTU included in the IBC reference area may be determined based on dividing the amount of reconstructed reference samples that the IBC reference sample memory can store by the size of the CTUs in the current picture. For example, 128 x 128 reconstructed reference samples may be stored for the IBC reference area, and for an IBC reference sample memory with a CTU size of 128 x 128 samples, the amount of reconstructed CTUs to the left of the current CTU included in the IBC reference area may be equal to (128 x 128) / (128 x 128), or 1 CTU. As another example, for a memory that may store 128x128 reconstructed reference samples for an IBC reference area and has a CTU size of 64x64 samples, the amount of CTUs reconstructed to the left of the current CTU included in the IBC reference area may be equal to (128x128) / (64x64), or 4 CTUs.
[0139] FIG. 21A shows an example IBC reference region. The IBC reference region 2100 may be determined based on the IBC reference sample memory size and the CTU size. The IBC reference sample memory size may be equal to the CTU size. The IBC reference sample memory size may be equal to 128×128 samples (or any other number of samples). The CTU size may be equal to 128×128 samples (or any other number of samples). The amount of reconstructed CTUs to the left of the current CTU 2104 to be included in the IBC reference region 2100 may be equal to (128×128) / (128×128), or 1 CTU. The IBC reference region 2100 may be a portion of the reconstruction region 2110. The samples in the IBC reference region 2100 may be a subset of the samples in the reconstruction region 2110. The samples of the current block 2102 to be coded may be a subset of the samples in the VPDU 2108.
[0140] FIG. 21A shows a current block 2102 in a current CTU 2104. The current block 2102 may be the first block encoded in the current CTU 2104 and may be encoded using IBC mode. As described with reference to FIG. 17, a block may be encoded using IBC mode by determining the best-matching reference block in an IBC reference region 2100. The IBC reference region 2100 may be constrained to a single reconstructed CTU 2106 to the left of the current CTU 2104 that does not include the reconstructed portion of the current CTU 2104 and the portion of the reconstructed CTU 2106 that is co-located with either the reconstructed portion of the current CTU 2104 or the VPDU 2108 in which the current block 2102 is located. A CTU may be divided into multiple VPDUs. For example, FIG. 21A may be divided into four VPDUs of size 64×64 samples. The IBC reference region 2100 of the current block 2102 may include a reconstructed region 2110 (shown with hatching), excluding the 64x64 region of the reconstructed CTU 2106 that is collocated with the VPDU 2108. The collocated region is marked with an X in FIG. 21A . The IBC reference region 2100 may include a different amount of CTUs to the left of the current CTU 2102. The amount of CTUs in the IBC reference region 2100 to the left of the current CTU 2102 may vary for different CTU sizes. For example, for a 64x64 CTU size, the IBC reference region 2100 may include 4 CTUs to the left of the current CTU 2102, based on the amount of reconstructed reference samples that the IBC reference sample memory can store divided by the size of the CTU in the current picture. For ease of illustration, FIG. 21A does not show the L-shaped region surrounding the current block as described with reference to FIG. 17 . Such L-shaped regions may be excluded from the IBC reference region 2100.
[0141] FIG. 21B shows an example IBC reference region. FIG. 21B shows an IBC reference region 2118 for a block coded after the current CTU 2104. The later-coded block may be the current block 2112. The current block 2112 may be coded using an IBC mode (e.g., as described above with reference to FIGs. 17A-D). The current block 2112 may be coded by determining the best-matching reference block within the IBC reference region 2118. The IBC reference region 2118 for the current block 2112 may be constrained to a reconstructed portion of the current CTU 2104 and a reconstructed CTU 2106 that does not include a portion of the reconstructed CTU 2106 that is co-located with either the reconstructed portion of the current CTU 2104 or the VPDU 2114 in which the current block 2112 is located. The current CTU 2104 may be divided into four VPDUs of size 64x64 samples (e.g., as described with reference to FIG. 21A). The IBC reference region 2118 for the current block 2112 may consist of the reconstructed region 2116 (shown hatched) excluding the portions of the CTU 2106 that are collocated with either the reconstructed portion of the current CTU 2104 and / or the VPDU 2114. In FIG. 21B, the collocated regions are each indicated by an X. For ease of illustration, FIG. 21B does not show an L-shaped region surrounding the current block as described with respect to FIG. 17A. Such an L-shaped region may be excluded from the IBC reference region 2118.
[0142] 22 shows an example method for replacing a BVP with an adjusted BVP. Method 2200 may be performed by a device in a video encoding or decoding system. For example, the device may be an encoder and / or a decoder (e.g., encoder 200 shown in FIG. 2 and / or decoder 300 shown in FIG. 3).
[0143] In step 2202, the device may determine that the sum of the horizontal component of the BVP and the width of the current block (e.g., cbWidth) is greater than zero. The device may further determine that the sum of the vertical component of the BVP and the height of the current block (e.g., cbHeight) is greater than zero.
[0144] In step 2204, the BVP may be replaced with an adjusted BVP based on the determination made in step 2202. The horizontal component of the adjusted BVP may be equal to -cbWidth and the vertical component of the adjusted BVP may be equal to the vertical component of the BVP. The horizontal component of the adjusted BVP may be equal to the horizontal component of the BVP and the vertical component of the adjusted BVP may be equal to -cbHeight. The horizontal component of the adjusted BVP may be equal to -cbWidth and the vertical component of the adjusted BVP may be equal to -cbHeight.
[0145] Replacing the BVP with an adjusted BVP may include determining multiple adjusted BVP candidates, including at least two of: a first adjusted BVP candidate having a horizontal component equal to −cbWidth and a vertical component equal to a vertical component BVPy of the BVP; a second adjusted BVP candidate having a horizontal component equal to a horizontal component BVPx of the BVP and a vertical component equal to −cbHeight; and a third adjusted BVP candidate having a horizontal component equal to −cbWidth and a vertical component equal to −cbHeight.
[0146] Replacing a BVP with an adjusted BVP may include selecting one of the adjusted BVP candidates as the adjusted BVP. The selection may be among valid adjusted BVP candidates. The adjusted BVP may be selected from the adjusted BVP candidates based on the BVD determined / calculated for each of the adjusted BVP candidates. The adjusted BVP may be associated with the smallest BVD among the BVDs calculated for the adjusted BVP candidates.
[0147] Method 2200 may include transmitting / signaling an indication of the adjusted BVP within / via a bitstream. Method 2200 may include receiving an indication of the adjusted BVP within / via a bitstream. At step 2206, the adjusted BVP may be used to determine or predict a BV for the current block (e.g., at an encoder or decoder). For example, the encoder may determine a BVD based on the adjusted BVP and BV (e.g., using Equations 17 and 18). The decoder may decode the BV using received indications of the adjusted BVP and BVD.
[0148] An encoder (e.g., the encoder 200 shown in FIG. 2) may encode BVs using a merge mode. An encoder using the merge mode may reuse the same BVs as neighboring blocks of the current block or other blocks for IBC prediction of the current block. The BVD corresponding to the current block does not need to be transmitted / signaled, for example, because the same BVs of at least the neighboring blocks or other blocks may be used. The signaling overhead for signaling the BVs of the current block may be reduced. In a manner similar to BV prediction and differential coding (or AMVP for IBC), both the encoder and decoder may generate a candidate list of BVPs from neighboring blocks or other blocks of the current block for the merge mode. The encoder may decide to use (or inherit) one BV from the BVPs in the candidate list to predict the BV information of the current block to be encoded. The encoder may transmit / signal an indication of the BVP determined from the candidate list within / via the bitstream. For example, the encoder may transmit / signal an indicator / index indicating the determined BVP in the list of candidate BVPs. The decoder may generate, determine, and / or construct a list of candidate BVPs in the same manner as an encoder for merge mode. The BVP may be indicated to the decoder in / via the bitstream as an index / indicator of the BVP in the list of candidate BVPs. The decoder may decode the current block by determining and / or generating a reference block using the determined BV associated with the BVP. The reference block may correspond to / form (e.g., take into account) a prediction of the current block. The decoder may decode the current block by combining the prediction with the prediction error. The list of candidate BVPs for merge mode (e.g., HEVC, VVC, and / or other coding standards / formats / protocols) may include up to four (or any other number) spatial merge candidates derived from the five (or any other number) spatial neighboring blocks used in AMVP of IBC and / or one or more additional history-based BVPs.
[0149] 23A, 23B, and 23C show examples of candidate BVP adjustments. An encoder (e.g., the encoder 200 shown in FIG. 2) and / or a decoder (e.g., the decoder 300 of FIG. 3) may use ICB to encode a current block 2300 in a CTU 2302. The encoder and / or decoder may use IBC to encode the current block 2300 as described herein.
[0150] An encoder using IBC may search for a reference block in the current picture that is the same as that of the current block. Only a portion of the current picture may be available for searching for the reference block. For example, only a portion of the current picture that was decoded before encoding the current block may be available for searching for the reference block. The portion of the current picture available for searching for the reference block may be an IBC reference region. Searching only a portion of the current picture that was decoded before encoding the current block may ensure that the encoding and decoding systems may produce identical results, but may limit the IBC reference region.
[0151] Blocks may be scanned in a particular order. For example, blocks may be scanned from left to right and top to bottom using z-scan (e.g., HEVC, VVC, any other coding standard / format / protocol) to form a sequence order for encoding / decoding. Based on the z-scan, CTU 2302 and blocks (not shown in FIG. 23A ) to the left and / or above current block 2300 within CTU 2302 may form an exemplary IBC reference region 2304 for determining reference blocks for predicting current block 2300. Different sequence orders or picture partitioning methods for encoding / decoding may be used in some video encoders and / or decoders. Using a different sequence order or picture partitioning method may change the IBC reference region 2304 accordingly.
[0152] One or more additional IBC reference region constraints may be placed on the IBC reference region 2304, for example, in addition to the encoding / decoding sequence order. For example, the IBC reference region 2304 may be constrained based on slice boundaries, tile boundaries, WPPs, and / or limited memory for storing reference samples for predicting the current block 2300. Tiles may be used as part of a picture partitioning process to flexibly subdivide a picture into rectangular regions of CTUs such that coding dependencies between CTUs of different tiles are not allowed. WPPs may also be used to divide a picture into CTU rows as part of the picture partitioning process. The division into CTU rows may be such that dependencies between CTUs of different partitions are not allowed. The use of tiles and / or WPPs may enable parallel processing of picture partitioning. One or more CTUs (not shown in FIG. 23A ) to the left of and / or above the CTU 2302 may not be part of the IBC reference region 2304, for example, due to limited memory for storing reference samples and / or due to one of the parallel processing approaches.
[0153] The IBC reference region 2304 may be constrained so that any BV (e.g., determined to encode the current block 2300 based on IBC) indicates a displacement from the position of the current block 2300 to the position of a reference block that does not overlap (or does not partially overlap) with the current block 2300. The constraint on the BV may result in an inverted L-shaped gap between the current block 2300 and the reference region 2304 (or a gap between the current block 1700 and the reference region 1706), as shown in FIG. 23A. The L-shaped gap may have a width of cbWidth-1 (where cbWidth is the width of the current block 2300) to the left of the current block 2300 and / or a length of cbHeight-1 (where cbHeight is the height of the current block 2300) above the current block 2300. The location of a block herein may refer to the location of the top-left sample of the block. The location of a block may be determined by the location of another sample within the block in other example scenarios. The location of a sample in a picture may be indicated by a horizontal sample number (given by the variable x) and a vertical sample number (given by the variable y). The horizontal and vertical sample numbers may be relative to the origin of the picture coordinate system in the upper left corner of the picture (e.g., assigned (x, y) = (0, 0)). The horizontal and vertical sample numbers may be relative to the top left sample of the block (e.g., CTU) in which the sample is located. The positive direction may be perpendicular to the horizontal x direction. A larger value of x may mean the sample position is further to the right in the positive horizontal direction. The positive direction may be downward in the vertical y direction. A larger value of y may mean the sample position may be further down in the positive vertical direction.
[0154] The encoder may apply / use a block matching technique to determine the BV 2306. The BV 2306 may indicate the displacement from the position of the current block 2300 to the position of the reference block 2308 (e.g., according to intrablock compensation prediction). The reference block 2308 may be a block within the IBC reference region 2304 that best matches the current block 2300. The BV 2306 may be constrained by the IBC reference region 2304 (e.g., as described herein) to indicate the displacement from the position of the current block 2300 to the position of the reference block within the IBC reference region 2304. The encoder may determine the best matching reference block from blocks having positions within the IBC reference region 2304 that may have been tested during the search process. The encoder may determine that the reference block is the best matching reference block based on one or more cost criteria, such as a rate-distortion criterion (e.g., a Lagrangian rate-distortion cost). The one or more cost criteria may be based, for example, on differences (e.g., differences determined based on SSD, SAD, SATD, and / or a hash function) between predicted samples of a reference block and original samples of the current block. The reference block may include decoded (and / or reconstructed) samples of the current picture before being processed by in-loop filtering operations (e.g., deblocking and / or SAW filtering).
[0155] The encoder may determine a difference (e.g., corresponding sample-by-sample difference) between the current block 2300 and the reference block 2308, for example, based on determining the reference block 2308 using IBC. The difference may be referred to as a prediction error and / or a residual. The encoder may then store and / or transmit / signal the prediction error and / or associated prediction information in / via a bitstream for decoding by a decoder.
[0156] The encoder and / or decoder may determine a list of candidate BVPs for predictively encoding the BV 2306. The encoder and / or decoder may construct / determine the list of candidate BVPs from candidate BVPs derived from multiple sources. The multiple sources may include spatially neighboring blocks of the current block 2300, temporally co-located blocks of the current block 2300, and / or IBC information of history-based BVs. The encoder and / or decoder may construct / determine the list of candidate BVPs for predictively encoding the BV based on using AMVP for IBC or merge mode.
[0157] A candidate BVP (e.g., from a source such as a BV of a spatially neighboring block and / or a history-based BV) may indicate a displacement from the position of the current block 2300 to the position of a reference block, such as BV 2306, that may not be within the IBC reference region 2304. The candidate BVP may not provide an accurate prediction of BV 2306, for example, because the candidate BVP may indicate a displacement from the position of the current block 2300 to the position of a reference block that may not be within the IBC reference region 2304 (e.g., unlike BV 2306). The amount of bits required to transmit / transmit the BVD between BV 2306 and the candidate BVP may be high due to the inaccurate prediction. Furthermore, the BV can only be predicted by the BVP (e.g., in merge mode). A candidate BVP that points outside the IBC reference region cannot be used to predict the BV because all / some of the samples of the reference block may not be decoded.
[0158] 23A shows an example of a candidate BVP. A candidate BVP 2310 (or simply BVP 2310) may indicate a displacement from the position of a current block 2300 to the position of a reference block 2312. The reference block 2312 may not be within the IBC reference region 2304.
[0159] The encoder and / or decoder may determine that the BVP 2310 indicates a displacement from the position of the current block 2300 to a position of the reference block 2312 that is neither to the left of the current block 2300 by an amount at least equal to the width (cbWidth) of the current block 2300 nor above the current block 2300 by an amount at least equal to the height (cbHeight) of the current block 2300. For example, the BVP 2310 may indicate a displacement from the position of the current block 2300 to a position of the reference block 2312 within an L-shaped region surrounding the left and top of the current block 2300, or within a region to the right and below the L-shaped region, both / either of which may be outside the IBC reference region 2304. The non-inclusion of an L-shaped region (e.g., surrounding the left and top portions of the current block 2300, with the left portion having a width equal to (cbWidth-1) and the top portion having a height equal to (cbHeight-1)) in the IBC reference region 1704 is to prevent the BV 2306 from pointing to a reference block that overlaps (at least partially) with the current block 2300.
[0160] The encoder or decoder is the horizontal component of the BVP2310. x and the sum of cbWidth is greater than zero, and the vertical component of BVP2310 BVP y and cbHeight is greater than zero, indicating a displacement from the position of the current block 2300 to a position of the reference block 2312 that is neither to the left of the current block 2300 by an amount at least equal to cbWidth nor above the current block 2300 by an amount at least equal to cbHeight. x is greater than -cbWidth, and the vertical component of BVP2310 y The first and second determinations can be equivalently described as determining that the horizontal component of BVP2310 is greater than -cbHeight. x The magnitude of cbWidth is less than the vertical component of BVP2310.y may equivalently be stated as determining that the size of is less than cbHeight.
[0161] The encoder and / or decoder may replace the BVP 2310 with an adjusted BVP based on, for example, determining that the position of the reference block 2312 is neither to the left of the current block 2300 by an amount at least equal to cbWidth nor above the current block 2300 by an amount at least equal to cbHeight. The adjusted BVP may indicate a displacement from the position of the current block 2300 to the position of the reference block within the IBC reference region 2304. For example, the encoder and / or decoder may determine and / or calculate a horizontal distance 2314 from the position of the reference block 2312 to the nearest vertical boundary of the IBC reference region 2304 of the current block 2300 and a vertical distance 2316 from the position of the reference block 2312 to the nearest horizontal boundary of the IBC reference region 2304 of the current block 2300. Horizontal distance 2314 is the horizontal component of cbWidth and BVP 2310. x The vertical distance 2316 may be determined based on the sum of cbHeight and the vertical component BVP of BVP 2310. y The encoder and / or decoder may replace the BVP 2310 with an adjusted BVP based on a comparison of the horizontal distance 2314 and the vertical distance 2316. The adjusted BVP may be based on one or more of whether the horizontal distance 2314 is greater than the vertical distance 2316, whether the horizontal distance 2314 is less than the vertical distance 2316, and / or whether the horizontal distance 2314 is equal to the vertical distance 2316. The BVP adjustment may be in such a manner that a change in the position of the reference block 2312 may be minimized.
[0162] The encoder and / or decoder may replace BVP 2310 with adjusted BVP 2318, for example, based on horizontal distance 1714 being less than vertical distance 1716. The encoder and / or decoder may replace BVP 2310 with adjusted BVP 2318, for example, based on horizontal distance 1714 being less than vertical distance 1716, including a horizontal component equal to the negative of the width of current block 2300 (-cbWidth) and a vertical component equal to the vertical component of BVP 2310. FIG. 23A shows an example in which horizontal distance 2314 is less than vertical distance 2316. The adjusted BVP 2318 in FIG. 23A may include a horizontal component equal to -cbWidth and a vertical component equal to the vertical component of BVP 2310, for example, based on horizontal distance 2314 being less than vertical distance 2316.
[0163] The encoder and / or decoder may replace BVP 2310 with adjusted BVP 2318, for example, based on horizontal distance 1714 being greater than vertical distance 1716. The encoder and / or decoder may replace BVP 2310 with adjusted BVP 2318 that includes a horizontal component equal to the horizontal component of BVP 2310 and a vertical component equal to the negative of the height of the current block (−cbHeight).
[0164] Figure 23B illustrates an example where horizontal distance 2314 is greater than vertical distance 2316. Adjusted BVP 2318 in Figure 23B includes a horizontal component equal to the horizontal component of BVP 2310 and a vertical component equal to -cbHeight, for example, based on horizontal distance 2314 being greater than vertical distance 2316.
[0165] The encoder and / or decoder may replace BVP 2310 with adjusted BVP 2318, for example, based on horizontal distance 2314 equaling vertical distance 2316. The encoder and / or decoder may replace BVP 2310 with adjusted BVP 2318 that includes a horizontal component equal to −cbWidth and a vertical component equal to −cbHeight.
[0166] FIG. 23C illustrates an example where horizontal distance 2314 is equal to vertical distance 2316. In FIG. 23C, an adjusted BVP 2318 is, for example, based on horizontal distance 2314 being equal to vertical distance 2316: Contains a horizontal component equal to -cbWidth and a vertical component equal to -cbHeight.
[0167] FIG. 24 shows an example of a distance check. The distance check may be performed before adjusting the BVP. The BVP may be BVP 2310 as described with reference to FIGS. 23A, 23B, and / or 23C. As described with reference to FIGS. 23A, 23B, and / or 23C, one or both of the horizontal and vertical components of BVP 2310 may be adjusted to generate adjusted BVP 2324. The horizontal component of BVP 2310 may be adjusted to −cbWidth, and / or the vertical component of BVP 2310 may be adjusted to −cbHeight. Before adjusting either or both of the horizontal and vertical components of the BVP 2310, for example, the encoder and / or decoder may determine a horizontal distance 2402 from the position of the current block 2300 to a vertical boundary (e.g., the left-most vertical boundary) of the IBC reference region 2304 and / or may determine a vertical distance 2404 from the position of the current block 2300 to a horizontal boundary (e.g., the top-most horizontal boundary) of the IBC reference region 2304. The horizontal distance 2402 may be greater than or equal to cbWidth, for example, to adjust the horizontal component of the BVP 2310 to −cbWidth. The vertical distance 2404 may be greater than or equal to cbHeight, for example, to adjust the vertical component of the BVP 2310 to −cbHeight.
[0168] Replacing BVP 2310 based on horizontal distance 2314 and vertical distance 2316 may be performed as described above with reference to Figures 23A, 23B, and / or 23C, for example, based on both horizontal distance 2402 being greater than or equal to cbWidth and vertical distance 2404 being greater than or equal to -cbHeight. BVP 2310 may be replaced by adjusted BVP 2324, for example, based on both horizontal distance 2402 being greater than or equal to cbWidth and vertical distance 2404 being greater than or equal to -cbHeight. BVP2310 may be replaced by adjusted BVP2324, which includes a horizontal component equal to -cbWidth and a vertical component equal to the vertical component of BVP2310 based on horizontal distance 2314 being less than vertical distance 2316, a horizontal component equal to the horizontal component of BVP2310 and a vertical component equal to -cbHeight based on horizontal distance 2314 being greater than vertical distance 2316, or a horizontal component equal to -cbWidth and a vertical component equal to -cbHeight based on horizontal distance 2314 being equal to vertical distance 2316.
[0169] BVP 2310 may be replaced by an adjusted BVP that includes a horizontal component equal to −cbWidth and a vertical component equal to the vertical component of BVP 2310, for example, based on horizontal distance 2402 being greater than or equal to cbWidth and vertical distance 2404 being less than −cbHeight. BVP 2310 may be replaced by an adjusted BVP that includes a horizontal component equal to the horizontal component of BVP 2310 and a vertical component equal to −cbHeight, for example, based on horizontal distance 2402 being less than cbWidth and vertical distance 2404 being greater than or equal to −cbHeight.
[0170] The IBC reference region 2304 shown in Figures 23A, 23B, 23C, and / or 24 is merely exemplary, and the IBC reference region may be different from the IBC reference region 2304. The methods, apparatus, and systems described with reference to Figures 23A, 23B, 23C, and / or 24 may be used / applied to an IBC reference region different from the IBC reference region 2304. For example, the IBC reference region 2304 may be replaced by an IBC reference region determined based on a different set of IBC reference region constraints. The IBC reference region 2304 may be (further) constrained to include the number / amount of decoded or reconstructed samples that can be stored in a limited memory size (e.g., IBC reference sample memory). The size of the IBC reference sample memory may be limited based on implementing an on-chip encoder and / or an on-chip decoder. The IBC reference region 2304 may be increased in size by using a larger-sized IBC reference sample memory off-chip from the encoder and / or off-chip from the decoder. Using off-chip memory may require higher memory bandwidth requirements and / or increased latency in writing and / or reading samples (e.g., in IBC reference region 2304) to and / or from IBC reference sample memory. IBC reference region 2304 may be determined and / or constrained according to various considerations described above with reference to IBC reference regions 1706, 2100, and 2118.
[0171] 25 shows an example method for replacing a BVP with an adjusted BVP. The method 2500 shown in FIG. 25 may be implemented by an encoder and / or a decoder (e.g., the encoder 200 shown in FIG. 2 and / or the decoder 300 shown in FIG. 3).
[0172] In step 2502, the sum of the horizontal component of the BVP and the width of the current block (cbWidth) may be determined to be greater than zero, and the sum of the vertical component of the BVP and the height of the current block (cbHeight) may be determined to be greater than zero. The BVP may indicate a displacement from the position of the current block to the position of the reference block.
[0173] In step 2504, a first horizontal distance and a first vertical distance may be determined. The first horizontal distance may be the distance from the position of the reference block to the first vertical boundary of the IBC reference area of the current block. The first vertical distance may be the distance from the position of the reference block to the first horizontal boundary of the IBC reference area of the current block. The first horizontal distance may be determined / calculated based on the sum of cbWidth and the horizontal component of the BVP. The first vertical distance may be determined / calculated based on the sum of cbHeight and the vertical component of the BVP.
[0174] In step 2506, the BVP may be replaced with an adjusted BVP. The BVP may be replaced with an adjusted BVP, for example, based on a comparison between the first horizontal distance and the first vertical distance. The BVP may be replaced with an adjusted BVP including a horizontal component equal to -cbWidth and a vertical component equal to the vertical component of the BVP, for example, based on the first horizontal distance being less than the first vertical distance. The BVP may be replaced with an adjusted BVP including a horizontal component equal to the horizontal component of the BVP and a vertical component equal to -cbHeight, for example, based on the first horizontal distance being greater than the first vertical distance. The BVP may be replaced with an adjusted BVP including a horizontal component equal to -cbWidth and a vertical component equal to -cbHeight, for example, based on the first horizontal distance being equal to the first vertical distance.
[0175] The BVP may be replaced with an adjusted BVP, for example, based on the fact that a second horizontal distance from the position of the current block to a second vertical boundary of the IBC reference region of the current block is greater than cbWidth and that a second vertical distance from the position of the current block to a second horizontal boundary of the IBC reference region of the current block is greater than cbHeight. The second vertical boundary of the IBC reference region may be the leftmost vertical boundary of the IBC reference region, and the second horizontal boundary of the IBC reference region may be the topmost horizontal boundary of the IBC reference region.
[0176] Method 2500 may include determining a BV based on the adjusted BVP. Method 2500 may include predicting a BV based on the adjusted BVP. For example, the BV may be predicted based on the BVP adjusted according to AMVP and / or merge mode. Method 2500 may include adding the adjusted BVP to a list of candidate BVPs. The list of candidate BVPs may be for AMVP and / or merge mode.
[0177] Various embodiments herein may be implemented in hardware (e.g., using analog and / or digital circuitry), software (e.g., through execution of stored / received instructions by one or more general-purpose or special-purpose processors), and / or a combination of hardware and software. Various embodiments herein may be implemented in an environment that includes a computer system or other processing system.
[0178] Figure 26 illustrates an example of a computer system that may use any of the embodiments described herein. For example, the exemplary computer system 2600 illustrated in Figure 26 may implement one or more of the methods described herein. For example, the various devices and / or systems described herein (e.g., Figures 1, 2, and 3) may be implemented in the form of one or more computer systems 2600. Furthermore, each of the steps of the flowcharts illustrated in this disclosure may be implemented on one or more computer systems 2600.
[0179] Computer system 2600 may include one or more processors, such as processor 2604. Processor 2604 may be a special purpose processor, a general purpose processor, a microprocessor, and / or a digital signal processor. Processor 2604 may be connected to a communications infrastructure 2602 (e.g., a bus or network). Computer system 2600 may also include main memory 2606 (e.g., random access memory (RAM)) and / or secondary memory 2608.
[0180] The secondary memory 2608 may include a hard disk drive 2610 and / or a removable storage drive 2612 (e.g., a magnetic tape drive, an optical disk drive, and / or the like). The removable storage drive 2612 may be read from and / or written to a removable storage unit 2616. The removable storage unit 2616 may include a magnetic tape, an optical disk, and / or the like. The removable storage unit 2616 may be read by and / or written to the removable storage drive 2612. The removable storage unit 2616 may include a computer-usable storage medium having computer software and / or data stored therein.
[0181] The secondary memory 2608 may include other similar means for allowing computer programs or other instructions to be loaded into the computer system 2600. Such means may include a removable storage unit 2618 and / or an interface 2614. Examples of such means may include a program cartridge and / or cartridge interface (such as a video game device), a removable memory chip (such as an erasable programmable read-only memory (EPROM) or a programmable read-only memory (PROM)), and associated sockets, thumb drives, and USB ports, and / or other removable storage units 2618 and interfaces 2614 that may allow software and / or data to be transferred from the removable storage unit 2618 to the computer system 2600.
[0182] Computer system 2600 may also include a communications interface 2620. Communications interface 2620 may allow software and data to be transferred between computer system 2600 and external devices. Examples of communications interface 2620 may include a modem, a network interface (e.g., an Ethernet card), a communications port, etc. Software and / or data transferred via communications interface 2620 may be in the form of signals, which may be electronic, electromagnetic, optical, and / or other signals that can be received by communications interface 2620. The signals may be provided to communications interface 2620 via communications path 2622. Communications path 2622 may transmit signals and may be implemented using wire or cable, fiber optics, a phone line, a cellular phone link, an RF link, and / or any other communications channel.
[0183] Computer program medium and / or computer-readable medium may be used to refer to tangible storage media, such as removable storage units 2616 and 2618, or a hard disk installed in hard disk drive 2610. A computer program product may be a means for providing software to computer system 2600. Computer programs (which may also be called computer control logic) may be stored in main memory 2606 and / or secondary memory 2608. Computer programs may be received via communications interface 2620. Such computer programs, when executed, may enable computer system 2600 to implement the present disclosure as discussed herein. In particular, the computer programs, when executed, may enable processor 2604 to perform processes of the present disclosure, such as any of the methods described herein. Thus, such computer programs may represent controllers of computer system 2600.
[0184] 27 shows exemplary elements of a computing device that may be used to implement any of the various devices described herein, including, for example, a source device (e.g., 102), an encoder (e.g., 200), a destination device (e.g., 106), a decoder (e.g., 300), and / or any computing device described herein. The computing device 2730 may include one or more processors 2731 that may execute instructions stored on random access memory (RAM) 2733, removable media 2734 (such as a universal serial bus (USB) drive, a compact disc (CD) or digital versatile disc (DVD), or a floppy disk drive), or any other desired storage medium. Instructions may also be stored on an attached (or internal) hard drive 2735. Computing device 2730 may also include a security processor (not shown) that may execute instructions of one or more computer programs to monitor processes running on processor 2731 and any process requesting access to any hardware and / or software components of computing device 2730 (e.g., ROM 2732, RAM 2733, removable media 2734, hard drive 2735, device controllers 2737, network interface 2739, GPS 2741, Bluetooth interface 2742, WiFi interface 2743, etc.). Computing device 2730 may include one or more output devices such as a display 2736 (e.g., a screen, display device, monitor, television, etc.) and may include one or more output device controllers 2737, such as a video processor. There may also be one or more user input devices 2738, such as a remote control, keyboard, mouse, touch screen, microphone, etc. Computing device 2730 may also include one or more network interfaces, such as network interface 2739, which may be a wired interface, a wireless interface, or a combination of the two.The network interface 2739 may provide an interface through which the computing device 2730 communicates with a network 2740 (e.g., a RAN, or any other network). The network interface 2739 may include a modem (e.g., a cable modem), and the external network 2740 may include a communications link, an external network, a home network, a provider's wireless, coaxial, fiber, or hybrid fiber / coaxial distribution system (e.g., a DOCSIS network), or any other desired network. Additionally, the computing device 2730 may include a location detection device such as a global positioning system (GPS) microprocessor 2741, which may be configured to receive and process global positioning signals and, with possible assistance from an external server and antenna, determine the geographic location of the computing device 2730.
[0185] While the example of FIG. 27 may be a hardware configuration, the components shown may be implemented as software. Changes may be made, as desired, to add, remove, combine, divide, etc., components of computing device 2730. Furthermore, components may be implemented using basic computing devices and components, and the same components (e.g., processor 2731, ROM storage 2732, display 2736, etc.) may be used to implement any of the other computing devices and components described herein. For example, the various components described herein may be implemented using a computing device having components such as a processor that executes computer-executable instructions stored on a computer-readable medium, as shown in FIG. 27. Some or all of the entities described herein may be software-based and coexist on a common physical platform (e.g., a requesting entity may be a separate software process and program from a dependent entity, both of which may run as software on a common computing device).
[0186] Various features are highlighted below in a set of numbered clauses or paragraphs. These features are not to be construed as limiting the invention or inventive concept, but are provided merely as highlighting some of the features described herein, without implying the importance or relevance of any particular order of such features.
[0187] Article 1. 1. A method comprising: determining, by a computing device, one or more candidate block vector predictors (BVPs), each of the one or more candidate BVPs including at least one of a horizontal component determined based on a width of a current block or a vertical component determined based on a height of the current block.
[0188] Article 2. The method of clause 1, further comprising determining a block vector (BV) of the current block based on a list of candidate BVPs that is modified to include at least one candidate BVP from among one or more candidate BVPs.
[0189] Article 3. 3. The method of any one of clauses 1 and 2, wherein at least one candidate BVP indicates a displacement from the current block to a location within an intra block copy (IBC) reference area.
[0190] Article 4. The method of any one of clauses 1 to 3, wherein at least one candidate BVP is selected from one or more candidate BVPs based on the horizontal distance as the sum of the horizontal component of a BVP in the list of candidate BVPs and the width of the current block being greater than zero, and the vertical distance as the sum of the vertical component of a BVP in the list of candidate BVPs and the height of the current block being greater than zero.
[0191] Article 5. 5. The method of any one of clauses 1 to 4, wherein the width of the current block is cbWidth, the height of the current block is cbHeight, and among at least one candidate BVP, the horizontal component of the candidate BVP is equal to -cbWidth and the vertical component of the candidate BVP is equal to -cbHeight.
[0192] Article 6. A method according to any one of clauses 1 to 5, wherein the width of the current block is cbWidth, and among at least one candidate BVP, the horizontal component of the candidate BVP is equal to -cbWidth, and the vertical component of the candidate BVP is equal to the vertical component of a BVP in the list of candidate BVPs.
[0193] Article 7. The method of any one of clauses 1 to 6, wherein the height of the current block is cbHeight, and among at least one candidate BVP, the vertical component of the candidate BVP is equal to -cbHeight, and the horizontal component of the candidate BVP is equal to the horizontal component of a BVP in the list of candidate BVPs.
[0194] Article 8. 8. The method of any one of clauses 1 to 7, wherein at least one candidate BVP is selected from among the one or more candidate BVPs based on determining a respective block vector difference (BVD) for each of the one or more candidate BVPs, and selecting the at least one candidate BVP based on the BVD for the at least one candidate BVP being smallest among the respective BVDs.
[0195] Article 9. 9. The method of any one of clauses 1 to 8, wherein determining one or more candidate BVPs is based on determining that a BVP in the list of candidate BVPs is outside the IBC reference region.
[0196] Article 10. 10. The method of any one of clauses 1 to 9, wherein the width of the current block is cbWidth, the height of the current block is cbHeight, and determining one or more candidate BVPs is based on determining that a horizontal component of the BVP is greater than -cbWidth and a vertical component of the BVP is greater than -cbHeight.
[0197] Article 11. 11. The method of any one of clauses 1-10, wherein the computing device includes an encoder, and the method further includes transmitting a representation of the at least one candidate BVP to a decoder.
[0198] Article 12. 12. The method of any one of clauses 1 to 11, further comprising replacing at least one BVP in the list of candidate BVPs with at least one candidate BVP.
[0199] Article 13. 13. The method of any one of clauses 1 to 12, further comprising: determining a block vector difference (BVD) based on the BV of at least one candidate BVP and the candidate BVP; and transmitting an indication of the BVD to a decoder.
[0200] Article 14. 14. The method of any one of clauses 1 to 13, further comprising encoding the current block based on a reference block indicated by BV.
[0201] Article 15. 15. The method of any one of clauses 1-14, further comprising receiving an indication of a block vector difference (BVD) and determining a BV based on the BVD and at least one candidate BVP.
[0202] Article 16. 16. The method of any one of clauses 1 to 15, further comprising decoding the current block based on the reference block indicated by BV.
[0203] Article 17. A computing device comprising one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the method of any one of clauses 1 to 16.
[0204] Article 18. A system comprising: a first computing device configured to perform the method of any one of clauses 1 to 16; and a second computing device configured to encode or decode a current block using at least one candidate BVP.
[0205] Article 19. A computer readable medium storing instructions that, when executed, cause the performance of the method of any one of clauses 1 to 16.
[0206] Article 20. The method includes determining that a horizontal distance as the sum of a horizontal component of a block vector predictor (BVP) and a width of a current block in a list of candidate BVPs is greater than zero, and a vertical distance as the sum of a vertical component of the BVP and a height of the current block is greater than zero.
[0207] Article 21. 21. The method of clause 20, further comprising replacing a BVP in the list of candidate BVPs with the adjusted BVP based on the determination.
[0208] Article 22. 22. The method of any one of clauses 20 and 21, further comprising determining a block vector (BV) for a current block using the list of candidate BVPs.
[0209] Article 23. 23. The method of any one of clauses 20 to 22, wherein the adjusted BVP includes at least one of a horizontal component determined based on a width of the current block or a vertical component determined based on a height of the current block.
[0210] Article 24. 24. The method of any one of clauses 20 to 23, wherein the width of the current block is cbWidth and the horizontal component of the adjusted BVP is equal to −cbWidth.
[0211] Article 25. 25. The method of any one of clauses 20 to 24, wherein the height of the current block is cbHeight and the vertical component of the adjusted BVP is equal to −cbHeight.
[0212] Article 26. 26. The method of any one of clauses 20 to 25, wherein the width of the current block is cbWidth, the horizontal component of the adjusted BVP is equal to -cbWidth, and the vertical component of the adjusted BVP is equal to the vertical component of BVP.
[0213] Article 27. 27. The method of any one of clauses 20 to 26, wherein the height of the current block is cbHeight, the vertical component of the adjusted BVP is equal to -cbHeight, and the horizontal component of the adjusted BVP is equal to the horizontal component of BVP.
[0214] Article 28. 28. The method of any one of clauses 20-27, wherein replacing the BVP with the adjusted BVP includes selecting the adjusted BVP from one or more candidate BVPs based on the block vector difference (BVD) for the adjusted BVP being the smallest among the BVDs determined for each of the one or more candidate BVPs.
[0215] Article 29. 29. The method of any one of clauses 20 to 28, wherein replacing the BVP with the adjusted BVP includes determining, by a computing device, one or more candidate BVPs, each of the one or more candidate BVPs including at least one of a horizontal component determined based on a width of the current block or a vertical component determined based on a height of the current block.
[0216] Article 30. 30. The method of any one of clauses 20-29, wherein the width of the current block is cbWidth, the height of the current block is cbHeight, and the one or more candidate BVPs include one or more of: a first candidate BVP that includes a horizontal component equal to -cbWidth and a vertical component equal to the vertical component of the BVP; a second candidate BVP that includes a horizontal component equal to the horizontal component of the BVP and a vertical component equal to -cbHeight; and a third candidate BVP that includes a horizontal component equal to cbWidth and a vertical component equal to -cbHeight.
[0217] Article 31. 31. The method of any one of clauses 20-30, wherein replacing the BVP with the adjusted BVP includes replacing the BVP with a selected one of one or more candidate BVPs.
[0218] Article 32. The method of any one of clauses 20 to 31, further comprising determining one or more valid candidate BVPs among the one or more candidate BVPs, and wherein replacing the BVP with the adjusted BVP comprises replacing the BVP with a selected one of the one or more valid candidate BVPs.
[0219] Article 33. 33. The method of any one of clauses 20-32, further comprising transmitting an indication of the adjusted BVP.
[0220] Article 34. 34. The method of any one of clauses 20-33, further comprising determining a block vector difference (BVD) using the adjusted BVP.
[0221] Article 35. 35. The method of any one of clauses 20-34, further comprising receiving an indication of the adjusted BVP.
[0222] Article 36. 36. The method of any one of clauses 20-35, further comprising determining the BV using the adjusted BVP.
[0223] Article 37. A computing device comprising one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the method of any one of clauses 20 to 36.
[0224] Article 38. A system comprising: a first computing device configured to perform the method of any one of clauses 20 to 36; and a second computing device configured to encode or decode a current block using the adjusted BVP.
[0225] Article 39. A computer readable medium storing instructions that, when executed, cause the performance of the method of any one of clauses 20-36.
[0226] Article 40. 1. A method comprising: determining, by a computing device, that a block vector predictor (BVP) in a list of candidate BVPs is outside an inter-block copy (IBC) reference region associated with a current block based on a BVP indicating a displacement from the current block to a position outside the IBC reference region.
[0227] Article 41. 41. The method of clause 40, further comprising, based on the determination, replacing a BVP in the list of candidate BVPs with an adjusted BVP indicating a displacement from the current block to a position within an intra block copy (IBC) reference region.
[0228] Article 42. 42. The method of any one of clauses 40 and 41, further comprising determining a block vector (BV) for a current block using the list of candidate BVPs.
[0229] Article 43. 43. The method of any one of clauses 40 to 42, wherein determining that the BVP is outside the ICB associated with the current block is based on the horizontal distance as the sum of the horizontal component of the BVP and the width of the current block being greater than zero, and the vertical distance as the sum of the vertical component of the BVP and the height of the current block being greater than zero.
[0230] Article 44. 44. The method of any one of clauses 40 to 43, further comprising selecting an adjusted BVP from one or more candidate BVPs, the adjusted BVP including at least one of a horizontal component determined based on a width of the current block or a vertical component determined based on a height of the current block.
[0231] Article 45. 45. The method of any one of clauses 40-44, wherein replacing the BVP with the adjusted BVP includes selecting the adjusted BVP from one or more candidate BVPs based on the block vector difference (BVD) for the adjusted BVP being smallest among the BVDs determined for each of the one or more candidate BVPs.
[0232] Article 46. 46. The method of any one of clauses 40-45, further comprising transmitting an indication of the adjusted BVP to the receiving device.
[0233] Article 47. A computing device comprising one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the method of any one of clauses 40 to 46.
[0234] Article 48. 47. A system comprising: a first computing device configured to perform the method of any one of clauses 40-46; and a second computing device configured to encode or decode a current block using the adjusted BVP.
[0235] Article 49. A computer readable medium storing instructions that, when executed, cause the performance of the method of any one of clauses 40-46.
[0236] Article 50. 1. A method comprising: calculating, by a computing device, a horizontal distance as the sum of a horizontal component of a block vector predictor (BVP) and a width of a current block of a video frame, and a vertical distance as the sum of a vertical component of the BVP and a height of the current block, wherein the BVP indicates a position of a reference block relative to a position of the current block.
[0237] Article 51. 51. The method of clause 50, further comprising determining that the horizontal distance is greater than zero and the vertical distance is greater than zero.
[0238] Article 52. 52. The method of any one of clauses 50 and 51, further comprising replacing the BVP with an adjusted BVP based on the determination and based on a comparison between the horizontal distance and the vertical distance and based on a comparison between the horizontal distance and the vertical distance, wherein the horizontal component of the adjusted BVP is based on a width of the current block or the vertical component of the adjusted BVP is based on a height of the current block.
[0239] Article 53. 53. The method of any one of clauses 50-52, wherein the width of the current block is cbWidth, and the method further includes, based on the horizontal distance being less than the vertical distance, setting a horizontal component of the adjusted BVP equal to -cbWidth, and setting a vertical component of the adjusted BVP equal to the vertical component of the BVP.
[0240] Article 54. 54. The method of any one of clauses 50-53, wherein the height of the current block is cbHeight, and the method further includes, based on the horizontal distance being greater than the vertical distance, setting a horizontal component of the adjusted BVP equal to the horizontal component of the BVP, and setting a vertical component of the adjusted BVP equal to -cbHeight.
[0241] Article 55. 55. The method of any one of clauses 50-54, wherein the width of the current block is cbWidth and the height of the current block is cbHeight, and the method further includes setting a horizontal component of the adjusted BVP equal to -cbWidth and setting a vertical component of the adjusted BVP equal to -cbHeight based on the horizontal distance being equal to the first vertical distance.
[0242] Article 56. 56. The method of any one of clauses 50 to 55, wherein replacing the BVP with the adjusted BVP is further based on at least one of: a second horizontal distance from the position of the current block to a left-most vertical boundary of an inter-block copying (IBC) reference area is greater than a width of the current block; or a second vertical distance from the position of the current block to a top-most horizontal boundary of the IBC reference area of the current block is greater than a height of the current block.
[0243] Article 57. 57. The method of any one of clauses 50-56, further comprising receiving an indication of a block vector difference (BVD).
[0244] Article 58. 58. The method of any one of clauses 50 to 57, further comprising determining a block vector (BV) of the current block as a combination of the adjusted BVP and a block vector difference (BVD).
[0245] Article 59. 59. The method of any one of clauses 50 to 58, further comprising determining a block vector difference (BVD) as a combination of the block vector (BV) of the current block and the adjusted BVP.
[0246] Article 60. 60. The method of any one of clauses 50-59, further comprising transmitting an indication of a block vector difference (BVD).
[0247] Article 61. 61. The method of any one of clauses 50-60, further comprising adding the adjusted BVP to a list of candidate BVPs associated with advanced motion vector prediction (AMVP) or associated with merge mode.
[0248] Article 62. 62. The method of any one of clauses 50-61, further comprising using the adjusted BVP to encode or decode a current block.
[0249] Article 63. 63. The method of any one of clauses 50 to 62, wherein determining one or more candidate BVPs is based on determining that a BVP in the list of candidate BVPs is outside the IBC criteria region.
[0250] Article 64. A computing device comprising one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the method of any one of clauses 50 to 63.
[0251] Article 65. A system comprising: a first computing device configured to perform the method of any one of clauses 50 to 63; and a second computing device configured to encode or decode a current block using the adjusted BVP.
[0252] Article 66. A computer readable medium storing instructions that, when executed, cause the performance of the method of any one of clauses 50-63.
[0253] Article 67. 1. A method comprising: calculating, by a computing device, a horizontal distance from a position of a reference block to a vertical boundary of an intra block copy (IBC) reference area of a current block of a video frame; and a vertical distance from the position of the reference block to a horizontal boundary of the IBC reference area of the current block.
[0254] Article 68. 68. The method of clause 67, further comprising determining that the sum of a horizontal component of a block vector predictor (BVP) and a width of the current block is greater than zero, and determining that the sum of a vertical component of the BVP and a height of the current block is greater than zero, wherein the BVP indicates a position of a reference block relative to a position of the current block.
[0255] Article 69. 69. The method of any one of clauses 67 and 68, further comprising replacing the BVP with an adjusted BVP based on the determination and based on a comparison of the horizontal distance and the vertical distance.
[0256] Article 70. 69. The method of any one of clauses 67-69, wherein the width of the current block is cbWidth, and the method further comprises, based on the horizontal distance being less than the vertical distance, setting a horizontal component of the adjusted BVP equal to -cbWidth, and setting a vertical component of the adjusted BVP equal to the vertical component of the BVP.
[0257] Article 71. 71. The method of any one of clauses 67-70, further comprising: setting a horizontal component of the adjusted BVP equal to the horizontal component of the BVP based on the height of the current block being cbHeight and the horizontal distance being greater than the first vertical distance; and setting a vertical component of the adjusted BVP equal to -cbHeight.
[0258] Article 72. 72. The method of any one of clauses 67-71, wherein the width of the current block is cbWidth and the height of the current block is cbHeight, and the method further includes setting a horizontal component of the adjusted BVP equal to -cbWidth and setting a vertical component of the adjusted BVP equal to -cbHeight based on the horizontal distance being equal to the first vertical distance.
[0259] Article 73. 73. The method of any one of clauses 67 to 72, wherein replacing the BVP with the adjusted BVP is further based on at least one of: a second horizontal distance from the position of the current block to a second vertical boundary of the IBC reference area of the current block is greater than a width of the current block, and the second vertical boundary of the IBC reference area is the leftmost vertical boundary of the IBC reference area; a second vertical distance from the position of the current block to a second horizontal boundary of the IBC reference area of the current block is greater than a height of the current block, and the second horizontal boundary of the IBC reference area is the topmost horizontal boundary of the IBC reference area.
[0260] Article 74. 74. The method of any one of clauses 67 to 73, wherein the horizontal distance is based on the sum of the horizontal component of the BVP and the width of the current block, and the vertical distance is based on the sum of the vertical component of the BVP and the height of the current block.
[0261] Article 75. A computing device comprising one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the method of any one of clauses 67 to 74.
[0262] Article 76. A system comprising: a first computing device configured to perform the method of any one of clauses 67 to 74; and a second computing device configured to encode or decode a current block using the adjusted BVP.
[0263] Article 77. A computer readable medium storing instructions that, when executed, cause the performance of the method of any one of clauses 67-74.
[0264] Article 78. A method comprising: determining, by a computing device and based on a block vector predictor (BVP) of the reference block, that the reference block overlaps with a current block of a video frame.
[0265] Article 79. 79. The method of clause 78, further comprising: calculating, based on the determination, a horizontal distance from the position of the reference block to a vertical boundary of an intra block copy (IBC) reference area of the current block, and a vertical distance from the position of the reference block to a horizontal boundary of the IBC reference area of the current block.
[0266] Article 80. 80. The method of any one of clauses 78 and 79, further comprising replacing the BVP with an adjusted BVP based on a comparison of the horizontal distance and the vertical distance.
[0267] Article 81. 81. The method of any one of clauses 78 to 80, wherein replacing the BVP with the adjusted BVP includes setting a horizontal component of the adjusted BVP equal to -cbWidth based on the horizontal distance being less than the vertical distance, where cbWidth is the width of the current block, and setting a vertical component of the adjusted BVP equal to the vertical component of the BVP.
[0268] Article 82. 82. The method of any one of clauses 78-81, wherein replacing the BVP with the adjusted BVP includes setting a horizontal component of the adjusted BVP equal to the horizontal component of the BVP based on the horizontal distance being greater than the vertical distance, and setting a vertical component of the adjusted BVP equal to -cbHeight, where cbHeight is the height of the current block.
[0269] Article 83. 83. The method of any one of clauses 78 to 82, wherein replacing the BVP with the adjusted BVP includes setting a horizontal component of the adjusted BVP equal to -cbWidth based on the horizontal distance being equal to the first vertical distance, where cbWidth is the width of the current block, and setting a vertical component of the adjusted BVP equal to -cbHeight, where cbHeight is the height of the current block.
[0270] Article 84. 84. The method of any one of clauses 78 to 83, further comprising calculating a second horizontal distance from the position of the current block to a second vertical boundary of the IBC reference area, wherein the second vertical boundary is the leftmost vertical boundary of the IBC reference area, and a second vertical distance from the position of the current block to a second horizontal boundary of the IBC reference area, wherein the second horizontal boundary is the topmost horizontal boundary of the IBC reference area.
[0271] Article 85. 85. The method of any one of clauses 78 to 84, wherein replacing the BVP with the adjusted BVP includes setting a horizontal component of the adjusted BVP equal to -cbWidth, where cbWidth is the width of the current block, based on the second horizontal distance being greater than the width of the current block and the second vertical distance being less than the height of the current block, and setting a vertical component of the adjusted BVP equal to the vertical component of the BVP.
[0272] Article 86. 86. The method of any one of clauses 78 to 85, wherein replacing the BVP with the adjusted BVP includes setting a horizontal component of the adjusted BVP equal to the horizontal component of the BVP based on the second horizontal distance being less than the width of the current block and the second vertical distance being greater than the height of the current block, and setting a vertical component of the adjusted BVP equal to -cbHeight, where cbHeight is the height of the current block.
[0273] Article 87. A computing device comprising one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the method of any one of clauses 78 to 86.
[0274] Article 88. A system comprising: a first computing device configured to perform the method of any one of clauses 78 to 86; and a second computing device configured to encode or decode a current block using the adjusted BVP.
[0275] Article 89. A computer readable medium storing instructions that, when executed, cause the performance of the method of any one of clauses 78-86.
[0276] Article 90. A method comprising: determining, by a computing device and based on a block vector predictor (BVP) of the reference block, that the reference block overlaps with a current block of a video frame.
[0277] Article 91. 91. The method of clause 90, further comprising: calculating, based on the determination, a first horizontal distance from the position of the current block to a first vertical boundary of an intra block copy (IBC) reference area of the current block, and a first vertical distance from the position of the current block to a first horizontal boundary of the IBC reference area of the current block.
[0278] Article 92. 92. The method of any one of clauses 90 and 91, further comprising replacing the BVP with an adjusted BVP based on the first horizontal distance and the first vertical distance.
[0279] Article 93. The method of any one of clauses 90 to 92, wherein replacing the BVP with the adjusted BVP further comprises setting a horizontal component of the adjusted BVP equal to -cbWidth, where cbWidth is the width of the current block, based on the first horizontal distance being greater than the width of the current block and the first vertical distance being less than the height of the current block, and setting a vertical component of the adjusted BVP equal to the vertical component of the BVP.
[0280] Article 94. The method of any one of clauses 90 to 93, wherein replacing the BVP with the adjusted BVP includes setting a horizontal component of the adjusted BVP equal to the horizontal component of the BVP based on the first horizontal distance being less than the width of the current block and the first vertical distance being greater than the height of the current block, and setting a vertical component of the adjusted BVP equal to -cbHeight, where cbHeight is the height of the current block.
[0281] Article 95. The method of any one of clauses 90 to 94, further comprising: calculating, based on the first horizontal distance being greater than the width of the current block and the first vertical distance being greater than the height of the current block, a second horizontal distance from the position of the reference block to a second vertical boundary of the IBC reference area of the current block, and a second vertical distance from the position of the reference block to the second horizontal boundary of the IBC reference area of the current block.
[0282] Article 96. The method of any one of clauses 90 to 95, wherein replacing the BVP with the adjusted BVP includes setting a horizontal component of the adjusted BVP equal to -cbWidth based on the second horizontal distance being less than the second vertical distance, where cbWidth is the width of the current block, and setting a vertical component of the adjusted BVP equal to the vertical component of the BVP.
[0283] Article 97. 97. The method of any one of clauses 90-96, wherein replacing the BVP with the adjusted BVP includes setting a horizontal component of the adjusted BVP equal to the horizontal component of the BVP based on the second horizontal distance being greater than the second vertical distance, and setting a vertical component of the adjusted BVP equal to -cbHeight, where cbHeight is the height of the current block.
[0284] Article 98. 98. The method of any one of clauses 90-97, wherein replacing the BVP with the adjusted BVP includes setting a horizontal component of the adjusted BVP equal to -cbWidth based on the second horizontal distance being equal to the second vertical distance, where cbWidth is the width of the current block, and setting a vertical component of the adjusted BVP equal to -cbHeight, where cbHeight is the height of the current block.
[0285] Article 99. A computing device comprising one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the method of any one of clauses 90 to 98.
[0286] Article 100. A system comprising: a first computing device configured to perform the method of any one of clauses 90-98; and a second computing device configured to encode or decode a current block using the adjusted BVP.
[0287] Article 101. A computer readable medium storing instructions that, when executed, cause the performance of the method recited in any one of clauses 90-98.
[0288] Article 102. A method comprising: determining, by a computing device and based on a block vector predictor (BVP) of the reference block, that the reference block overlaps with a current block of a video frame.
[0289] Article 103. Clause 103. The method of clause 102, further comprising: based on determining, calculating a first horizontal distance from the position of the reference block to a first vertical boundary of an intra block copy (IBC) reference area of the current block, and a first vertical distance from the position of the reference block to a first horizontal boundary of the IBC reference area of the current block.
[0290] Article 104. 104. The method of any one of clauses 102 and 103, further comprising replacing the BVP with an adjusted BVP based on a comparison of the first horizontal distance and the first vertical distance.
[0291] Article 105. 105. The method of any one of clauses 102 to 104, wherein determining that the reference block overlaps with the current block comprises determining that a sum of a horizontal component of the BVP and a width of the current block is greater than zero and a sum of a vertical component of the BVP and a height of the current block is greater than zero.
[0292] Article 106. The method of any one of clauses 102 to 105, wherein replacing the BVP with the adjusted BVP is further based on the following: a second horizontal distance from the position of the current block to a second vertical boundary of the IBC reference area of the current block is greater than the width of the current block; and a second vertical distance from the position of the current block to a second horizontal boundary of the IBC reference area of the current block is greater than the height of the current block.
[0293] Article 107. 107. The method of any one of clauses 102 to 106, wherein the second vertical boundary of the IBC reference area is the left-most vertical boundary of the IBC reference area and the second horizontal boundary of the IBC reference area is the top-most horizontal boundary of the IBC reference area.
[0294] Article 108. The method of any one of clauses 102 to 107, wherein replacing the BVP with the adjusted BVP includes setting a horizontal component of the adjusted BVP equal to -cbWidth based on the first horizontal distance being less than the first vertical distance, where cbWidth is the width of the current block, and setting a vertical component of the adjusted BVP equal to the vertical component of the BVP.
[0295] Article 109. 109. The method of any one of clauses 102-108, wherein replacing the BVP with the adjusted BVP includes setting a horizontal component of the adjusted BVP equal to the horizontal component of the BVP based on the first horizontal distance being greater than the first vertical distance, and setting a vertical component of the adjusted BVP equal to -cbHeight, where cbHeight is the height of the current block.
[0296] Article 110. 109. The method of any one of clauses 102-109, wherein replacing the BVP with the adjusted BVP includes setting a horizontal component of the adjusted BVP equal to -cbWidth based on the first horizontal distance being equal to the first vertical distance, where cbWidth is the width of the current block, and setting a vertical component of the adjusted BVP equal to -cbHeight, where cbHeight is the height of the current block.
[0297] Article 111. A computing device comprising one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the method of any one of clauses 102 to 110.
[0298] Article 112. A system comprising: a first computing device configured to perform the method of any one of clauses 102 to 110; and a second computing device configured to encode or decode a current block using the adjusted BVP.
[0299] Article 113. A computer-readable medium storing instructions that, when executed, cause the performance of the method recited in any one of clauses 102-110.
[0300] A computing device may perform a method including multiple operations. The computing device may determine one or more candidate block vector predictors (BVPs), each of which includes at least one of a horizontal component determined based on the width of the current block or a vertical component determined based on the height of the current block. The computing device may determine a block vector (BV) for the current block based on a list of candidate BVPs modified to include at least one candidate BVP from among the one or more candidate BVPs that indicates a displacement from the current block to a position within an intra block copy (IBC) reference region. The computing device may also perform one or more additional operations. At least one candidate BVP is selected from among one or more candidate BVPs based on a horizontal distance, as the sum of the horizontal component of a BVP in the list of candidate BVPs and the width of the current block, being greater than zero, and a vertical distance, as the sum of the vertical component of a BVP in the list of candidate BVPs and the height of the current block, being greater than zero. The width of the current block may be cbWidth. The height of the current block may be cbHeight. Among at least one candidate BVP, the horizontal component of the candidate BVP may be equal to -cbWidth and the vertical component of the candidate BVP may be equal to -cbHeight. Among at least one candidate BVP, the horizontal component of the candidate BVP may be equal to -cbWidth and the vertical component of the candidate BVP may be equal to the vertical component of a BVP in the list of candidate BVPs. Among at least one candidate BVP, the vertical component of the candidate BVP may be equal to -cbHeight and the horizontal component of the candidate BVP may be equal to the horizontal component of a BVP in the list of candidate BVPs. At least one candidate BVP may be selected from among one or more candidate BVPs based on determining a respective block vector difference (BVD) for each of the one or more candidate BVPs and selecting the at least one candidate BVP based on the BVD of the at least one candidate BVP being smallest among the respective BVDs.Determining one or more candidate BVPs may be based on determining that a BVP in the list of candidate BVPs is outside the IBC reference region. Determining one or more candidate BVPs may be based on determining that a horizontal component of the BVP is greater than -cbWidth and a vertical component of the BVP is greater than -cbHeight. The computing device may include an encoder. The computing device may send an indication of at least one candidate BVP to a decoder. The computing device may replace at least one BVP in the list of candidate BVPs with at least one candidate BVP. The computing device may determine a block vector difference (BVD) based on the BV and the candidate BVP of the at least one candidate BVP. The computing device may send an indication of the BVD to a decoder. The computing device may encode a current block based on a reference block indicated by the BV. The computing device may receive an indication of the block vector difference (BVD). The computing device may determine a BV based on the BVD and the candidate BVP of the at least one candidate BVP. The computing device may decode the current block based on the reference block indicated by the BV. A computing device may include one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements. A system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to encode or decode a current block using at least one candidate BVP. A computer-readable medium may store instructions that, when executed, cause performance of the described methods, additional operations, and / or include additional elements.
[0301] A computing device may perform a method including multiple operations. The computing device may determine that a horizontal distance as the sum of a horizontal component of a block vector predictor (BVP) and the width of a current block in a list of candidate BVPs is greater than zero, and a vertical distance as the sum of a vertical component of the BVP and the height of the current block is greater than zero. The computing device may replace a BVP in the list of candidate BVPs with an adjusted BVP based on the determination. The computing device may determine a block vector (BVP) for the current block using the list of candidate BVPs. The computing device may also perform one or more additional operations. The adjusted BVP may include at least one of a horizontal component determined based on the width of the current block or a vertical component determined based on the height of the current block. The width of the current block may be cbWidth, and the height of the current block may be cbHeight. The horizontal component of the adjusted BVP may be equal to −cbWidth. The vertical component of the adjusted BVP may be equal to −cbHeight. The horizontal component of the adjusted BVP may be equal to -cbWidth, and the vertical component of the adjusted BVP may be equal to the vertical component of the BVP. The vertical component of the adjusted BVP may be equal to -cbHeight, and the horizontal component of the adjusted BVP may be equal to the horizontal component of the BVP. Replacing the BVP with the adjusted BVP may include selecting the adjusted BVP from one or more candidate BVPs based on a block vector difference (BVD) for the adjusted BVP being smallest among the BVDs determined for each of the one or more candidate BVPs. Replacing the BVP with the adjusted BVP may include determining, by a computing device, one or more candidate BVPs. Each of the one or more candidate BVPs may include at least one of a horizontal component determined based on the width of the current block or a vertical component determined based on the height of the current block.The one or more candidate BVPs may include one or more of: a first candidate BVP including a horizontal component equal to -cbWidth and a vertical component equal to the vertical component of the BVP; a second candidate BVP including a horizontal component equal to the horizontal component of the BVP and a vertical component equal to -cbHeight; and a third candidate BVP including a horizontal component equal to -cbWidth and a vertical component equal to -cbHeight. Replacing the BVP with the adjusted BVP may include replacing the BVP with a selected one of the one or more candidate BVPs. The computing device may determine one or more valid candidate BVPs among the one or more candidate BVPs. Replacing the BVP with the adjusted BVP may include replacing the BVP with a selected one of the one or more valid candidate BVPs. The computing device may send an indication of the adjusted BVP. The computing device may determine a block vector difference (BVD) using the adjusted BVP. The computing device may receive an indication of the adjusted BVP. The computing device may determine a BV using the adjusted BVP. The computing device may include one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements. A system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to encode or decode a current block using the adjusted BVP. A computer-readable medium may store instructions that, when executed, cause performance of the described methods, additional operations, and / or include additional elements.
[0302] A computing device may perform a method including multiple operations. The computing device may determine that a block vector predictor (BVP) in a list of candidate BVPs is outside an inter-block copy (IBC) reference region associated with a current block based on the BVP indicating a displacement from the current block to a position outside the IBC reference region. Based on the determination, the computing device may replace the BVP in the list of candidate BVPs with an adjusted BVP indicating a displacement from the current block to a position within the intra-block copy (IBC) reference region. The computing device may also perform one or more additional operations. Determining that a BVP is outside an IBC associated with the current block may be based on a horizontal distance, as the sum of a horizontal component of the BVP and a width of the current block, being greater than zero, and a vertical distance, as the sum of a vertical component of the BVP and a height of the current block, being greater than zero. The computing device may select an adjusted BVP from one or more candidate BVPs that includes at least one of a horizontal component determined based on the width of the current block or a vertical component determined based on the height of the current block. Replacing the BVP with the adjusted BVP may include selecting the adjusted BVP from the one or more candidate BVPs based on a block vector difference (BVD) for the adjusted BVP being smallest among the BVDs determined for each of the one or more candidate BVPs. The computing device may transmit an indication of the adjusted BVP to a receiving device. The computing device may include one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements.The system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to encode or decode the current block using the adjusted BVP. A computer-readable medium may store instructions that, when executed, cause the performance of the described methods, additional operations, and / or include additional elements.
[0303] A computing device may perform a method including multiple operations. The computing device may calculate a horizontal distance as the sum of a horizontal component of a block vector predictor (BVP) and the width of a current block of a video frame, and a vertical distance as the sum of a vertical component of the BVP and the height of the current block. The BVP may indicate the position of a reference block relative to the position of the current block. The computing device may determine that the horizontal distance is greater than zero and that the vertical distance is greater than zero. The computing device may replace the BVP with an adjusted BVP based on the determination and based on a comparison of the horizontal distance and the vertical distance. The horizontal component of the adjusted BVP may be based on the width of the current block. The vertical component of the adjusted BVP may be based on the height of the current block. The computing device may also perform one or more additional operations. The width of the current block may be cbWidth, and the height of the current block may be cbHeight. The computing device may set the horizontal component of the adjusted BVP equal to −cbWidth and set the vertical component of the adjusted BVP equal to the vertical component of the BVP based on the horizontal distance being less than the vertical distance. The computing device may set a horizontal component of the adjusted BVP equal to the horizontal component of the BVP and set a vertical component of the adjusted BVP equal to −cbHeight based on the horizontal distance being greater than the vertical distance. The computing device may set a horizontal component of the adjusted BVP equal to −cbWidth and set a vertical component of the adjusted BVP equal to −cbHeight based on the horizontal distance being equal to the first vertical distance. Replacing the BVP with the adjusted BVP may be further based on at least one of a second horizontal distance from the position of the current block to a left-most vertical boundary of the inter-block copy (IBC) reference area being greater than the width of the current block, or a second vertical distance from the position of the current block to a top-most horizontal boundary of the IBC reference area of the current block being greater than the height of the current block.The computing device may receive an indication of the block vector difference (BVD). The computing device may determine a block vector (BV) of a current block as a combination of the adjusted BVP and the block vector difference (BVD). The computing device may determine a block vector difference (BVD) as a combination of the block vector (BV) of the current block and the adjusted BVP. The computing device may transmit an indication of the block vector difference (BVD). The computing device may add the adjusted BVP to a list of candidate BVPs associated with advanced motion vector prediction (AMVP) or associated with merge mode. The computing device may use the adjusted BVP to encode or decode the current block. Determining one or more candidate BVPs may be based on determining that a BVP in the list of candidate BVPs is outside the IBC reference region. The computing device may include one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements. The system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to encode or decode the current block using the adjusted BVP. A computer-readable medium may store instructions that, when executed, cause the performance of the described methods, additional operations, and / or include additional elements.
[0304] A computing device may perform a method including multiple operations. The computing device may calculate a horizontal distance from the position of the reference block to a vertical boundary of an intra block copy (IBC) reference area of a current block of a video frame and a vertical distance from the position of the reference block to a horizontal boundary of the IBC reference area of the current block. The computing device may determine that a sum of a horizontal component of a block vector predictor (BVP) and a width of the current block is greater than zero, and a sum of a vertical component of the BVP and a height of the current block is greater than zero. The BVP may indicate a position of the reference block relative to a position of the current block. Based on the determination and a comparison of the horizontal distance and the vertical distance, the computing device may replace the BVP with an adjusted BVP. The computing device may also perform one or more additional operations. The width of the current block may be cbWidth, and the height of the current block may be cbHeight. Based on the horizontal distance being less than the vertical distance, the computing device may set a horizontal component of the adjusted BVP equal to −cbWidth and a vertical component of the adjusted BVP equal to the vertical component of the BVP. The computing device may set a horizontal component of the adjusted BVP equal to the horizontal component of the BVP and a vertical component of the adjusted BVP equal to −cbHeight based on the horizontal distance being greater than the first vertical distance. The computing device may set a horizontal component of the adjusted BVP equal to −cbWidth and a vertical component of the adjusted BVP equal to −cbHeight based on the horizontal distance being equal to the first vertical distance.Replacing the BVP with the adjusted BVP may be further based on at least one of: a second horizontal distance from the position of the current block to a second vertical boundary of the IBC reference region of the current block is greater than the width of the current block, and the second vertical boundary of the IBC reference region is the left-most vertical boundary of the IBC reference region; a second vertical distance from the position of the current block to the second horizontal boundary of the IBC reference region of the current block is greater than the height of the current block, and the second horizontal boundary of the IBC reference region is the top-most horizontal boundary of the IBC reference region. The horizontal distance may be based on the sum of the horizontal component of the BVP and the width of the current block. The vertical distance may be based on the sum of the vertical component of the BVP and the height of the current block. The computing device may include one or more processors and memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements. The system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to encode or decode the current block using the adjusted BVP. A computer-readable medium may store instructions that, when executed, cause the performance of the described methods, additional operations, and / or include additional elements.
[0305] A computing device may perform a method including multiple operations. The computing device may determine, based on a block vector predictor (BVP) of the reference block, that the reference block overlaps with a current block of a video frame. Based on the determination, the computing device may calculate a horizontal distance from the position of the reference block to a vertical boundary of an intra block copy (IBC) reference area of the current block and a vertical distance from the position of the reference block to a horizontal boundary of the IBC reference area of the current block. The computing device may replace the BVP with an adjusted BVP based on a comparison of the horizontal and vertical distances. The computing device may also perform one or more additional operations. The width of the current block may be cbWidth, and the height of the current block may be cbHeight. Replacing the BVP with the adjusted BVP may include setting a horizontal component of the adjusted BVP equal to −cbWidth based on the horizontal distance being less than the vertical distance, and setting a vertical component of the adjusted BVP equal to the vertical component of the BVP. Replacing the BVP with the adjusted BVP may include setting a horizontal component of the adjusted BVP equal to the horizontal component of the BVP based on the horizontal distance being greater than the vertical distance, and setting a vertical component of the adjusted BVP equal to -cbHeight. Replacing the BVP with the adjusted BVP may include setting a horizontal component of the adjusted BVP equal to -cbWidth based on the horizontal distance being equal to the first vertical distance, where cbWidth is the width of the current block, and setting the vertical component of the adjusted BVP equal to -cbHeight. The computing device may calculate a second horizontal distance from the position of the current block to a second vertical boundary of the IBC reference region, the second vertical boundary being a left-most vertical boundary of the IBC reference region, and a second vertical distance from the position of the current block to the second horizontal boundary of the IBC reference region, the second horizontal boundary being a topmost horizontal boundary of the IBC reference region.Replacing the BVP with the adjusted BVP may include setting a horizontal component of the adjusted BVP equal to -cbWidth and setting a vertical component of the adjusted BVP equal to the vertical component of the BVP based on the second horizontal distance being greater than the width of the current block and the second vertical distance being less than the height of the current block. Replacing the BVP with the adjusted BVP may include setting a horizontal component of the adjusted BVP equal to the horizontal component of the BVP and setting the vertical component of the adjusted BVP equal to -cbHeight based on the second horizontal distance being less than the width of the current block and the second vertical distance being greater than the height of the current block. A computing device may include one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements. A system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to encode or decode the current block using the adjusted BVP. The computer-readable medium may store instructions that, when executed, cause the performance of the described methods, additional actions, and / or include additional elements.
[0306] A computing device may perform a method including multiple operations. The computing device may determine, based on a block vector predictor (BVP) of the reference block, that the reference block overlaps with a current block of a video frame. Based on the determination, the computing device may calculate a first horizontal distance from a position of the current block to a first vertical boundary of an intra block copy (IBC) reference region of the current block and a first vertical distance from the position of the current block to a first horizontal boundary of the IBC reference region of the current block. The computing device may replace the BVP with an adjusted BVP based on the first horizontal distance and the first vertical distance. The computing device may also perform one or more additional operations. The width of the current block may be cbWidth, and the height of the current block may be cbHeight. Replacing the BVP with the adjusted BVP may include setting a horizontal component of the adjusted BVP equal to −cbWidth and setting a vertical component of the adjusted BVP equal to the vertical component of the BVP based on the first horizontal distance being greater than the width of the current block and the first vertical distance being less than the height of the current block. Replacing the BVP with the adjusted BVP may include setting a horizontal component of the adjusted BVP equal to the horizontal component of the BVP and setting a vertical component of the adjusted BVP equal to -cbHeight based on the first horizontal distance being less than the width of the current block and the first vertical distance being greater than the height of the current block. The computing device may calculate a second horizontal distance from the position of the reference block to a second vertical boundary of the IBC reference area of the current block and a second vertical distance from the position of the reference block to a second horizontal boundary of the IBC reference area of the current block based on the first horizontal distance being greater than the width of the current block and the first vertical distance being greater than the height of the current block.Replacing the BVP with the adjusted BVP may include setting a horizontal component of the adjusted BVP equal to -cbWidth and setting a vertical component of the adjusted BVP equal to the vertical component of the BVP based on the second horizontal distance being less than the second vertical distance. Replacing the BVP with the adjusted BVP may include setting a horizontal component of the adjusted BVP equal to the horizontal component of the BVP and setting a vertical component of the adjusted BVP equal to -cbHeight based on the second horizontal distance being greater than the second vertical distance. Replacing the BVP with the adjusted BVP may include setting a horizontal component of the adjusted BVP equal to -cbWidth and setting a vertical component of the adjusted BVP equal to -cbHeight based on the second horizontal distance being equal to the second vertical distance. The computing device may include one or more processors and memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements. The system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to encode or decode the current block using the adjusted BVP. A computer-readable medium may store instructions that, when executed, cause the performance of the described methods, additional operations, and / or include additional elements.
[0307] A computing device may perform a method including multiple operations. The computing device may determine, based on a block vector predictor (BVP) of the reference block, that the reference block overlaps with a current block of a video frame. Based on the determination, the computing device may calculate a first horizontal distance from the location of the reference block to a first vertical boundary of an intra block copy (IBC) reference area of the current block and a first vertical distance from the location of the reference block to a first horizontal boundary of the IBC reference area of the current block. The computing device may also replace the BVP with an adjusted BVP based on a comparison of the first horizontal distance and the first vertical distance. The computing device may also perform one or more additional operations. The width of the current block may be cbWidth, and the height of the current block may be cbHeight. Determining that the reference block overlaps with the current block may include determining that a sum of a horizontal component of the BVP and the width of the current block is greater than zero and a sum of a vertical component of the BVP and the height of the current block is greater than zero. Replacing the BVP with the adjusted BVP may be further based on the following: a second horizontal distance from the position of the current block to a second vertical boundary of the IBC reference region of the current block is greater than the width of the current block; and a second vertical distance from the position of the current block to a second horizontal boundary of the IBC reference region of the current block is greater than the height of the current block. The second vertical boundary of the IBC reference region may be the leftmost vertical boundary of the IBC reference region. The second horizontal boundary of the IBC reference region may be the topmost horizontal boundary of the IBC reference region. Replacing the BVP with the adjusted BVP may include setting a horizontal component of the adjusted BVP equal to minus cbWidth based on the first horizontal distance being less than the first vertical distance, and setting a vertical component of the adjusted BVP equal to the vertical component of the BVP.Replacing the BVP with the adjusted BVP may include setting a horizontal component of the adjusted BVP equal to the horizontal component of the BVP based on the first horizontal distance being greater than the first vertical distance, and setting a vertical component of the adjusted BVP equal to -cbHeight. Replacing the BVP with the adjusted BVP may include setting a horizontal component of the adjusted BVP equal to -cbWidth and setting a vertical component of the adjusted BVP equal to -cbHeight based on the first horizontal distance being equal to the first vertical distance. A computing device may include one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements. A system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to encode or decode a current block using the adjusted BVP. The computer-readable medium may store instructions that, when executed, cause the performance of the described methods, additional actions, and / or include additional elements.
[0308] One or more embodiments herein may be described as a process, which may be depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, and / or a block diagram. A flowchart may describe operations as a sequential process, but one or more operations may be performed in parallel or simultaneously. The order of operations shown may be rearranged. A process may terminate when its operations are completed, but may have additional steps not shown in the figures. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to a calling function or main function.
[0309] The operations described herein may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, program code or code segments (e.g., computer program product) to perform the necessary tasks may be stored on a computer-readable or machine-readable medium. A processor may perform the necessary tasks. Features of the present disclosure may be implemented in hardware using, for example, hardware components such as application-specific integrated circuits (ASICs) and gate arrays. Implementation of hardware state machines to perform the functions described herein will also be apparent to those skilled in the art.
[0310] One or more features described herein may be implemented in computer-usable data and / or computer-executable instructions, such as one or more program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types when executed by a processor or data processing device within a computer. Computer-executable instructions may be stored on one or more computer-readable media, such as hard disks, optical disks, removable storage media, solid-state memory, RAM, etc. The functionality of the program modules may be combined or distributed as desired. Functionality may be implemented in whole or in part in firmware or hardware equivalents, such as integrated circuits, field programmable gate arrays (FPGAs), etc. Particular data structures may be used to more efficiently implement one or more features described herein, and such data structures are contemplated within the scope of the computer-executable instructions and computer-usable data described herein. Computer-readable media may include, but are not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or carrying instructions and / or data. The computer-readable medium may include non-transitory media on which data may be stored and which do not include carrier waves and / or transitory electronic signals propagated via wireless or wired connections. Examples of non-transitory media include, but are not limited to, magnetic disks or tapes, optical storage media such as compact disks (CDs) or digital versatile disks (DVDs), flash memory, memory or memory devices. The computer-readable medium may store code and / or machine-executable instructions, which may represent procedures, functions, subprograms, programs, routines, subroutines, modules, software packages, classes, or any combination of instructions, data structures, or program statements.A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.
[0311] A non-transitory tangible computer-readable medium may include instructions executable by one or more processors configured to cause the operations described herein. An article of manufacture may include a non-transitory tangible computer-readable machine-accessible medium encoded with instructions for enabling programmable hardware to cause a device (e.g., an encoder, decoder, transmitter, receiver, etc.) to perform the operations described herein. One or more units, such as in a device or system, may include one or more processors, memory, interfaces, and / or the like.
[0312] Communications described herein may be determined, generated, sent, and / or received using any amount of messages, information elements, fields, parameters, values, indications, information, bits, and / or the like. While one or more embodiments may be described herein using any of the terms / phrases message, information element, field, parameter, value, indication, information, bit, and / or the like, those skilled in the art will understand that such communications may be implemented using any one or more of these terms, including other such terms. For example, one or more parameters, fields, and / or information elements (IEs) may include one or more information objects, values, and / or any other information. An information object may include one or more other objects. At least some (or all) parameters, fields, IEs, and / or the like may be used and may be interchangeable depending on the context. Where meanings or definitions are given, such meanings or definitions are controlling.
[0313] One or more elements of the embodiments described herein may be implemented as a module. A module may be an element that performs a defined function and / or has a defined interface to other elements. A module may be implemented in hardware, software combined with hardware, firmware, wetware (e.g., hardware with biological components), or a combination thereof, all of which may be behaviorally equivalent. For example, a module may be implemented as a software routine written in a computer language configured to run on a hardware machine (e.g., C, C++, Fortran, Java, Basic, Matlab, etc.) or Simulink, Stateflow, GNU Octave, or LabVIEW MathScript. Additionally or alternatively, it may be possible to implement a module using physical hardware incorporating discrete or programmable analog, digital, and / or quantum hardware. Examples of programmable hardware include computers, microcontrollers, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and / or complex programmable logic devices (CPLDs). Computers, microcontrollers, and / or microprocessors may be programmed using languages such as assembly, C, C++, etc. FPGAs, ASICs, and CPLDs are often programmed using hardware description languages (HDLs) such as Verilog or Verilog Hardware Description Language (VHDL), which allow for the configuration of connections between the less functional internal hardware modules of the programmable device. The techniques described above can be used in combination to achieve a functionally modular result.
[0314] One or more of the operations described herein may be conditional. For example, one or more operations may be performed if certain criteria are met, such as by a computing device, a communications device, an encoder, a decoder, a network, a combination of the above, and / or the like. Exemplary criteria may be based on one or more conditions, such as device configuration, traffic load, initial system setup, packet size, traffic characteristics, a combination of the above, and the like. If one or more criteria are met, various embodiments may be used. It may be possible to implement any part of the embodiments described herein in any order and based on any condition.
[0315] Although embodiments are described above, the features and / or steps of these embodiments may be combined, divided, omitted, rearranged, revised, and / or extended in any desired manner. Various changes, modifications, and improvements will readily occur to those skilled in the art. Such changes, modifications, and improvements, although not expressly described herein, are intended to be part of this specification and are intended to be within the spirit and scope of the description herein. Accordingly, the foregoing description is illustrative only and not limiting.
Claims
1. determining, by a computing device, one or more candidate block vector predictors (BVPs), each of the one or more candidate BVPs comprising: the horizontal component, determined based on the width of the current block in the video frame, or a vertical component determined based on a height of the current block; determining a block vector (BV) for the current block based on a list of candidate BVPs modified to include at least one candidate BVP from the one or more candidate BVPs that indicates a displacement from the current block to a location within an intra block copy (IBC) reference area; encoding the current block using at least one of the candidate BVPs.
2. The at least one candidate BVP is: the horizontal distance as the sum of the horizontal component of a BVP in the list of candidate BVPs and the width of the current block is greater than zero; and 2. The method of claim 1, wherein a selection is made from among the one or more candidate BVPs based on a vertical distance as a sum of a vertical component of the BVP in the list of candidate BVPs and the height of the current block being greater than zero.
3. the width of the current block is cbWidth, the height of the current block is cbHeight, Among the at least one candidate BVP, the horizontal component of the candidate BVP is equal to −cbWidth; and The method of claim 1 , wherein the vertical component of the candidate BVP is equal to −cbHeight.
4. the width of the current block is cbWidth, Among the at least one candidate BVP, the horizontal component of the candidate BVP is equal to −cbWidth; and The method of claim 1 , wherein the vertical component of the candidate BVP is equal to the vertical component of a BVP in the list of candidate BVPs.
5. the height of the current block is cbHeight, Among the at least one candidate BVP, the vertical component of the candidate BVP is equal to −cbHeight; The method of claim 1 , wherein the horizontal component of the candidate BVP is equal to the horizontal component of a BVP in the list of candidate BVPs.
6. The at least one candidate BVP is: determining a respective block vector difference (BVD) for each of the one or more candidate BVPs; and 2. The method of claim 1, wherein the at least one candidate BVP is selected from among the one or more candidate BVPs based on the BVD of the at least one candidate BVP being smallest among the respective BVDs.
7. The method of claim 1 , wherein the determining the one or more candidate BVPs is based on determining that a BVP in the list of candidate BVPs is outside the IBC reference region.
8. the width of the current block is cbWidth, the height of the current block is cbHeight, and determining the one or more candidate BVPs includes: The horizontal component of the BVP is greater than -cbWidth, and The method of claim 1 , based on determining that the vertical component of the BVP is greater than −cbHeight.
9. The method of claim 1 , wherein the computing device includes an encoder, and the method further includes transmitting the representation of the at least one candidate BVP to a decoder.
10. 2. The method of claim 1, wherein the list of candidate BVPs is modified by replacing at least one BVP in the list of candidate BVPs with the at least one candidate BVP.
11. determining a block vector difference (BVD) based on the BV and the at least one candidate BVP; 10. The method of claim 1, further comprising: transmitting the representation of the BVD to a decoder.
12. receiving an indication of a block vector difference (BVD); 2. The method of claim 1, further comprising: determining the BV based on the BVD and a candidate BVP of the at least one candidate BVP.
13. 1. A computing device comprising: one or more processors; a memory storing instructions that, when executed by said one or more processors, cause said computing device to perform the method of any one of claims 1 to 12.
14. a first computing device configured to perform the method of any one of claims 1 to 12; a second computing device configured to encode the current block using the at least one candidate BVP.
15. A computer readable medium storing instructions that, when executed, cause the performance of the method of any one of claims 1 to 12.
16. By a computing device, the horizontal distance as the sum of the horizontal component of the block vector predictor (BVP) and the width of the current block of the video frame; and calculating a vertical distance as the sum of a vertical component of the BVP and a height of the current block, the BVP indicating the position of a reference block relative to the position of the current block; determining that the horizontal distance is greater than zero and that the vertical distance is greater than zero; replacing the BVP with an adjusted BVP based on the determination and based on a comparison of the horizontal distance and the vertical distance; the horizontal component of the adjusted BVP is based on the width of the current block, or a vertical component of the adjusted BVP being based on the height of the current block; encoding the current block using at least one of the adjusted BVPs.
17. the width of the current block is cbWidth, and the method comprises: Based on the horizontal distance being less than the vertical distance, the horizontal component of the adjusted BVP equal to −cbWidth; and 17. The method of claim 16, further comprising setting the vertical component of the adjusted BVP equal to the vertical component of the BVP.
18. the height of the current block is cbHeight, and the method comprises: Based on the horizontal distance being greater than the vertical distance, the horizontal component of the adjusted BVP equal to the horizontal component of the BVP; and 17. The method of claim 16, further comprising setting the vertical component of the adjusted BVP equal to -cbHeight.
19. the width of the current block is cbWidth, the height of the current block is cbHeight, and the method comprises: Based on the horizontal distance being equal to the vertical distance, the horizontal component of the adjusted BVP equal to −cbWidth; and 17. The method of claim 16, further comprising setting the vertical component of the adjusted BVP equal to -cbHeight.
20. said replacing said BVP with said adjusted BVP; a second horizontal distance from the position of the current block to a leftmost vertical boundary of an interblock copy (IBC) reference area is greater than the width of the current block; or a second vertical distance from the position of the current block to a top horizontal boundary of the IBC reference area of the current block is greater than the height of the current block.
21. 17. The method of claim 16, further comprising receiving an indication of a block vector difference (BVD).
22. 17. The method of claim 16, further comprising determining a block vector (BV) of the current block as a combination of the adjusted BVP and a block vector difference (BVD).
23. 17. The method of claim 16, further comprising determining a block vector difference (BVD) as a combination of the block vector (BV) of the current block and the adjusted BVP.
24. 17. The method of claim 16, further comprising transmitting an indication of a block vector difference (BVD).
25. 17. The method of claim 16, further comprising adding the adjusted BVP to a list of candidate BVPs associated with advanced motion vector prediction (AMVP) or associated with a merge mode.
26. The method of claim 16 , further comprising using the adjusted BVP to encode the current block.
27. The method of claim 16, further comprising determining one or more candidate BVPs based on determining that a BVP in the list of candidate BVPs is outside an inter-block copy (IBC) reference region.
28. 1. A computing device comprising: one or more processors; a memory storing instructions that, when executed by said one or more processors, cause said computing device to perform a method according to any one of claims 16 to 27.
29. a first computing device configured to perform the method of any one of claims 16 to 27; a second computing device configured to encode the current block based on the adjusted BVP.
30. A computer readable medium storing instructions that, when executed, cause the performance of the method of any one of claims 16 to 27.
Citation Information
Patent Citations
Block Vector Prediction in Video and Image Encoding / Decoding
JP2017510115A
Intra-block copy block vector signaling for video coding
JP2017519447A
Intra-block copying enhancements for HEVC in-range-extension (REXT)
WO2015023689A2