Improvements to IBC signaling

Abstract

Various embodiments described herein include methods and systems for coding video. In one embodiment, the method comprises receiving a video bitstream having a plurality of blocks, including a current block. When one or more characteristics of the current block satisfy one or more criteria, the method comprises parsing one or more indicators indicating IBC information from the video bitstream; identifying a prediction block for the current block based on the IBC information; and reconstructing the current block using the prediction block. When one or more characteristics of the current block do not satisfy one or more criteria, the method comprises decoding the current block without parsing one or more indicators indicating IBC information.

Description

Technology Field

[0001] <Related Applications>

[0002] This application is a continuation of U.S. Patent Application No. 19 / 093,050 filed on March 27, 2025, which claims priority to U.S. Provisional Patent Application No. 63 / 572,273, filed on March 31, 2024, titled “Improvements on IBC signaling”, each of which is incorporated herein by reference in its entirety.

[0003] Technology Field

[0004] The disclosed embodiments generally relate to video coding, including but not limited to systems and methods for IBC or Intra-BC (intra block copy). Background Technology

[0005] Digital video is supported by various electronic devices, such as digital televisions, laptops or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video teleconferencing devices, and video streaming devices. These electronic devices transmit and receive digital video data over communication networks or communicate in other ways, and / or store digital video data on storage devices. Due to the limited bandwidth capacity of communication networks and the limited memory resources of storage devices, video coding may be used to compress video data according to one or more video coding standards before it is transmitted or stored. Such video coding may be performed by hardware and / or software on servers providing cloud services or on electronic / client devices.

[0006] Video coding generally utilizes prediction methods (e.g., inter-prediction, intra-prediction, etc.) that leverage the inherent redundancy of video data. The goal of video coding is to compress video data using lower bit rates while avoiding or minimizing degradation in video quality. Numerous video codec standards have been developed. For example, High-Efficiency Video Coding (HEVC / H.265) is a video compression standard designed as part of the MPEG-H project. The ITU-T and ISO / IEC released the HEVC / H.265 standards in 2013 (Version 1), 2014 (Version 2), 2015 (Version 3), and 2016 (Version 4). Versatile Video Coding (VVC / H.266) is a video compression standard intended as a successor to HEVC. ITU-T and ISO / IEC released the VVC / H.266 standard in 2020 (Version 1) and 2022 (Version 2). AOMedia Video 1 (AV1) is an open video coding format designed as an alternative to HEVC. As of January 8, 2019, verified version 1.0.0 with Errata 1 of its specification was released.

[0007] The present disclosure describes a set of methods for video (image) compression that, among other things, relate more specifically to the conditional signaling of indicators associated with intra-block copy information. For example, indicators associated with intra-block copy information may be conditionally signaled based on one or more characteristics of the current block (such as the current block size, superblock size, and corresponding search area). By conditionally signaling indicators associated with intra-block copy information, compression efficiency may be increased when it is determined that a smaller local search area is suitable for finding the predicted block. When it is determined that a smaller local search area is not suitable for finding the predicted block, signaling overhead may be reduced by not signaling the block copy information, while other prediction methods are used to maintain coding accuracy.

[0008] According to some embodiments, a method for video decoding is provided. The method comprises: (i) receiving a video bitstream having a plurality of blocks, including a current block; (ii) when one or more characteristics of the current block satisfy one or more criteria, (a) parsing one or more indicators indicating IBC information from the video bitstream; (b) identifying a prediction block for the current block based on the IBC information; and (c) reconstructing the current block using the prediction block; and (iii) when one or more characteristics of the current block do not satisfy one or more criteria, decoding the current block without parsing one or more indicators indicating IBC information.

[0009] According to some embodiments, a method for video encoding comprises: (i) receiving video data comprising a plurality of blocks, including a current block; (ii) when one or more characteristics of the current block satisfy one or more criteria, (a) encoding the current block using an IBC mode; and (b) signaling one or more indicators indicating IBC information for the current block in a video bitstream; and (iii) when one or more characteristics of the current block do not satisfy one or more criteria, not signaling IBC information for the current block.

[0010] According to some embodiments, a computing system, such as a streaming system, a server system, a personal computer system, or other electronic device, is provided. Such a computing system includes control circuitry and memory for storing one or more sets of instructions. One or more sets of instructions include instructions for performing any of the methods described herein. In some embodiments, such a computing system includes an encoder component and a decoder component (e.g., a transcoder).

[0011] According to some embodiments, a non-transient computer-readable storage medium is provided. Such a non-transient computer-readable storage medium stores one or more sets of instructions for execution by a computing system. Such one or more sets of instructions include instructions for performing any of the methods described herein.

[0012] Accordingly, devices and systems having methods for encoding and decoding video are disclosed. These methods, devices, and systems may complement or replace conventional methods, devices, and systems for video encoding and / or decoding.

[0013] The features and benefits described herein are not necessarily all-encompassing, and some additional features and benefits will be apparent to a person skilled in the art, in particular, by considering the drawings, specification, and claims provided in this disclosure. Furthermore, it should be noted that the language used herein has been chosen primarily for readability and educational purposes and has not been chosen to describe or limit the subject matter described herein. Brief explanation of the drawing

[0014] To enable a more detailed understanding of the present disclosure, a more specific description may be made by referring to the features of various embodiments, some of which are illustrated in the accompanying drawings. However, the accompanying drawings merely illustrate the relevant features of the present disclosure and should not be construed as limiting, as a person skilled in the art may recognize other effective features when reading the present disclosure with respect to this description. FIG. 1 is a block diagram illustrating an exemplary communication system according to some embodiments. FIG. 2a is a block diagram illustrating exemplary elements of an encoder component according to some embodiments. FIG. 2b is a block diagram illustrating exemplary elements of a decoder component according to some embodiments. FIG. 3 is a block diagram illustrating an exemplary server system according to some embodiments. FIG. 4a illustrates an exemplary intra-block copy technique according to some embodiments. FIG. 4b illustrates examples of different intra-block copy modes according to some embodiments. FIG. 5a illustrates an exemplary video decoding process according to some embodiments. FIG. 5b illustrates an exemplary video encoding process according to some embodiments. According to common practice, various features exemplified in the drawings are not necessarily drawn to scale, and similar reference numbers may be used to denote similar features throughout the specification and drawings. Specific details for implementing the invention

[0015] The present disclosure describes video / image compression techniques comprising the conditional signaling of indicators associated with intra-block copy information (e.g., IBC on / off flags, block vectors, block vector predictors, etc.) based on one or more characteristics of the current block. Searching for a predicted block within a smaller local search area can increase the coding efficiency of the intra-block copy method. However, when one or more characteristics of the current block indicate a reduced probability of finding a suitable predicted block within a smaller local search area, the encoder may discard the signaling flags and syntax associated with the intra-block copy method. If a larger global search area is available to find a predicted block, the current block may still be encoded using the intra-block copy method even when the intra-block copy flags are not signaled by the encoder. Otherwise, the intra-block copy method is not used to encode the current block. There is an advantage in conditionally signaling indicators associated with intra-block copy information based on one or more characteristics of the current block to reduce signaling overhead while preserving coding accuracy (e.g., using fewer bits when a local search area is determined to be less suitable for finding a prediction block), and thus improving coding efficiency. Using a smaller local search area to find a prediction block based on one or more characteristics of the current block improves the coding efficiency of the intra-block copy method.

[0016] Exemplary systems and devices

[0017] FIG. 1 is a block diagram illustrating a communication system (100) according to some embodiments. The communication system (100) includes a source device (102) and a plurality of electronic devices (120) (e.g., electronic device (120-1) to electronic device (120-m)) that are connected to each other so as to be communicable through one or more networks. In some embodiments, the communication system (100) is a streaming system for use with video-enabled applications, such as video conferencing applications, digital TV applications, and media storage and / or distribution applications.

[0018] The source device (102) includes a video source (104) (e.g., a camera component or media storage) and an encoder component (106). In some embodiments, the video source (104) is a digital camera (e.g., configured to generate an uncompressed video sample stream). The encoder component (106) generates one or more encoded video bitstreams from the video stream. The video stream from the video source (104) may have a high data volume compared to the encoded video bitstream (108) generated by the encoder component (106). Because the encoded video bitstream (108) has a lower data volume (less data) compared to the video stream from the video source, the encoded video bitstream (108) requires less bandwidth for transmission and less storage space for storage compared to the video stream from the video source (104). In some embodiments, the source device (102) does not include an encoder component (106) (e.g., configured to transmit uncompressed video to network(s) (110)).

[0019] One or more networks (110) represent any number of networks that transmit information between a source device (102), a server system (112), and / or electronic devices (120), including, for example, wireline (wired) and / or wireless communication networks. One or more networks (110) may exchange data on circuit-switched and / or packet-switched channels. Representative networks include telecommunication networks, local area networks, wide area networks, and / or the Internet.

[0020] One or more networks (110) include a server system (112) (e.g., a distributed / cloud computing system). In some embodiments, the server system (112) is a streaming server (e.g. configured to store and / or distribute video content, such as an encoded video stream from a source device (102)) or includes such a server system. The server system (112) includes a coder component (114) (e.g. configured to encode and / or decode video data). In some embodiments, the coder component (114) includes an encoder component and / or a decoder component. In various embodiments, the coder component (114) is instantiated as hardware, software, or a combination thereof. In some embodiments, the coder component (114) is configured to decode an encoded video bitstream (108) and to re-encode video data using different encoding standards and / or methodologies to generate encoded video data (116). In some embodiments, the server system (112) is configured to generate multiple video formats and / or encodings from an encoded video bitstream (108). In some embodiments, the server system (112) functions as a Media-Aware Network Element (MANE). For example, the server system (112) may be configured to prune the encoded video bitstream (108) to potentially customize different bitstreams for one or more of the electronic devices (120). In some embodiments, the MANE is provided separately from the server system (112).

[0021] The electronic device (120-1) includes a decoder component (122) and a display (124). In some embodiments, the decoder component (122) is configured to decode encoded video data (116) to generate an outgoing video stream that can be rendered on a display or other type of rendering device. In some embodiments, one or more of the electronic devices (120) do not include a display component (e.g., communicateably connected to an external display device and / or include media storage). In some embodiments, the electronic devices (120) are streaming clients. In some embodiments, the electronic device (120) is configured to access a server system (112) to obtain the encoded video data (116).

[0022] The source device and / or a plurality of electronic devices (120) are sometimes referred to as "terminal devices" or "user devices". In some embodiments, one or more of the source device (102) and / or electronic devices (120) are instances of a server system, a personal computer, a portable device (e.g., a smartphone, tablet, or laptop), a wearable device, a video conferencing device, and / or other types of electronic devices.

[0023] In an exemplary operation of the communication system (100), the source device (102) transmits an encoded video bitstream (108) to the server system (112). For example, the source device (102) may encode a stream of images captured by the source device. The server system (112) receives the encoded video bitstream (108) and may decode and / or encode the encoded video bitstream (108) using a coder component (114). For example, the server system (112) may apply an encoding to the video data that is more optimal for network transmission and / or storage. The server system (112) may transmit the encoded video data (116) (e.g., one or more encoded video bitstreams) to one or more of the electronic devices (120). Each electronic device (120) may decode the encoded video data (116) and optionally display the video images.

[0024] FIG. 2a is a block diagram illustrating exemplary elements of an encoder component (106) according to some embodiments. The encoder component (106) receives video data (e.g., source video sequence) from a video source (104). In some embodiments, the encoder component includes a receiver (e.g., transceiver) component configured to receive the source video sequence. In some embodiments, the encoder component (106) receives a video sequence from a remote video source (e.g., a video source that is a component of a device different from the encoder component (106)). The video source (104) may provide the source video sequence in the form of a digital video sample stream which may be of any suitable bit depth (e.g., 8-bit, 10-bit, or 12-bit), any color space (e.g., BT.601 Y CrCB, or RGB), and any suitable sampling structure (e.g., Y CrCb 4:2:0, or Y CrCb 4:4:4). In some embodiments, the video source (104) is a storage device that stores previously captured / prepared video. In some embodiments, the video source (104) is a camera that captures local image information as a video sequence. Video data may be provided as a plurality of individual images that impart motion when viewed sequentially. These images themselves may be organized as a spatial array of pixels, and each pixel may contain one or more samples depending on the sampling structure, color space, etc. used. A person skilled in the art can easily understand the relationship between pixels and samples.

[0025] The encoder component (106) is configured to code and / or compress images of a source video sequence into a coded video sequence (216) in real time or under other time constraints as required by the application. In some embodiments, the encoder component (106) is configured to perform a conversion between the source video sequence and a bitstream of visual media data (e.g., a video bitstream). Enforcing an appropriate coding rate is one function of the controller (204). In some embodiments, the controller (204) controls other function units as described below and is functionally connected to these other function units. Parameters set by the controller (204) may include rate-control-related parameters (e.g., image skips, quantizers, and / or lambda values ​​of rate-distortion optimization techniques), image size, group of pictures (GOP) layout, maximum motion vector search range, etc. A person skilled in the art can easily identify other functions of the controller (204) because they may be related to the encoder component (106) being optimized for a specific system design.

[0026] In some embodiments, the encoder component (106) is configured to operate in a coding loop. In a simplified example, the coding loop includes a source coder (202) (responsible for generating symbols, such as a symbol stream, based on the input image and reference image(s) to be coded, for example), and a (local) decoder (210). The decoder (210) reconstructs the symbols to generate sample data in a manner similar to the (remote) decoder (when compression between the symbols and the coded video bitstream is lossless). The reconstructed sample stream (sample data) is input into a reference image memory (208). Since the decoding of the symbol stream leads to bit-exact results independently of the decoder location (local or remote), the content of the reference image memory (208) is also bit exact between the local encoder and the remote encoder. In this way, the prediction part of the encoder interprets the same sample values ​​that the decoder interprets when using prediction during decoding as reference image samples.

[0027] The operation of the decoder (210) may be the same as that of a remote decoder, such as the decoder component (122) described in detail below together with FIG. 2b. However, with brief reference to FIG. 2b, since symbols are available and the encoding / decoding of symbols for a coded video sequence by the entropy coder (214) and the parser (254) may be lossless, the entropy decoding portions of the decoder component (122), including the buffer memory (252) and the parser (254), may not be fully implemented in the local decoder (210).

[0028] The decoder technology described herein may exist in a substantially identical functional form in the corresponding encoder, except for parsing / entropy decoding. For this reason, the subject matter disclosed focuses on decoder operations. Additionally, the description of encoder technology may be abbreviated as it may be the inverse of decoder technology.

[0029] As part of the operation, the source coder (202) may perform motion-compensated predictive coding, which predictively codes an input frame by referencing one or more previously coded frames from a video sequence designated as reference frames. In this way, the coding engine (212) codes the differences between pixel blocks of the reference frame(s) and pixel blocks of the input frame, which can be selected as predictive reference(s) for the input frame. The controller (204) may manage the coding operations of the source coder (202), including, for example, the settings of parameters and subgroup parameters used to encode video data.

[0030] The decoder (210) decodes the coded video data of frames that can be designated as reference frames based on symbols generated by the source coder (202). The operations of the coding engine (212) may advantageously be lossy processes. When the coded video data is decoded in a video decoder (not shown in FIG. 2a), the reconstructed video sequence may be a copy of the source video sequence having some errors. The decoder (210) may duplicate the decoding processes that can be performed by a remote video decoder of the reference frames and allow the reconstructed reference frames to be stored in the reference image memory (208). In this way, the encoder component (106) locally stores copies of the reconstructed reference frames having common content as reconstructed reference frames to be acquired by the remote video decoder (without transmission errors).

[0031] The predictor (206) can perform predictive searches for the coding engine (212). That is, for a new frame to be coded, the predictor (206) can search the reference image memory (208) for sample data (as candidate reference pixel blocks) or specific metadata such as reference image motion vectors, block shapes, etc., which can serve as appropriate predictive references for the new images. The predictor (206) can operate on a sample block-by-pixel block basis to discover appropriate predictive references. As determined by the search results obtained by the predictor (206), the input image may have predictive references derived from a number of reference images stored in the reference image memory (208).

[0032] The outputs of all the aforementioned functional units may undergo entropy coding in an entropy coder (214). The entropy coder (214) converts the symbols generated by the various functional units into a coded video sequence by losslessly compressing them according to techniques known to a person of ordinary skill in the art (e.g., Huffman coding, variable length coding, and / or arithmetic coding).

[0033] In some embodiments, the output of the entropy coder (214) is connected to a transmitter. This transmitter may be configured to buffer the coded video sequence(s) generated by the entropy coder (214) to prepare them for transmission over a communication channel (218), which may be a hardware / software link to a storage device for storing the encoded video data. The transmitter may be configured to merge the coded video data from the source coder (202) with other data to be transmitted, e.g., coded audio data and / or auxiliary data streams (sources not shown). In some embodiments, the transmitter may transmit additional data along with the encoded video. The source coder (202) may include such data as part of the coded video sequence. The additional data may include other forms of redundant data such as time / space / SNR enhancement layers, redundant images and slices, Supplementary Enhancement Information (SEI) messages, Visual Usability Information (VUI) parameter set fragments, etc.

[0034] The controller (204) can manage the operation of the encoder component (106). During coding, the controller (204) can assign a specific coded image type to each coded image, which can influence the coding techniques that can be applied to each image. For example, images may be assigned as an I image (Intra Picture), a P image (Predictive Picture), or a B image (Bi-directionally Predictive Picture). An intra image may be coded and decoded without using any other frame in the sequence as a source of prediction. Some video codecs allow different types of intra images, including, for example, Independent Decoder Refresh (IDR) images. A person skilled in the art is aware of these variations of I images and their respective applications and features, and thus, they are not repeated here. A predictive image may be coded and decoded using intra prediction or inter prediction, which uses at most one motion vector and reference index to predict the sample values ​​of each block. Bidirectional prediction images can be coded and decoded using intra prediction or inter prediction, which use up to two motion vectors and reference indices to predict sample values ​​of each block. Similarly, multiple prediction images can use more than two reference images and associated metadata for the reconstruction of a single block.

[0035] Source images are typically spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples, respectively) and can be coded on a block-by-block basis. Blocks can be predictively coded by referencing other (already coded) blocks determined by the coding assignment applied to each of the images in the blocks. For example, blocks of images I can be non-predictively coded, or they can be predictively coded by referencing already coded blocks of the same image (spatial prediction or intra prediction). Pixel blocks of images P can be non-predictively coded through temporal prediction or spatial prediction by referencing one previously coded reference image. Blocks of images B can be non-predictively coded through temporal prediction or spatial prediction by referencing one or two previously coded reference images.

[0036] Video can be captured as multiple source images (video images) in a time sequence. Intra-image prediction (often abbreviated as intra prediction) uses spatial correlations within a given image, while inter-image prediction uses (temporal or other) correlations between images. In the example, a specific image under encoding / decoding, referred to as the current image, is partitioned into blocks. When a block in the current image is similar to a reference block in a previously coded and still buffered reference image in the video, the block in the current image can be coded by a vector referred to as a motion vector. This motion vector points to the reference block in the reference image and, if multiple reference images are in use, may have a third dimension that identifies the reference image.

[0037] The encoder component (106) may perform coding operations according to a predetermined video coding technique or standard, such as any one described herein. In the operation, the encoder component (106) may perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancies in the input video sequence. Thus, the coded video data may conform to the syntax specified by the video coding technique or standard in use.

[0038] FIG. 2b is a block diagram illustrating exemplary elements of a decoder component (122) according to some embodiments. The decoder component (122) in FIG. 2b is connected to a channel (218) and a display (124). In some embodiments, the decoder component (122) includes a transmitter connected to a loop filter (256) and configured to transmit data to the display (124) (e.g., via a wired or wireless connection).

[0039] In some embodiments, the decoder component (122) includes a receiver connected to the channel (218) and configured to receive data from the channel (218) (e.g., via a wired or wireless connection). The receiver may be configured to receive one or more coded video sequences to be decoded by the decoder component (122). In some embodiments, the decoding of each coded video sequence is independent of other coded video sequences. Each coded video sequence may be received from the channel (218), which may be a hardware / software link to a storage device that stores the encoded video data. The receiver may receive the encoded video data along with other data, e.g., coded audio data and / or auxiliary data streams, which may be delivered to each of the respective user entities (not described). The receiver may separate the coded video sequence from the other data. In some embodiments, the receiver receives additional (redundant) data along with the encoded video. This additional data may be included as part of the coded video sequence(s). Additional data may be used to decode data by the decoder component (122) and / or to reconstruct the original video data more accurately. The additional data may be in the form of, for example, temporal, spatial, or SNR enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0040] According to some embodiments, the decoder component (122) includes a buffer memory (252), a parser (254) (sometimes also referred to as an entropy decoder), a scaler / inverse conversion unit (258), an intra-image prediction unit (262), a motion compensation prediction unit (260), an aggregator (268), a loop filter unit (256), a reference image memory (266), and a current image memory (264). In some embodiments, the decoder component (122) is implemented as an integrated circuit, a series of integrated circuits, and / or other electronic circuits. The decoder component (122) may be implemented at least partially in software.

[0041] A buffer memory (252) is connected between the channel (218) and the parser (254) (e.g., to prevent network jitter). In some embodiments, the buffer memory (252) is separate from the decoder component (122). In some embodiments, a separate buffer memory is provided between the output of the channel (218) and the decoder component (122). In some embodiments, in addition to the buffer memory (252) inside the decoder component (122) (e.g. configured to handle playout timing), a separate buffer memory is provided outside the decoder component (122) (e.g., to prevent network jitter). When receiving data from a storage / transmission device of sufficient bandwidth and controllability, or from an isosynchronous network, the buffer memory (252) may not be required or may be small. For use in best effort packet networks such as the Internet, a buffer memory (252) may be required, may be relatively large and / or adaptively sized, and may be implemented at least partially in an operating system or similar elements outside the decoder component (122).

[0042] The parser (254) is configured to reconstruct symbols (270) from a coded video sequence. For example, the symbols may include information used to manage the operation of a decoder component (122) and / or information for controlling a rendering device such as a display (124). The control information for the rendering device(s) may be in the form, for example, Supplementary Enhancement Information (SEI) messages or Video Usability Information (VUI) parameter set fragments (not described). The parser (254) parses (entropy-decodes) the coded video sequence. The coding of the coded video sequence may follow video coding techniques or standards and may follow principles well known to a person of ordinary skill in such techniques, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (254) can extract a set of subgroup parameters for at least one of the subgroups of pixels in a video decoder from a coded video sequence based on at least one parameter corresponding to a group. The subgroups may include Groups of Pictures (GOPs), images, tiles, slices, macroblocks, Coding Units (CUs), blocks, Transform Units (TUs), Prediction Units (PUs), etc. The parser (254) can also extract information such as transform coefficients, quantizer parameter values, motion vectors, etc. from the coded video sequence.

[0043] The reconstruction of the symbols (270) may involve a number of different units depending on the type of the coded video image or parts thereof (such as inter and intra images, inter and intra blocks), and other factors. Which units are involved and how they are involved may be controlled by subgroup control information parsed by the parser (254) from the coded video sequence. The flow of this subgroup control information between the parser (254) and the number of units below is not described for clarity.

[0044] The decoder component (122) can be conceptually subdivided into a number of functional units, and in some implementations, these units may interact closely with each other and at least partially integrate with each other. However, for clarity, the conceptual subdivision of the functional units is retained in this specification.

[0045] The scaler / inverse transform unit (258) receives quantized transform coefficients as well as control information (such as which transform to use, block size, quantization factor, and / or quantization scaling matrices) as symbol(s) (270) from the parser (254). The scaler / inverse transform unit (258) may output blocks containing sample values ​​that can be input to the aggregator (268). In some cases, the output samples of the scaler / inverse transform unit (258) relate to intra-coded blocks; that is: blocks that do not use prediction information from previously reconstructed images, but can use prediction information from previously reconstructed parts of the current image. Such prediction information may be provided by the intra-image prediction unit (262). The intra-image prediction unit (262) may generate blocks of the same size and shape as the blocks under reconstruction by using surrounding already-reconstructed information fetched from the current (partially reconstructed) image from the current image memory (264). The aggregator (268) can add the prediction information generated by the intra-image prediction unit (262) on a per-sample basis to the output sample information provided by the scaler / inverse conversion unit (258).

[0046] In other cases, the output samples of the scaler / inverse conversion unit (258) are intercoded and potentially motion-compensated, and relate to a block. In these cases, the motion compensation prediction unit (260) may access the reference image memory (266) to fetch samples used for prediction. After motion-compensating the fetched samples according to the symbols (270) for the block, these samples may be added to the output of the scaler / inverse conversion unit (258) (in this case, called residual samples or residual signals) by the aggregator (268) to generate output sample information. The addresses in the reference image memory (266) from which the motion compensation prediction unit (260) fetches prediction samples may be controlled by motion vectors. Motion vectors may be available to the motion compensation prediction unit (260) in the form of symbols (270) that may have, for example, X, Y, and reference image components. Motion compensation may also include motion vector prediction mechanisms, for example, when sub-sample precision motion vectors are in use, for interpolation of sample values ​​fetched from reference image memory (266).

[0047] The output samples of the aggregator (268) may be subject to various loop filtering techniques in the loop filter unit (256). Video compression techniques may include in-loop filter techniques that are controlled by parameters included in the coded video bitstream and made available to the loop filter unit (256) as symbols (270) from the parser (254), but which respond not only to meta-information obtained during the decoding of previous (in decoding order) parts of the coded image or coded video sequence, but also to previously reconstructed and loop-filtered sample values. The output of the loop filter unit (256) may be a sample stream that may be output to a rendering device such as a display (124) and may also be stored in a reference image memory (266) for use in future inter-image prediction.

[0048] Certain coded images, once reconstructed, can be used as reference images for future predictions. Once a coded image is reconstructed and the coded image is identified as a reference image (e.g., by a parser (254)), the current reference image may become part of the reference image memory (266), and a new current image memory may be reallocated before starting the reconstruction of the next coded image.

[0049] The decoder component (122) may perform decoding operations according to a predetermined video compression technology that may be documented in a standard, such as any of the standards described herein. The coded video sequence may conform to the syntax specified by the video compression technology or standard used in the sense of adhering to the syntax of the video compression technology or standard, as specified in the video compression technology document or standard and specifically in the profile document therein. Additionally, for compliance with some video compression technologies or standards, the complexity of the coded video sequence may be within boundaries defined by the levels of the video compression technology or standard. In some cases, the levels limit the maximum image size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference image size, etc. In some cases, the limits set by the levels may be further limited through HRD (Hypothetical Reference Decoder) specifications and metadata for HRD buffer management signaled in the coded video sequence.

[0050] FIG. 3 is a block diagram illustrating a server system (112) according to some embodiments. The server system (112) includes a control circuit (302), one or more network interfaces (304), memory (314), a user interface (306), and one or more communication buses (312) for interconnecting these components. In some embodiments, the control circuit (302) includes one or more processors (e.g., CPU, GPU, and / or DPU). In some embodiments, the control circuit includes field-programmable gate array(s), hardware accelerators, and / or integrated circuit(s) (e.g., application-specific integrated circuits).

[0051] Network interface(s) (304) may be configured to interface with one or more communication networks (e.g., wireless, wireline, and / or optical networks). The communication networks may be local, wide-area, metropolitan, automotive and industrial, real-time, delay-tolerant, etc. Examples of communication networks include cellular networks including Ethernet, wireless LANs, GSM, 3G, 4G, 5G, LTE, etc., TV wireline or wireless wide-area digital networks including cable TV, satellite TV, and terrestrial broadcast TV, and short-range networks such as automotive and industrial networks including CANBus, etc. Such communication may be unidirectional, receive-only (e.g., broadcast TV), unidirectional transmit-only (e.g., CANbus to specific CANbus devices), or bidirectional (e.g., to other computer systems using local or wide-area digital networks). Such communication may include communication to one or more cloud computing networks.

[0052] The user interface (306) includes one or more output devices (308) and / or one or more input devices (310). The input device(s) (310) may include one or more of a keyboard, mouse, trackpad, touchscreen, data-glove, joystick, microphone, scanner, camera, etc. The output device(s) (308) may include one or more of an audio output device (e.g., speaker), a visual output device (e.g., display or monitor), etc.

[0053] The memory (314) may include high-speed random access memory (such as DRAM, SRAM, DDR RAM, and / or other random access solid-state memory devices) and / or non-volatile memory (such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, and / or other non-volatile solid-state storage devices). The memory (314) optionally includes one or more storage devices located remotely from the control circuit (302). The memory (314), or alternatively, the non-volatile solid-state memory device(s) within the memory (314), includes a non-transient computer-readable storage medium. In some embodiments, the memory (314), or the non-transient computer-readable storage medium of the memory (314), stores the following programs, modules, instructions, and data structures, or a subset or superset thereof:

[0054] An operating system (316) including procedures for handling various basic system services and performing hardware-dependent tasks;

[0055] A network communication module (318) used to connect the server system (112) to other computing devices through one or more network interfaces (304) (e.g., through wired and / or wireless connections);

[0056] A coding module (320) for performing various functions with respect to encoding and / or decoding data, such as video data. In some embodiments, the coding module (320) is an instance of the coder component (114). The coding module (320) includes, but is not limited to, one or more of the following:

[0057] A decoding module (322) for performing various functions regarding decoding encoded data, such as those previously described regarding the decoder component (122); and

[0058] An encoding module (340) for performing various functions with respect to encoding data, such as those previously described with respect to the encoder component (106); and

[0059] An image memory (352) for storing images and image data, for example, for use with a coding module (320). In some embodiments, the image memory (352) includes one or more of a reference image memory (208), a buffer memory (252), a current image memory (264), and a reference image memory (266).

[0060] In some embodiments, the decoding module (322) includes a parsing module (324) (e.g., configured to perform various functions previously described with respect to the parser (254)), a conversion module (326) (e.g., configured to perform various functions previously described with respect to the scaler / inverse conversion unit (258)), a prediction module (328) (e.g., configured to perform various functions previously described with respect to the motion compensation prediction unit (260) and / or intra-image prediction unit (262)), and a filter module (330) (e.g., configured to perform various functions previously described with respect to the loop filter (256)).

[0061] In some embodiments, the encoding module (340) includes a coding module (342) (e.g., configured to perform various functions previously described with respect to the source coder (202) and / or coding engine (212)) and a prediction module (344) (e.g., configured to perform various functions previously described with respect to the predictor (206). In some embodiments, the decoding module (322) and / or the encoding module (340) include a subset of the modules illustrated in FIG. 3. For example, a shared prediction module is used by both the decoding module (322) and the encoding module (340).

[0062] Each of the modules identified above stored in memory (314) corresponds to a set of instructions for performing the functions described herein. The modules identified above (e.g., sets of instructions) do not need to be implemented as separate software programs, procedures, or modules, and thus various subsets of these modules may be combined or otherwise rearranged in various embodiments. For example, the coding module (320) may optionally not include separate decoding and encoding modules, but rather use the same set of modules to perform sets of functions of both. In some embodiments, memory (314) stores a subset of the modules and data structures identified above. In some embodiments, memory (314) stores additional modules and data structures not described above.

[0063] FIG. 3 illustrates a server system (112) according to some embodiments, but FIG. 3 is intended more as a functional description of various features that may exist in one or more server systems rather than as a structural schematic diagram of the embodiments described herein. In practice, items shown separately may be combined and some items may be separated. For example, some items shown separately in FIG. 3 may be implemented on single servers, and single items may be implemented by one or more servers. The actual number of servers used to implement the server system (112), and how features are allocated among them, will vary from one implementation to another and, optionally, depend in part on the amount of data traffic handled by the server system during average usage periods as well as during peak usage periods.

[0064] Exemplary coding techniques

[0065] The coding processes and techniques described below may be performed on the devices and systems described above (e.g., source device (102), server system (112), and / or electronic device (120)). According to some embodiments, exemplary methods for conditional signaling indicators associated with intra-block copy information are described below.

[0066] The intra-block copy method is a method that uses block vectors to identify predicted blocks for a current block. Block vectors are used to identify other blocks within the same image of the current block that may or may not be adjacent to the current block. Block vectors can be explicitly signaled or implicitly derived. When block vectors are explicitly signaled, this is generally referred to as the intra-block copy method. When block vectors are implicitly derived, such as by comparing the template area (a group of neighboring reconstruction samples located adjacent to the block) between the current block and candidate predicted blocks, this is generally referred to as the intra-template matching method.

[0067] In some embodiments, a “local search region” refers to a reconstructed sample region neighboring the current block that can be used to identify a prediction block. For example, an area covered by a block region having a given size (e.g., 64x64) associated with the current block may be a local search region. A global search region may refer to a reconstructed sample region further from the current block that can be used to identify a prediction block, as used herein. For example, an area extending beyond an adjacent block region having a given size (e.g., any reconstructed sample region extending beyond two previously coded superblocks (or referred to as coding tree blocks).

[0068] "Superblock size" may refer to the largest coding block size applied to code an image / video frame or video sequence. Block size or area size may refer to the block / area width and / or height, the block / area area size, the number of samples in the block / area, the maximum (or minimum) between the block / area width and height, and / or the block / area aspect ratio.

[0069] FIG. 4a illustrates an example of an intra-block copy technique. In this example, the intra-block copy technique includes identifying a predicted block (404) within the same image (400) as the current block (402) through a predicted block vector (408). FIG. 4a illustrates an example where the predicted block (404) is not adjacent to the current block (402). In some embodiments, the predicted block (404) is adjacent to the current block (402). In some embodiments, the predicted block vector (408) is selected from a list of candidate block vectors. The list of candidate block vectors may be populated with block vectors from a block vector bank and / or block vectors used in neighboring blocks. A block vector predictor index may be signaled to indicate which candidate block vector in the candidate list is used to predict the block vector for the current block (402).

[0070] In some embodiments, the block vector difference (BVD) (410) represents the difference between the actual block vector (406) and the predicted block vector, which is a vector between corresponding parts of the current block (e.g., current block (402)) and the predicted block (e.g., predicted block (404)). The BVD (410) depicted in FIG. 4a has components along both horizontal and vertical dimensions, but the BVD may extend along a single dimension or span two or more dimensions. In some embodiments, the BVD (410) may include information representing the magnitude of the block vector difference and / or the direction of the block vector difference, one or both of which may be signaled as intra-block copy information or syntax in the bitstream.

[0071] FIG. 4b illustrates examples of different intra-block copy modes according to some embodiments. FIG. 4b illustrates an exemplary image (420) having a number of samples. For example, dark gray samples in the global search area (422) and cross-hatched samples in the local search area (424) both correspond to reconstructed samples in the image (420). In some embodiments, a prediction block (e.g., similar to the prediction block (404) in FIG. 4a) for a current block (426) in the superblock (428) (e.g., similar to the current block (402) in FIG. 4a) is searched among the reconstructed samples. The superblock (428) is schematically illustrated in FIG. 4b, and the number of dashed squares does not correspond to the actual number of samples. The superblock (428) may be, for example, 128x128, 64x64, or other sizes. The superblock (428) can be partitioned into smaller coding blocks, for example, the current block (426) and the coding block (430).

[0072] In some embodiments, searching for a prediction block among coding blocks containing all available reconstructed samples requires more processing time than searching only the local search area. In some embodiments, the reconstructed samples closest to the current block (426) contain information content more similar to the current block (426) (e.g., in scenarios where the image (420) contains more localized information, such as localized textures), and searching for a prediction block in the local search area (424) may be more time-efficient and may provide a more suitable prediction block. For example, the characteristics of the current block (426) may reflect the information content of the current block. If the current block contains more textured information, the current block may be smaller (e.g., 8x8, 16x16, 32x32, etc.). If the information content within the current block is smoother (e.g., not much texture), the current block may be larger (e.g., 64x64, etc.).

[0073] In some embodiments, the first intra-block copy mode involves searching for a predicted block in a more limited, smaller local area (424) instead of a larger global area (e.g., the sum of areas (422 and 424), or area (422) only). In some embodiments, the second intra-block copy mode involves searching for a predicted block in a larger global area that includes at least area (422). In some embodiments, the second intra-block copy mode (e.g., searching to a larger global area) is the default intra-block copy mode when the first intra-block copy mode is not available (e.g., searching for a predicted block is not done within the smaller local area (424), but the intra-block copy method is still used to encode the current block).

[0074] In some embodiments, one or more flags and / or syntax indicating the use of the intra-block copy method may be signaled as a bitstream by the encoder. For example, an intra-block copy on / off flag may be signaled to indicate whether the intra-block copy method is used for a coding block. Intra-block copy related parameters, such as a block vector (e.g., 406) and a block vector predictor index (e.g., indicating which block vector from a list of candidate block vectors will be used), may also be signaled. In some embodiments, whether the intra-block copy on flag is signaled (e.g., to indicate the use of the intra-block copy method) may depend on one or more of the coding block sizes, the superblock size, the selection of the search area, and / or the relative position of the current block relative to an associated block of a given size.

[0075] In some embodiments, block size threshold s 2 is the given superblock size (e.g., s 0 Indicated as) and the size of the local navigation area (e.g., s 1 It is derived based on (indicated as ). In some embodiments, the local search area s 1 The size of may vary depending on the position of the current block. For example, for the current block at the edge of the image (420), the local search area s 1 It may be smaller than the current block inside the image (420). In some embodiments, whether the first intra-block copy mode is used depends on the size of the current block. In some embodiments, the current block is a block size threshold s 2When less than or equal to the above, flags and syntax indicating the use of the intra-block copy method are signaled to indicate that the intra-block copy method is used to encode the current block. In some embodiments, the syntax further indicates whether the predicted block must be searched within a local search area (e.g., area (424)), or within both the local search area (area (424)) and a larger global search area (e.g., area (422)).

[0076] In some embodiments, the current block is a block size threshold s 2 When greater than, flags and syntax are not signaled. For example, in these situations, the current block is the block size threshold s 2 When larger, there may not be a sufficient number of reconstructed samples and / or potential prediction blocks, so that the probability of finding a good prediction block may be lower. Consequently, if an intra-block copy is used to encode the current block in these situations, the local search area (424) is then disabled and searching for prediction blocks is performed only in the global search area. In some embodiments, an intra-block copy is not used to encode the current block, and other prediction modes (e.g., angle prediction modes and / or non-angle prediction modes) are used instead.

[0077] In some embodiments, when intra-block copy flags and / or intra-block copy syntax are not signaled, a second intra-copy block mode (e.g., the prediction block is searched within the global search area (422)) is used by default. In some embodiments, when intra-block copy flags and / or intra-block copy syntax are not signaled, the intra-copy block method is not used for the current block (e.g., the current block uses angled or non-angled intra-prediction modes).

[0078] In some embodiments, block size threshold s 2 is the local navigation area s 1 Equivalent to, or the block size threshold s 2 Is s 1 / N and, here N is a power of 2, for example, 2, 4, 8, 16, etc. As an example, s 1 It can be 64x64, and the block size threshold s 2 64x64 / 4 corresponding to 32x32 (e.g., N ... can be derived as 4). When the current block size is 32x32 or less, flags and syntax indicating the use of the intra-block copy method (e.g., the use of the first intra-block copy mode or the second intra-block copy mode) are signaled. In some embodiments, when the intra-block copy flags and / or intra-block copy syntax are not signaled, the second intra-block copy block mode (e.g., the prediction block is searched within the global search area (422)) is used by default. In some embodiments, when the intra-block copy flags and / or intra-block copy syntax are not signaled, the intra-block copy method is not used for the current block (e.g., the current block uses angled or non-angled intra-predicted modes).

[0079] In some embodiments, block size threshold s 2 is the size of a superblock s 0 and local navigation area s 1 The smaller of the two (e.g., min( s 0 , s 1It is identical to (also indicated as). In some embodiments, the block size threshold s 2 is min( s 0 , s 1 ) / N It is identical to, and here N is a power of 2, for example, 2, 4, 8, 16, etc. As an example, the superblock size s 0 It can be 32x32, and the local navigation area s 1 It can be 64x64, and the block size threshold s 2 is, in this example, 16x16, min( s 0 , s 1 ) / N It can be derived as. In these scenarios, when the coding block size (e.g., the size of the region (426)) is 16x16 or less, the encoder signals the intra-block copy-on flag and intra-block copy syntax (e.g., block vector, block vector predictor index, and / or block vector difference). In some embodiments, when the intra-block copy flags and / or intra-block copy syntax are not signaled, a second intra-copy block mode (e.g., the predictor block is searched within the global search region (422)) is used by default. In some embodiments, when the flags and / or syntax are not signaled, the intra-copy block method is not used (e.g., the current block uses angled or non-angled intra-predictor modes).

[0080] In some embodiments, block size threshold s 2 is predefined and set as a fixed value for all video sequences. In some embodiments, the block size threshold s 2It is signaled as a bitstream in a single high-level syntax, such as a sequence header, frame header, slice header, etc.

[0081] In some embodiments, when the current block is located at selected relative positions within a block having a predefined block size, the encoder does not signal intra-block copy related syntax (e.g., the encoder abandons signaling intra-block copy related syntax). For example, when the current block is located at the top-left of a 64x64 block in a grid of 64x64 blocks of the current image (e.g., a partitioning grid), the encoder does not signal intra-block copy related syntax (e.g., the encoder abandons signaling intra-block copy related syntax). In some embodiments, as illustrated in FIG. 4b, the current block (426) is in the top-left portion of the superblock (428), and the search range for the prediction block of the current block (426) will occur in the bottom-left portion (432), reducing the likelihood that a good prediction block will be selected. For example, the bottom left portion (432) may be adjacent to the unprocessed area (434) of the current image (420). As a result, the encoder abandons signaling of intra-block copy related syntax. In contrast, in some embodiments, for a coding block (430) located at the bottom right corner of the superblock (428), the search range for the prediction block of the coding block (430) will not be limited to the bottom left portion (432), and the encoder may signal intra-block copy related syntax as a bitstream for the coding block (430).

[0082] In some embodiments, when the current block size is larger than the reconstructed area size of the associated 64x64 block in the 64x64 partitioning grid of the current image (or larger than that scaled value), the encoder does not signal intra-block copy related syntax (e.g., the encoder forgoes signaling intra-block copy related syntax). For example, in scenarios where only one or two rows of reconstructed samples are available in the image (420) instead of the four full rows of gray reconstructed samples exemplified in the global search area (422) in FIG. 4b, and the current block size is larger than the area of ​​at least some of the reconstructed samples, the encoder does not signal intra-block copy related syntax and the intra-block copy is not used to encode the current block.

[0083] In some embodiments, when a global search area (e.g., area (422)), or both a local search area (e.g., area (424)) and a global search area (e.g., area (422)) are applicable to searching for a prediction block, the conditions for signaling intra-block copy method-related syntax may differ from the conditions for signaling intra-block copy method-related syntax when only a local search area is applicable.

[0084] In some embodiments, the signaling of intra-block copy related parameters (such as block vector or block vector difference) may depend on one or more characteristics of the current block as described above.

[0085] In some embodiments, a maximum value of one or more intra-block copy related parameters (such as block vectors or block vector differences) may be derived, and the signaling of intra-block copy related parameters may depend on this maximum value. For example, if the search for a prediction block is performed in a smaller local area (424), there may be an upper limit for the block vector (e.g., the block vector is not too large). In some embodiments, if searching for prediction blocks in a smaller local search area (424) is disabled (e.g., by an encoder that does not signal IBC flags and / or syntax), searching for prediction blocks in a global search area (422) may still be permitted if the current block is coded using an intra-block copy. Alternatively, the current block is encoded using non-IBC methods (e.g., angle prediction modes or non-angle prediction modes).

[0086] In some embodiments, in some frames, block-level syntax elements such as flags for block partitioning, transform block partitioning, reference frame indices, motion vectors / block vectors, prediction mode, EOB, transform skip flags, transform type flags, etc. are not signaled in the bitstream, but are instead implicitly derived by the decoding component when such frames are identified by separate frame-level flags. In some embodiments, the previous IBC frame(s) are used as a reference having all skipped blocks by using zero motion vectors, unless explicitly signaled otherwise.

[0087] In some embodiments, the local search area extends to a local memory of 128x128 blocks in the luma samples. For example, for a sub-block of size 64x64 or less, reconstructed samples in the left superblock, if available, and already coded samples in the current superblock may be used as IBC references. The availability of reconstructed samples on the left superblock is based on the location of the current 64x64 block. For example, for a current block within the top-left 64x64 sub-block of the current superblock, only already reconstructed samples within the current sub-block, the top-right 64x64 sub-block of the left superblock, the bottom-left 64x64 sub-block, and / or the reconstructed samples of the bottom-right 64x64 sub-block may be used. In another example, for the current block within the top-right 64x64 sub-block of the current superblock, only the already reconstructed samples within the current sub-block, the top-left 64x64 sub-block of the current superblock, the bottom-left 64x64 sub-block of the left superblock, and / or the reconstructed samples of the bottom-right 64x64 sub-block may be used. In another example, for the current block within the bottom-left 64x64 sub-block of the current superblock, only the already reconstructed samples within the top-left and top-right sub-blocks of the current superblock, and / or the reconstructed samples of the bottom-right 64x64 sub-block of the left superblock may be used. In another example, for the current block within the bottom-right 64x64 sub-block of the current superblock, only the already reconstructed samples within the sub-blocks of the current superblock may be used.

[0088] In some embodiments, IBC derivation is performed as shown in Table 1 below.

[0089]

[0090] FIG. 5a is a flowchart illustrating a method (500) for decoding video according to some embodiments. The method (500) may be performed in a computing system (e.g., a server system (112), a source device (102), or an electronic device (120)) having a control circuit and a memory storing instructions for execution by the control circuit. In some embodiments, the method (500) is performed by executing instructions stored in the memory of the computing system (e.g., memory (314)).

[0091] The system receives a video bitstream (e.g., a coded video sequence) containing a plurality of blocks (e.g., corresponding to a set of images) including the current block (502). When one or more characteristics of the current block satisfy one or more criteria (504): the system parses one or more indicators indicating IBC information from the video bitstream (506). The system identifies a prediction block for the current block based on the IBC information (508), and the system reconstructs the current block using the prediction block (510). When one or more characteristics of the current block do not satisfy one or more criteria (514): the system decodes the current block without parsing one or more indicators indicating IBC information (514).

[0092] In some embodiments, flags and syntax indicating the use of the intra-block copy method, e.g., an intra-block copy on / off flag, intra-block copy related parameters such as a block vector, and a block vector predictor index are conditionally signaled, and the condition may include parameters related to coding block sizes, superblock size, selection of a search area, and / or the relative position of the current block relative to an associated block of a given size.

[0093] In some embodiments, the superblock size ( s 0Indicated as) and local navigation area size ( s 1 Given (indicated as), the coding block is such s 2 Only when the size is less than or equal to, block size s 2 There are flags and syntax indicating the use of the derived and signaled intra-block copy method. Otherwise, the flags and syntax are not signaled.

[0094] In some embodiments, s 2 Is s 1 Same as, or s 2 Is s 1 / N and, here N is a power of 2, for example, 2, 4, 8, 16, ... For example, s 1 It can be 64x64, and s 2 It can be derived as 64x64 / 4, which is 32x32. There are flags and syntax indicating the use of the signaled intra-block copy method only when the coding block size is 32x32 or less.

[0095] In some embodiments, s 2 is min( s 0 , s 1 Same as ) or s 2 is min( s 0 , s 1 ) / N and, here N is a power of 2, for example, 2, 4, 8, 16, ... For example, s 0 It can be 32x32, and s 1 It can be 64x64, ands 2 is 16x16 min( s 0 , s 1 It is derived as ) / 4. There are flags and syntax indicating the use of the signaled intra-block copy method only when the coding block size is 16x16 or less.

[0096] In some embodiments, when a coding block is located at selected relative positions within a given block size having predefined block sizes, intra-block copy related syntax is not signaled. For example, when the current coding block is located at the top-left of an associated 64x64 block in a 64x64 partitioning grid of the current image, then intra-block copy related syntax is not signaled.

[0097] In another example, when the current coding block size is larger than the reconstructed area size of the associated 64x64 block in the 64x64 partitioning grid of the current image (or larger than the scaled value), intra-block copy related syntax is not signaled.

[0098] In some embodiments, when a global search area, or both local and global search areas are applicable, the conditions for signaling syntax related to an intra-block copy method may differ from the conditions for signaling syntax related to an intra-block copy method when only a local search area is applicable.

[0099] In some embodiments, when a global search area, or both local and global search areas are applicable, the conditional signaling of the intra-block copy flags and syntax described above does not apply.

[0100] In some embodiments, s 2 The size is set as a predefined and fixed value for all video sequences. In some embodiments, s2 The size of is signaled as a bitstream in a single high-level syntax, such as a sequence header, frame header, slice header, etc.

[0101] In some embodiments, when flags and syntax indicating the use of the intra-block copy method are not signaled, the intra-block copy is not applied to the current block.

[0102] In some embodiments, the signaling of intra-block copy related parameters (such as block vectors or block vector differences) may depend on the conditions described above. For example, a maximum value of intra-block copy related parameters (such as block vectors or block vector differences) may be derived, and the signaling of intra-block copy related parameters may depend on this maximum value.

[0103] FIG. 5b is a flowchart illustrating a method (550) for encoding video according to some embodiments. The method (550) may be performed in a computing system (e.g., a server system (112), a source device (102), or an electronic device (120)) having a control circuit and a memory storing instructions for execution by the control circuit. In some embodiments, the method (550) is performed by executing instructions stored in the memory of the computing system (e.g., memory (314)). In some embodiments, the method (550) is performed by the same system as the method (500) described above.

[0104] The system receives video data (e.g., source video sequence) comprising a plurality of blocks (e.g., corresponding to a set of images) including a current block (552). When one or more characteristics of the current block satisfy one or more criteria (554): the system encodes the current block using an IBC mode (556), and the system signals one or more indicators in the video bitstream that indicate IBC information for the current block (558). When one or more characteristics of the current block do not satisfy one or more criteria (560), the system does not signal IBC information for the current block (562). As previously described, the encoding process may mirror the decoding processes described herein (e.g., conditional signaling of the intra-mode copy embodiments described above). For brevity, these details are not repeated herein.

[0105] FIGS. 5a and 5b illustrate a number of logical stages in a specific order, but stages that are not order-dependent may be rearranged, and other stages may be combined or separated. Since some rearrangements or other groupings not specifically mentioned will be obvious to a person skilled in the art, the orderings and groupings presented herein are not exhaustive. Furthermore, it should be recognized that the stages may be implemented in hardware, firmware, software, or any combination thereof.

[0106] Now, let's look at some exemplary embodiments.

[0107] (A1) In one embodiment, some embodiments include a method of video decoding (e.g., method (500)). In some embodiments, this method is performed in a computing system having memory and control circuits (e.g., server system (112)). In some embodiments, this method is performed in a coding module (e.g., coding module (320)). In some embodiments, this method is performed in a source coding component (e.g., source coder (202)), a coding engine (e.g., coding engine (212)), and / or an entropy coder (e.g., entropy coder (214)). This method includes the step of (i) receiving a video bitstream (e.g., coded video sequence) comprising a plurality of blocks, including a current block. When one or more characteristics of the current block satisfy one or more criteria, this method includes the step of (ii) parsing one or more indicators indicating IBC information from the video bitstream; and (iii) identifying a predicted block for the current block based on the IBC information. and (iv) a step of reconstructing the current block using a predicted block. When one or more characteristics of the current block do not satisfy one or more criteria, the method includes (v) a step of decoding the current block without parsing one or more indicators indicating IBC information. For example, flags and / or syntax elements indicating the use of the intra-block copy method (e.g., intra-block copy on / off flags, and / or intra-block copy related parameters such as block vectors, block vector predictor indices, etc.) are conditionally signaled based on one or more of coding block sizes, superblock size, selection of a search area (e.g., selection of either a global IBC or a local IBC), and / or the position of the current block relative to an associated block of a given size.In some embodiments, one or more indicators representing IBC information are parsed from the video bitstream based on a determination that one or more characteristics of the current block satisfy one or more criteria. In some embodiments, the current block is decoded without parsing one or more indicators representing IBC information based on a determination that one or more characteristics of the current block do not satisfy one or more criteria (e.g., the system abandons parsing one or more indicators).

[0108] (A2) In some embodiments of A1, one or more criteria include: the size of the coding block of the current block; the size of the superblock corresponding to the current block; the selection of a search area for the current block; and the position of the current block relative to an associated block of a given size.

[0109] (A3) In some embodiments of A1 or A2, the IBC information includes one or more of an IBC mode flag; a block vector; a block vector predictor index; and a block vector difference. For example, signaling of IBC-related parameters (such as block vector, block vector difference, and / or block vector precision) is based on whether one or more characteristics of the current block meet one or more criteria. In some embodiments, the IBC mode flag indicates whether the IBC mode is enabled for the current block. In some embodiments, the IBC mode flag indicates whether the global or local IBC mode is enabled for the current block.

[0110] (A4) In some embodiments of any of A1 to A3, the step of decoding the current block without parsing one or more indicators indicating IBC information includes the step of decoding the current block according to a non-IBC mode. For example, the non-IBC mode may be an intra-prediction mode (e.g., an angular intra-prediction mode or a non-directional intra-prediction mode). For example, when flags and syntax indicating the use of IBC are not signaled / parsed, IBC is not applied to the current block. In some embodiments, the step of decoding the current block without parsing one or more indicators indicating IBC information includes the step of deriving IBC information based on coded information (e.g., available in the decoder).

[0111] (A5) In some embodiments of any of A1 through A4, one or more criteria include a size threshold based on the superblock size and the local search area size; the current block satisfies one or more criteria when the size of the current block is smaller than the size threshold. For example, the size threshold s2 is derived based on the superblock size s0 and the local search area size s1. If the coding block is less than or equal to this s2 size threshold, flags and / or syntax indicating the use of the IBC are signaled, and otherwise, these flags and / or syntax are not signaled. If the block size is relatively large relative to the superblock size and / or search area, there may not be enough pixels to search (e.g., in the local search area) to acquire the next good candidate. In this case, the local IBC should not be used due to the low chance of finding a good candidate. Therefore, to save signaling overhead, the system may forgo signaling IBC information. In some embodiments, the local IBC is disabled in these situations (e.g., the system defaults to using the global IBC).

[0112] (A6) In some embodiments of A5, the size threshold is equal to the local search area size divided by N, where N is a power of 2. For example, N can be 1, 2, 4, 8, or 16. For example, if s2 is equal to s1 or s1 / N, where N is a power of 2. As a specific example, when s1 is 64x64, s2 can be derived as 64x64 / 4, which is 32x32. In this case, flags and syntax indicating the use of IBC are signaled only when the coding block size is 32x32 or less. Thus, local IBC is not allowed (disabled) when the block size is greater than s1 / N.

[0113] (A7) In some embodiments of A5, the size threshold is equal to S / N, where S is equal to the smaller of the superblock size and the local search area size, and N is a power of 2. For example, s2 is equal to min(s0, s1), or s2 is min(s0, s1) / N, where N is a power of 2 (e.g., 2, 4, 8, 16). As a specific example, s0 is 32x32, s1 is 64x64, and s2 is derived as min(s0, s1) / 4, which is 16x16. In this case, flags and syntax indicating the use of IBC are signaled only when the coding block size is 16x16 or less.

[0114] (A8) In some embodiments of A5, the size threshold is a predefined value. For example, the size threshold s2 is set as a predefined and fixed value for all video sequences.

[0115] (A9) In some embodiments of A5, the method further comprises the step of parsing a second indicator indicating a size threshold from a video bitstream. For example, the size threshold s2 is signaled into the video bitstream in high-level syntax, such as a sequence header, a frame header, or a slice header.

[0116] (A10) In some embodiments of any of A1 through A9, one or more criteria include a block size criterion and a relative block location criterion. For example, when a coding block is located at selected relative locations within a given block having predefined block sizes, IBC-related syntax is not signaled.

[0117] (A11) In some embodiments of A10, the relative block position criterion includes a criterion regarding whether the current block is located in the top-left position of the partitioning grid. For example, when the current coding block is located in the top-left of the associated 64x64 block in the 64x64 partitioning grid of the current image, the intra-block copy related syntax is not signaled. In this case, the search area may have a lower chance of yielding a good candidate. The search area may be in the bottom-left for the coding block in the top-left, which may result in a lower chance of having a good candidate.

[0118] (A12) In some embodiments of A11, the block size criterion includes a criterion regarding whether the size of the current block is larger than the search area size. For example, when the current coding block size is larger than the reconstructed area size (or a scaled value of such area size) of the associated 64x64 block in the 64x64 partitioning grid of the current image, then intra-block copy related syntax is not signaled (e.g., if the search area is not sufficiently large for the block, local IBC is not allowed).

[0119] (A13) In some embodiments of any of A1 to A12, the method further comprises: identifying a first set of criteria as one or more criteria when a global IBC mode is allowed for the current block; and identifying a second set of criteria as one or more criteria when a global IBC mode is not allowed for the current block. For example, when a global search area, or when both local and global search areas are applicable, the conditions for signaling IBC-related syntax may differ from the conditions for signaling IBC-related syntax when only a local search area is applicable.

[0120] (A14) In some embodiments of any of A1 through A13, the method further comprises the step of decoding the current block without considering whether one or more characteristics of the current block satisfy one or more criteria when a global IBC mode is allowed for the current block. For example, when a global search region, or both local and global search regions are applicable, conditional parsing of IBC information is not applied. In some embodiments, when a global IBC mode is allowed for the current block, one or more indicators are parsed from the video bitstream without considering whether one or more characteristics of the current block satisfy one or more criteria.

[0121] (A15) In some embodiments of any of A1 to A14, the method further comprises the step of deriving a maximum value for an IBC parameter, and one or more criteria include criteria regarding the maximum value for the IBC parameter. For example, a maximum value for an IBC-related parameter (e.g., block vector or block vector difference) may be derived, and the signaling of the IBC-related parameters may depend on such maximum value. For example, if the maximum value for a block vector or block vector difference is too large for the local IBC mode, it may be disabled.

[0122] (B1) In other embodiments, some embodiments include a method of video encoding (e.g., method (550)). In some embodiments, this method is performed in a computing system having memory and control circuitry (e.g., a server system (112)). In some embodiments, this method is performed in a coding module (e.g., a coding module (320)). This method includes receiving video data (e.g., a source video sequence) comprising a plurality of blocks, including a current block. When one or more characteristics of the current block satisfy one or more criteria, this method includes encoding the current block using an IBC mode; and signaling one or more indicators in the video bitstream that indicate IBC information for the current block. When one or more characteristics of the current block do not satisfy one or more criteria, this method includes not signaling IBC information for the current block. In some embodiments, when one or more characteristics of the current block do not satisfy one or more criteria, the current block is encoded in a non-IBC mode. In some embodiments, when one or more characteristics of the current block do not meet one or more criteria, the current block is encoded in global IBC mode.

[0123] (B2) In some embodiments of B1, the method includes the step of signaling an encoded current block in a video bitstream.

[0124] (B3) In some embodiments of B1 or B2, when a global IBC mode is allowed for the current block, the method includes the step of identifying a first set of criteria as one or more criteria. When a global IBC mode is not allowed for the current block, the method includes the step of identifying a second set of criteria as one or more criteria.

[0125] (B4) In some embodiments of any of B1 to B3, when a global IBC mode is allowed for the current block, the method includes the step of encoding the current block without considering whether one or more characteristics of the current block satisfy one or more criteria.

[0126] (C1) In other embodiments, some embodiments include a method for processing visual media data. In some embodiments, such a method is performed in a computing system having memory and control circuits (e.g., a server system (112)). In some embodiments, such a method is performed in a coding module (e.g., a coding module (320)). Such a method includes the steps of: acquiring a source video sequence comprising a plurality of frames; and performing a conversion between a video bitstream of visual media data and the source video sequence according to a format rule. The video bitstream comprises a set of encoded blocks including a current block, and when one or more characteristics of the current block satisfy one or more criteria, one or more indicators indicate IBC information for the current block. The format rule specifies that when one or more characteristics of the current block satisfy one or more criteria, the IBC information is parsed from the video bitstream, a prediction block is identified for the current block based on the IBC information, and the current block is to be reconstructed using the prediction block. In some embodiments, the format rule further specifies that when one or more characteristics of the current block do not satisfy one or more criteria, the current block is decoded without parsing one or more indicators representing IBC information. In some embodiments, the video bitstream does not include one or more indicators when one or more characteristics of the current block do not satisfy one or more criteria.

[0127] In another aspect, some embodiments include a computing system (e.g., a server system (112)) comprising a control circuit (e.g., a control circuit (302)) and a memory (e.g., a memory (314)) connected to the control circuit, wherein the memory stores one or more sets of instructions configured to be executed by the control circuit, and one or more sets of instructions include instructions for performing any of the methods described herein (e.g., A1 to A15, B1 to B4, and C1).

[0128] In another aspect, some embodiments comprise a non-transient computer-readable storage medium storing one or more sets of instructions for execution by a control circuit of a computing system, wherein one or more sets of instructions include instructions for performing any of the methods described herein (e.g., A1 to A15 and B1 to B4, and C1 above). In some embodiments, the non-transient computer-readable storage medium stores a video bitstream generated by any of the video encoding methods described herein.

[0129] Unless otherwise specified, any of the syntax elements (e.g., indicators) described herein may be high-level syntax (HLS). As used herein, HLS is signaled at a level higher than the block level. For example, HLS may correspond to the sequence level, frame level, slice level, or tile level. As another example, HLS elements may be signaled in the video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), adaptation parameter set (APS), slice header, image header, tile header, and / or CTU header.

[0130] It will be understood that while terms such as "first," "second," etc., may be used in this specification to describe various elements, these elements should not be limited by these terms. These terms are used solely to distinguish one element from another. The technical terms used in this specification are for the purpose of describing specific embodiments and are not intended to limit the claims. As used in the description of the embodiments and in the appended claims, singular forms (“a,” “an,” and “the”) are likewise intended to include plural forms unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used in this specification refers to and includes any and all possible combinations of one or more of the associated enumerated items. It will be further understood that the terms “comprises” and / or “comprising,” as used herein, specify the presence of the stated features, integers, steps, actions, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, actions, elements, components, and / or groups thereof.

[0131] As used herein, the term “if” may be interpreted, depending on the context, to mean “when” or “upon” or “in response to determining” or “in accordance with a determination” or “in response to detecting” that a stated condition precedent is true. Similarly, the phrases "[if it is determined [that a stated condition precedent is true]]" or "[if [a stated condition precedent is true]]" or "[when [a stated condition precedent is true]]" may be interpreted, depending on the context, to mean "upon determining" or "in response to determining" or "in accordance with a determination" or "upon detecting" or "in response to detecting" that a stated condition precedent is true.

[0132] The foregoing description has been explained with reference to specific embodiments for the purposes of explanation. However, the above exemplary discussion is not intended to be exhaustive or to limit the claims to the exact form disclosed. Many modifications and variations are possible in consideration of the above teachings. The embodiments have been selected and described to best explain the principles of operation and practical applications so as to enable other people skilled in the art to do so.

Claims

Claim 1 A method for video decoding performed in a computing system having memory and one or more processors, the method comprising: receiving a video bitstream comprising a plurality of blocks, including a current block; and when one or more characteristics of the current block satisfy one or more criteria, A step of parsing one or more indicators representing IBC (intra block copy) information from the above video bitstream; A step of identifying a predicted block for the current block based on the above IBC information; and A method comprising: a step of reconstructing the current block using the prediction block; and a step of decoding the current block without parsing the one or more indicators displaying the IBC information when one or more characteristics of the current block do not satisfy the one or more criteria. Claim 2 A method according to claim 1, wherein one or more criteria include: the size of the coding block of the current block; the size of the superblock corresponding to the current block; the selection of a search area for the current block; and one or more of the position of the current block relative to an associated block of a given size. Claim 3 A method according to claim 1, wherein the IBC information comprises one or more of an IBC mode flag; a block vector; a block vector predictor index; and a block vector difference. Claim 4 In claim 1, the step of decoding the current block without parsing the one or more indicators displaying the IBC information includes the step of decoding the current block according to a non-IBC mode. Claim 5 In claim 1, the one or more criteria include a size threshold based on a superblock size and a local search area size; and a method in which the current block satisfies the one or more criteria when the size of the current block is smaller than the size threshold. Claim 6 In paragraph 5, the size threshold is equal to the value obtained by dividing the local search area size by N, and N is a power of 2. Claim 7 In paragraph 5, the size threshold is equal to S / N, S is equal to the smaller of the superblock size and the local search area size, and N is a power of 2. Claim 8 In paragraph 5, the method wherein the size threshold is a predefined value. Claim 9 A method according to claim 5, further comprising the step of parsing a second indicator indicating the size threshold from the video bitstream. Claim 10 A method according to claim 1, wherein one or more of the criteria include a block size criterion and a relative block position criterion. Claim 11 In paragraph 10, the relative block position criterion includes a criterion regarding whether the current block is located in the top-left position in the partitioning grid. Claim 12 In paragraph 10, the above block size criterion includes a criterion regarding whether the size of the current block is larger than the search area size. Claim 13 A method according to claim 1, further comprising: a step of identifying a first set of criteria as one or more criteria when a global IBC mode is allowed for the current block; and a step of identifying a second set of criteria as one or more criteria when a global IBC mode is not allowed for the current block. Claim 14 A method according to claim 1, further comprising the step of decoding the current block without considering whether one or more characteristics of the current block satisfy one or more criteria when a global IBC mode is allowed for the current block. Claim 15 A method according to claim 1, further comprising the step of deriving a maximum value for an IBC parameter, wherein one or more criteria include criteria regarding the maximum value for the IBC parameter. Claim 16 A method of video encoding performed on a computing system having memory and one or more processors, the method comprising: receiving video data comprising a plurality of blocks, including a current block; and when one or more characteristics of the current block satisfy one or more criteria, A step of encoding the current block using IBC (intra block copy) mode; and A method comprising: signaling one or more indicators indicating IBC information for the current block in a video bitstream; and not signaling IBC information for the current block when one or more characteristics of the current block do not satisfy one or more criteria. Claim 17 A method according to claim 16, further comprising the step of signaling the encoded current block in the video bitstream. Claim 18 A method according to claim 16, further comprising: a step of identifying a first set of criteria as one or more criteria when a global IBC mode is allowed for the current block; and a step of identifying a second set of criteria as one or more criteria when a global IBC mode is not allowed for the current block. Claim 19 In paragraph 16, additionally, a method comprising the step of encoding the current block without considering whether one or more characteristics of the current block satisfy one or more criteria when a global IBC mode is allowed for the current block. Claim 20 A non-transient computer-readable storage medium for storing a video bitstream generated by a video encoding method, the method comprising: receiving video data comprising a plurality of blocks, including a current block; and when one or more characteristics of the current block satisfy one or more criteria, A step of encoding the current block using IBC (intra block copy) mode; and A non-transient computer-readable storage medium comprising: a step of signaling one or more indicators indicating IBC information for the current block in the video bitstream; and a step of not signaling IBC information for the current block when one or more characteristics of the current block do not satisfy one or more criteria.

Images