Block or motion vector encoding and decoding

Abstract

A decoding method is disclosed. First, for each component of the vector difference, a prefix portion is decoded (S200). For each non-zero component, information indicating whether a suffix portion is predicted for that component is decoded (S202). For each non-zero component, a suffix portion is obtained (S204) based on information decoded for that component. Finally, the vector difference is obtained (S206) from the prefix portion and the suffix portion.

Description

Cross Reference to Related Applications

[0001] This application claims the benefit of European application No. 23306413.8, filed August 24, 2023, which is incorporated by reference in its entirety. TECHNICAL FIELD

[0002] At least one example herein relates generally to a method and apparatus for encoding (respectively, decoding) a vector (e.g., a block vector or a motion vector). BACKGROUND

[0003] To achieve high compression efficiency, image and video coding schemes usually employ prediction and transform to exploit the spatial and temporal redundancy in the video content. Typically, inter or intra picture correlation is utilized using inter or intra prediction, and then the difference (often referred to as prediction error or prediction residual) between the original block and the predicted block is transformed, quantized, and entropy coded. To reconstruct the video, the compressed data is decoded by the inverse processes corresponding to the entropy coding, quantization, transform, and prediction. SUMMARY

[0004] In one implementation, the prefix part is decoded for each component of the vector difference.

[0005] Information indicating whether the suffix part is predicted can be decoded for each component. The suffix part can be obtained (e.g., decoded or predicted) based on the information decoded for the component. In another implementation, if both components are non-zero, information (e.g., a flag) indicating whether the suffix parts for both components are predicted can be decoded, and the suffix part for each component can be obtained (e.g., decoded or predicted) based on the decoded information. If only one component is non-zero or both components are non-zero but at least one suffix part cannot be predicted, information indicating whether the suffix part is predicted for the component (e.g., for each non-zero component) can be decoded, and the suffix part for the component is obtained (e.g., decoded or predicted) based on the decoded information.

[0006] The vector difference can finally be obtained from the prefix and suffix parts and used for reconstructing the image block. BRIEF DESCRIPTION OF DRAWINGS

[0007] Figure 1 A block diagram of a system in which aspects of the embodiments herein can be implemented is shown; Figure 2 A block diagram of a video encoder embodiment is shown; Figure 3 A block diagram of a video decoder embodiment is shown; Figure 4 A current block predicted from a reference block identified by a block vector is depicted; Figure 5a reference region representing an intra block copy (IBC) mode; Figure 6 An example of a reference region for encoding and / or decoding a coding tree unit (CTU) is shown; Figure 7 An example of a block vector predictor (BVP) candidate position that can be used to replace a zero vector in an IBC merge / AMVP (advanced motion vector prediction) list is shown; Figure 8 BVP candidate clustering based on L2 distance and template matching (TM) cost is shown; Figure 9 An intra template matching (Intra TMP) search region is shown; Figure 10 Using intra template matching prediction (Intra TMP) block vectors for IBC blocks is shown; Figure 11 BV prediction encoding is shown; Figure 12 A process for template matching in a given search region is shown; Figure 13 A flowchart of an encoding method according to a first example is depicted; Figure 14A A flowchart of a decoding method according to an example is depicted; Figure 14B A flowchart of suffix portion fetching according to an example is depicted; Figure 15 A flowchart of an encoding method according to a second example is depicted; and Figure 16 A flowchart of a decoding method according to another example is depicted. DETAILED DESCRIPTION

[0008] This application describes a variety of aspects, including tools, features, embodiments, models, schemes, and the like. Many of these aspects are described in detail, and are generally described in a manner that can sound limiting, at least to demonstrate the individual features. However, this is for clarity of description, and does not limit the application or scope of these aspects. In fact, all different aspects can be combined and interchanged to provide more aspects. Moreover, these aspects can be combined and interchanged with aspects described in earlier applications.

[0009] The aspects described and contemplated in this application can be implemented in a number of different forms. The following Figure 1 , 2 and 3 provide some embodiments, but other embodiments are also contemplated, and the scope of the Figure 1 , 2The discussion in section 3 does not limit the breadth of implementation. At least one aspect generally relates to video encoding and decoding, and at least another aspect generally relates to the transmission of generated or encoded bitstreams. These and other aspects can be implemented as methods, apparatus, computer-readable storage media storing instructions for encoding or decoding video data according to any of the methods, and / or computer-readable storage media storing bitstreams generated according to any of the methods.

[0010] In this application, the terms "reconstruction" and "decoding" are used interchangeably, as are the terms "encoded" and "coded," "pixel" and "sample," and "image," "picture," and "frame." Generally, but not necessarily, the term "reconstruction" is used on the encoder side, while "decoding" is used on the decoder side.

[0011] This document describes various methods, each including one or more steps or actions for implementing the method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions can be modified or combined. Furthermore, terms such as "first" and "second" can be used in various embodiments to modify elements, components, steps, operations, etc., for example, "first decoding" and "second decoding." The use of such terms does not imply an ordering of the modified operations unless specifically required. Therefore, in this example, the first decoding does not need to be performed before the second decoding and can occur, for example, before, during, or within an overlapping period of the second decoding.

[0012] For clarity, in the embodiments described herein, met conditions, unmet conditions, and configuration condition parameters are described relative to a threshold (e.g., greater than or less than a threshold), a value (e.g., a threshold), a configuration value (e.g., a threshold), etc. For example, a met condition may be described as being above (e.g., a threshold) value, while an unmet condition (e.g., a performance criterion) may be described as being below (e.g., a threshold) value. The embodiments described herein are not limited to threshold-based conditions. Any other types of conditions and parameters (e.g., belonging to or not belonging to a value range) may be applied to the embodiments described herein.

[0013] The aspects described herein are not limited to ECM, VVC, or HEVC, and can be applied, for example, to other standards and recommendations (whether pre-existing or developed in the future) and any extensions of such standards and recommendations (including VVC and HEVC). Unless otherwise stated or technically impractical, the aspects described herein may be used individually or in combination.

[0014] Figure 1Example block diagrams of systems that can implement various aspects and embodiments are shown. System 100 can be embodied as a device including various components described below and configured to perform one or more aspects described in this application. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptops, smartphones, tablets, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The elements of system 100 can be embodied individually or in combination in a single integrated circuit, multiple integrated circuits, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 100 are distributed across multiple integrated circuits and / or discrete components. In various embodiments, system 100 is communicatively coupled to other systems or other electronic devices via, for example, a communication bus or dedicated input and / or output ports. In various embodiments, system 100 is configured to implement one or more aspects described in this application.

[0015] System 100 includes at least one processor 110 configured to execute instructions loaded thereon to implement various aspects, such as those described in this application. Processor 110 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 100 includes at least one memory 120 (e.g., a volatile memory device and / or a non-volatile memory device). System 100 includes a storage device 140, which may include non-volatile memory and / or volatile memory, including but not limited to EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash memory, disk drives, and / or optical disk drives. Storage device 140 may include internal storage devices, additional storage devices, and / or network-accessible storage devices, as non-limiting examples.

[0016] System 100 includes an encoder / decoder module 130 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 130 may include its own processor and memory. The encoder / decoder module 130 represents a module that can be included in a device to perform encoding and / or decoding functions. It is well known that a device may include one or both of an encoding module and a decoding module. Furthermore, the encoder / decoder module 130 may be implemented as a separate element of system 100, or it may be incorporated into processor 110 in a combination of hardware and software, as is known to those skilled in the art.

[0017] Program code intended for loading onto processor 110 or encoder / decoder 130 to execute the various aspects described herein may be stored in storage device 140 and subsequently loaded into memory 120 for execution by processor 110. According to various embodiments, one or more of processor 110, memory 120, storage device 140, and encoder / decoder module 130 may store one or more items of a variety during execution of the processes described herein. Such stored items may include, but are not limited to, input video, decoded video or a portion thereof, bitstreams, matrices, variables, and intermediate or final results from processing equations, formulas, operations, and operational logic.

[0018] In some embodiments, memory within processor 110 and / or encoder / decoder module 130 is used to store instructions and provide processing working memory required during encoding or decoding. However, in other embodiments, external memory (e.g., the processing device may be processor 110 or encoder / decoder module 130) is used for one or more of these functions. External memory may be memory 120 and / or storage device 140, such as dynamic volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store the television's operating system. In at least one embodiment, fast external dynamic volatile memory (e.g., RAM) is used as working memory for video encoding and decoding operations, such as for MPEG-2 (MPEG stands for Moving Picture Experts Group, MPEG-2 is also known as ISO / IEC 13818, 13818-1 is also known as H.222, 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Various Video Coding, JVET is a new standard being developed by the Joint Video Experts Group)).

[0019] Inputs to the components of system 100 can be provided through a variety of input devices as shown in block 105. Such input devices include, but are not limited to, (i) an RF section that receives radio frequency (RF) signals, for example, transmitted wirelessly via a broadcaster, (ii) component input terminals (or sets of COMP input terminals), (iii) universal serial bus (USB) input terminals, and / or (iv) high-definition multimedia interface (HDMI) input terminals. Other examples ( Figure 1 (Not shown in the image) Includes composite video.

[0020] In various embodiments, the input device of block 105 has corresponding associated input processing elements known in the art. For example, the RF section may be adapted to (i) select a desired frequency (also known as selecting a signal, or limiting a signal to a certain frequency band), (ii) downconvert the selected signal, (iii) limit it again to a narrower frequency band to select (e.g.) a signal band (which may be referred to as a channel in some embodiments), (iv) demodulate the downconverted and band-limited signal, (v) perform error correction, and (vi) demultiplex to select the desired data packet stream. The RF section in various embodiments includes one or more elements for performing these functions, such as a frequency selector, signal selector, band limiter, channel selector, filter, downconverter, demodulator, error corrector, and demultiplexer. The RF section may include a tuner that performs a variety of these functions, such as downconverting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to the desired frequency band. Various embodiments rearrange the order of the above (and other) components, remove some of them, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as inserting amplifiers and analog-to-digital converters. In many embodiments, the RF section includes an antenna.

[0021] Furthermore, the USB and / or HDMI terminals may include corresponding interface processors for connecting system 100 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented as needed, for example, within a separate input processing IC or within processor 110. Similarly, various aspects of USB or HDMI interface processing may be implemented within a separate interface IC or within processor 110. The demodulated, error-corrected, and demultiplexed stream is provided to various processing elements, including, for example, processor 110 and encoder / decoder 130 operating in conjunction with memory and storage elements, to process the data stream as needed for presentation on the output device.

[0022] Various components of system 100 can be housed within an integrated housing. Within the integrated housing, various components can be interconnected and transmit data therebetween using suitable connection means 115 (e.g., internal buses known in the art, including I2C buses, wiring, and printed circuit boards).

[0023] System 100 includes a communication interface 150 that enables communication with other devices via a communication channel 190. The communication interface 150 may include, but is not limited to, a transceiver configured to transmit and receive data via the communication channel 190. The communication interface 150 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 190 may be implemented, for example, in a wired and / or wireless medium.

[0024] In various embodiments, data is streamed to system 100 using a Wi-Fi network such as IEEE 802.11 (IEEE stands for Institute of Electrical and Electronics Engineers). In these embodiments, the Wi-Fi signal is received via a communication channel 190 and a communication interface 150 adapted for Wi-Fi communication. The communication channel 190 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top communications. Other embodiments use a set-top box that transmits data via an HDMI connection to input block 105 to provide streaming data to system 100. Still other embodiments use an RF connection to input block 105 to provide streaming data to system 100. As described above, various embodiments provide data in a non-streaming manner. Furthermore, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0025] System 100 can provide output signals to a variety of output devices, including a display 165, a speaker 175, and other peripheral devices 185. The display 165 in various embodiments includes one or more of the following: for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 165 can be used in a television, tablet computer, laptop computer, mobile phone, or other device. The display 165 can also be integrated with other components (e.g., in a smartphone) or separate (e.g., an external display for a laptop computer). Other peripheral devices 185 in various example embodiments include one or more of the following: a stand-alone digital video disc (or digital versatile disc) (DVR, both), an optical disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 185 that provide functionality based on the output of system 100. For example, an optical disc player performs the function of playing the output of system 100.

[0026] In various embodiments, control signals communicate between system 100 and display 165, speaker 175, or other peripheral devices 185 using signals with protocols such as AV Link, CEC, or other communication protocols that enable device-to-device control with or without user intervention. Output devices can be communicatively coupled to system 100 via dedicated connections through corresponding interfaces 160, 170, and 180. Alternatively, output devices can be connected to system 100 via communication interface 150 using communication channel 190. Display 165 and speaker 175 can be integrated with other components of system 100 into a single unit of an electronic device (e.g., a television set). In various embodiments, display interface 160 includes a display driver, such as a timing controller (TCon) chip.

[0027] Display 165 and speaker 175 may also be separate from one or more other components, for example, if the RF portion of input 105 is part of a separate set-top box. In various embodiments where display 165 and speaker 175 are external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0028] The embodiments can be executed by computer software implemented by processor 110, by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments can be implemented by one or more integrated circuits. Memory 120 can be of any type suitable for the technical environment and can be implemented using any suitable data storage technology (e.g., optical storage devices, magnetic storage devices, semiconductor-based storage devices, fixed memory, and removable memory, as non-limiting examples). Processor 110 can be of any type suitable for the technical environment and can include one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures, as non-limiting examples.

[0029] Figure 2 An exemplary video encoder 200, such as a VVC (Various Video Coding) encoder, is shown. Figure 2 An encoder that improves upon the VVC standard or employs a technology similar to VVC can also be shown.

[0030] Before encoding, the video sequence can undergo pre-coding (201), for example, by applying a color transformation to the input color image (e.g., from RGB 4:4:4 to YCbCr 4:2:0), or by performing a remapping of the input image components to obtain a more compression-resistant signal distribution (e.g., using histogram equalization with one of the color components). Metadata can be associated with the pre-processing and appended to the bitstream.

[0031] In encoder 200, the image is encoded by encoder elements as described below. The image to be encoded is divided (202) and processed in units such as CUs (coding units). Each unit is encoded using, for example, an intra-frame or inter-frame mode. When a unit is encoded in intra-frame mode, it performs intra-frame prediction (260), for example, using an intra-frame prediction tool such as decoder-side intra-frame mode derivation (DIMD). In inter-frame mode, motion estimation (275) and compensation (270) are performed. The encoder decides (205) whether to use intra-frame or inter-frame mode to encode the unit and indicates the intra-frame / inter-frame decision by, for example, a prediction mode flag. For example, the prediction residual is calculated by subtracting (210) the predicted block (also called the prediction block) from the original image block.

[0032] Then, the prediction residuals are transformed (225) into transformation coefficients c (also known as prediction residual transformation coefficients), and these coefficients are quantized (230) into quantization indices. (Also known as transform coefficient levels or quantized transform coefficients on the encoder side). Quantization level (also known as quantization index). The motion vectors and other syntax elements (such as image segmentation information) are entropy encoded (245) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transform and quantization, that is, the residual is directly encoded without applying the transform or quantization process.

[0033] The encoder decodes the encoded blocks to provide a reference for further prediction. The quantization transform coefficients are dequantized (240) (also known as scaling) and inverse transformed (250) to decode the prediction residuals. The decoded prediction residuals and the predicted blocks are combined (255) to reconstruct the image blocks. A loop filter (265) is applied to the reconstructed image to perform filtering such as deblocking / SAO (Sample Adaptive Offset) / ALF (Adaptive Loop Filtering) to reduce coding artifacts. The filtered image is stored in a reference image buffer (280). Thus, the loop filter (265) is used to enhance the reconstructed image before storing it in the reference image buffer (280). Loop filters constitute a series. Among them, the deblocking filter (DBF) is designed to reduce block artifacts that occur along block boundaries. Deblocking filters are generally designed to improve subjective quality, i.e., the perceptibility of such coding errors by the human psychological visual system. In typical video coding standards such as HEVC and VVC, deblocking filters are pre-determined based on coding information (e.g., prediction modes, motion vectors, transform coefficients) and local variations across block boundaries. Adaptive Loop Filters (ALFs), on the other hand, are learned on the encoder side to minimize the mean square error relative to the source image, and the learned filter weights are then encoded into the bitstream. ALFs are typically applied at the CTU level, while deblocking filters are applied along block boundaries.

[0034] Figure 3 A block diagram of an exemplary video decoder 300 is shown. In decoder 300, the bitstream is decoded by decoder elements as described below. Video decoder 300 typically performs operations similar to... Figure 2 The encoding process is the inverse of the decoding process. Encoder 200 typically also performs video decoding as part of the video data encoding.

[0035] Specifically, the decoder's input includes a video bitstream, which can be generated by the video encoder 200. The bitstream is first entropy-decoded (330) to obtain the quantization level. (Also known as transform coefficient level or quantization level on the decoder side), prediction mode, motion vectors, and other encoded information. Image partitioning information indicates how the image is partitioned. The decoder can therefore segment the (335) image based on the decoded image partitioning information. Quantization level The transformed coefficients are dequantized (340) into reconstructed coefficients. Dequantization, also known as scaling, involves reconstructing the transform coefficients. The image is inversely transformed (350) to obtain the prediction residual. The prediction residual and the predicted block (also called the prediction block) are combined (355) to reconstruct the image block. The predicted block can be obtained from intra-frame prediction (360) or motion-compensated prediction (i.e., inter-frame prediction) (375). A loop filter (365) is applied to the reconstructed image. The filtered image is stored in a reference image buffer (380). Note that for a given image, the contents of the reference image buffer 380 on the decoder 300 side are the same as the contents of the reference image buffer 280 on the encoder 200 side for the same image.

[0036] The decoded image can be further processed (385), such as inverse color transformation (e.g., from YcbCr 4:2:0 to RGB 4:4:4) or inverse remapping of the inverse of the remapping process performed in pre-encoding (201). The post-decoding process can utilize metadata derived in the pre-encoding process and signaled in the bitstream.

[0037] HEVC, VVC, and JVET's ECM (an abbreviation for Enhanced Compression Model) use Intra-Block Copying (IBC). IBC is a tool that can be used for encoding both screen content and natural content (e.g., in ECM). IBC-encoded coding units (CUs) can be viewed as independent prediction modes, rather than intra- or inter-frame prediction modes (as in VVC), or as part of an inter-frame mode (e.g., as in HEVC). Therefore, in VVC and ECM, IBC is a separate coding mode with its own vector coding engine compared to the motion vector coding scheme in the inter-frame mode. It's important to note that previously in HEVC, a unified design was used between IBC and the inter-frame mode because both shared the concept of representing shifted blocks; this is no longer the case in VVC and ECM. In any case, IBC prediction is performed based on reconstructed samples before loop filtering.

[0038] Inter-frame coding (IBC) is a predictive coding technique that explores similarities within the same image, such as repeating patterns. In this mode, the current image block (also called a CU) is predicted by a reference block that has been reconstructed within the same image (the reference sample originates from within the reconstructed portion of the current image). The offset from the current block to its reference block is called the block vector (BV) or displacement vector, which is a vector indicating the displacement from the current block to the reference block. If a particular CU uses IBC, the CU is represented by its BV and the residual signal of that CU, similar to inter-frame patterns in inter-frame motion estimation. Merging and skipping modes can also be performed in IBC.

[0039] Because IBC mode is an encoding mode implemented at the block level, block matching (BM) can be performed at the encoder to find the optimal BV for each CU. The luma block vector of an IBC-encoded CU can be integer precision (e.g., for simplicity). The chroma block vector can also be rounded to integer precision. IBC mode can switch between 1-pixel (pel) and 4-pixel motion vector precision, for example, when combined with Adaptive Motion Vector Resolution (AMVR). In the example, IBC mode can be applied to some CUs (e.g., CUs with a width and height less than or equal to 64 luma samples). IBC is also known as intra-picture block compensation or current picture reference (CPR).

[0040] In the example, all reconstructed portions of the current image can be used as the IBC reference region (also known as the IBC search range) for the current block encoded in IBC mode. However, to support hardware implementation, some constraints can be imposed on the full-frame-based IBC solution to make it more implementation-friendly.

[0041] Figure 4An image segment is depicted, in which the current CU (also called the current block) with width w and height h is predicted by a reference block selected in the IBC reference region and identified by the block vector. Figure 4 The image shows the IBC reference region in HEVC Screen Content Coding Extension (HEVC-SCC), where the white areas represent the undecoded (also known as reconstructed) portions of the currently decoded image. The gray areas represent the reconstructed portions. The solid line L1 within this region forms the boundary of the available IBC reference region (also known as the IBC search range). When the video format is 4:2:0, the L2 line immediately adjacent to the boundary can be excluded from the search range. For parallel processing considerations, reconstructed portions not included within the L1 line boundary can be excluded from the IBC reference region.

[0042] More restricted reference regions can be used, for example, to limit memory consumption and / or decoder complexity. For instance, the IBC reference region (also known as the IBC search range) can be limited to the current coding tree unit (CTU). In another example, when employing efficient memory handling (memory reuse), some reference samples from the left-hand CTU can be used for the IBC mode. In a specific example, to limit memory consumption, the maximum block size in the IBC mode can be limited to 64×64.

[0043] Therefore, in order to limit memory consumption and decoder complexity, IBC (e.g. in VVC) can allow the IBC reference area to include only the area of ​​the current CTU and some areas of the left CTU. Figure 5 This diagram shows the reference area for the IBC mode, where each block represents a 64×64 luminance sample cell. On this diagram, each cross indicates a block in the left-hand CTU that is excluded from the available IBC reference area. Gray blocks without crosses belong to the IBC reference area.

[0044] Based on the current CU's position within the current CTU, the following rules can be applied to define the IBC reference region. If the current block (also called the current CU) falls within the upper-left block (e.g., a 64×64 block) of the current CTU, then in addition to the reconstructed samples in the current CTU, it can also use the IBC mode to reference reference samples in the lower-right block (e.g., a 64×64 block) of the left CTU. The current block can also use the IBC mode to reference reference samples in the lower-left 64×64 block of the left CTU and reference samples in the upper-right block (e.g., a 64×64 block) of the left CTU.

[0045] If the current block falls within the upper right block (e.g., a 64×64 block) of the current CTU, then in addition to the reconstructed samples in the current CTU, if the brightness position (0, 64) relative to the current CTU has not yet been reconstructed, the current block can also use IBC mode to reference the reference samples in the lower left block (e.g., a 64×64 block) and lower right block (e.g., a 64×64 block) of the left CTU; otherwise, the current block can also reference the reference samples in the lower right block (e.g., a 64×64 block) of the left CTU.

[0046] If the current block falls within the lower-left block (e.g., a 64×64 block) of the current CTU, then in addition to the reconstructed samples in the current CTU, if the brightness position (64, 0) relative to the current CTU has not yet been reconstructed, the current block can also use IBC mode to reference reference samples in the upper-right (e.g., a 64×64 block) and lower-right (e.g., a 64×64 block) of the left CTU. Otherwise, the current block can use IBC mode to reference reference samples in the lower-right (e.g., a 64×64 block) of the left CTU. If the current block falls within the lower-right (e.g., a 64×64 block) of the current CTU, then only the IBC mode can be used to reference reconstructed samples in the current CTU.

[0047] One or more examples in this article (e.g., exemplary constraints) can allow the IBC mode to be implemented in a hardware implementation using local on-chip memory (or multiple memories).

[0048] In some examples, the reference area for IBC can extend to two CTU lines above the CTU being processed by the encoder and / or decoder. Figure 6 An example of a reference region for encoding and / or decoding a CTU(m,n) is shown. For a CTU(m,n) to be encoded (and correspondingly decoded), the reference region may include CTUs indexed as (m-2,n-2)...(W,n-2), (0,n-1)...(W,n-1), (0,n)...(m,n), where W represents the maximum horizontal index within the current tile, slice, or image. The sample-by-sample block vector search (or local search) range can be limited to [-(C<<1), C>>2] in the horizontal direction and [-C, C>>2] in the vertical direction, for example, to accommodate an expansion of the reference region, where C represents the CTU size, >> is the right shift operator, and << is the left shift operator.

[0049] IBC modes can be signaled at the CU level (e.g., with flags). In one or more examples in this paper, an IBC mode can be signaled as an IBC Advanced Motion Vector Prediction (AMVP) mode or an IBC skip / merge mode. In the example, an IBC mode can be signaled as an IBC skip / merge mode, where a merge candidate index can be used to indicate which block vector from a list derived from neighboring candidate IBC coding blocks is used to predict the current block. The merge list can include (e.g., composed of) spatial candidates, history-based motion vector predictors (HMVPs), and paired candidates. A simplified merge candidate list with two spatially neighboring block BVs and five HMVPs can be used. Up to six candidates can be used in the list. The first two entries of the same predictor list can also be used for non-merge BV prediction modes.

[0050] In some examples, the IBC mode can be signaled as the IBC AMVP mode, where the block vector difference (BVD) can be encoded (and decoded accordingly) in a similar or identical manner to the motion vector difference (MVD). The block vector prediction method can use multiple (e.g., two) candidates as predictors, selected from a merge list based on minimum cost (e.g., if IBC encoded). In the example, the block vector prediction method can use two candidates as predictors, one from the left neighbor and one from the top neighbor (e.g., if IBC encoded). If one or more of the candidates (e.g., neighbors) are unavailable, a default block vector can be used as the predictor. An indication (e.g., a flag) can be signaled to indicate the block vector predictor index.

[0051] In the example, the IBC merge / AMVP list construction can be modified as follows: IBC merge / AMVP candidates can be inserted into the IBC merge / AMVP candidate list if and / or only if the IBC merge / AMVP candidates are valid. One or more top-right, bottom-left, and / or top-left space candidates and / or a pairwise average candidate can be added to the IBC merge / AMVP candidate list. Template-based adaptive reordering (e.g., Template-based Merge Candidate Adaptive Reordering (ARMC-TM)) can be applied to the IBC merge list.

[0052] The size of the HMVP table used for IBC can be increased to 25 entries. After one or more IBC merge candidates (e.g., up to 20 IBC merge candidates) are derived through full pruning, they can be reordered together. After reordering, the top few candidates with the lowest template matching cost (e.g., the top 6 candidates) can be selected as the final candidates in the IBC merge list.

[0053] Candidates from the zero vector (e.g., used to populate the IBC merge / AMVP list, such as in the case of an incomplete list) can be replaced with a set of block vector predictor (BVP) candidates located in the IBC reference region. In the example, the zero vector as a block vector in the IBC merge mode can be invalid (e.g., therefore it is discarded as a BVP in the IBC candidate list).

[0054] Figure 7 An example of a BVP candidate position that can be used to replace the zero vector in the IBC merge / AMVP list is shown. Figure 7 As shown, three candidates are located at the nearest corner of the reference area defined by the dashed line, and three other candidates are determined in the middle of the three sub-regions (A, B, and C), whose coordinates can be determined by the width and height of the current block and the ΔX and ΔY parameters.

[0055] Clustering of BVP candidates can be applied when both BV candidate components (the horizontal and vertical projections of the vectors, BVx and Bvy) are non-zero (e.g., during the construction of the IBCAMVP list). Figure 8 Clustering based on L2 distance and TM cost is illustrated. In the example, clustering is applied if there are more than 2 valid BV candidates and up to 6 candidates. The cluster radius can be defined as...

[0056] in and These are the width and height of the current block (also known as the CU). It is the minimum allowed CU size. It is the right shift operator.

[0057] Clustering methods can be applied sequentially from the candidate list, and candidates assigned to a group can be removed from the list used for subsequent clustering. Within a group (e.g., each group), the BVP with the lowest TM cost is selected as the representative candidate for that group. Finally, the representative candidates from the first two groups can be selected as candidates for the IBC AMVP list.

[0058] Furthermore, if one of the BV candidate components is zero or the block is encoded using Reconstruction Reordered IBC (RRIBC), a flag can be signaled to indicate this situation, carrying a direction flag indicating whether the horizontal or vertical component is non-zero. Instead of the usual IBC AMVP list, two new BVP candidates can be derived, and the sign of the non-zero BV component can be deduced on the decoder side. AMVP BVP0 can be set to the nearest valid position to the current block (-cbWidth or -cbHeight), so the non-zero BVD is always negative, pointing to the left relative to the BV with zero vertical component or above relative to the BV with zero horizontal component. Similarly, AMVP BVP1 can be set to the position furthest from the current block within the valid reference area, i.e., the left or upper boundary of the IBC search area. Therefore, if BVP1 is selected, BVD is always positive, pointing to the right relative to the BV with zero vertical component or below relative to the BV with zero horizontal component.

[0059] The optimal IBC AMVP index can be signaled, which allows the sign of the non-zero BVD component to be obtained (e.g., derived) on the decoder side. The absolute amplitude of the non-zero BVD component can be further signaled. In RRIBC, the direction of the flip mode can be derived from the signaled direction flag.

[0060] IBC can be used in conjunction with template matching. In some examples, template matching (TM)-based motion search and / or refinement can be applied to IBC, for example, simultaneously for IBC merge mode and IBC AMVP mode.

[0061] IBC-TM merging mode can be used. IBC-TM merging mode can involve a merging candidate list for BV prediction, which differs from the list used in regular IBC merging mode. Candidates can be selected based on a pruning method based on the motion distance between candidates (e.g., as in regular TM merging mode). Zero-motion candidates can be replaced by block vectors (e.g., at (-W,0), (0, -H), (-W, -H)).

[0062] In IBC-TM merging mode, selected candidates can be refined using template matching, for example, before RDO (Rate Distortion Optimization) or the decoding process. IBC-TM merging mode can compete with regular IBC merging mode. A TM merge indicator (e.g., a TM merge flag) can be signaled to instruct template matching to merge IBC modes.

[0063] In the IBC-TM AMVP mode, one or more (e.g., up to three) candidates can be selected from the IBC-TM merging list. These candidates (e.g., each candidate) can be refined (e.g., according to a typical template matching method) and can be sorted, for example, according to their resulting TM costs. For example, only the first one or two (e.g., only the first two) can be considered during the block vector estimation process.

[0064] TM thinning (e.g., when used for IBC) can be performed at integer pixel precision, and in IBC-TM AMVP mode, it is performed at integer or 4-pixel precision, depending on the AMVR value. Thinning can be done within the IBC reference area (e.g., an existing IBC reference area).

[0065] TM thinning (e.g., for IBC-TM merge and AMVP modes) is fairly straightforward because the IBC block vectors can be constrained to (i) integers and (ii) within the reference region. In IBC-TM merge mode, some (e.g., all) thinning can be performed at integer precision, and in IBC-TM AMVP mode, they can be performed at integer or 4-pixel precision based on the AMVR value. This thinning only accesses samples that do not require interpolation. In both cases, the thinned block vectors and templates used in each thinning step must adhere to the constraints of the IBC reference region.

[0066] Figure 9 The IntraTMP (Intra-Temporal Template Matching) search area is shown. IntraTMP is a special intra-prediction mode that copies the best prediction block from the reconstructed portion of the current frame, whose L-shaped template matches the current template. For a predefined search range, the encoder searches for the template most similar to the current template in the reconstructed portion of the current frame and uses the corresponding block as the prediction block. The encoder then signals the use of this mode and performs the same prediction operation on the decoder side. Therefore, IntraTMP does not require transmitting BVD to the decoder to find the prediction block.

[0067] The prediction signal can be generated by matching the L-shaped causal neighbors of the current block with another block (e.g., in a predefined search area), such as... Figure 9As shown in the example, the search area can include several regions (e.g., four regions). The first region R1 can indicate the current CTU, the second region R2 can indicate the top-left CTU, the third region R3 can indicate the top CTU, and the fourth region R4 can indicate the left-side CTU. The sum of absolute differences (SAD) can be used as the cost function. Within (e.g., each) region, the decoder can search for the template with the minimum SAD relative to the current template and use its corresponding block as the prediction block. The size of (e.g., all) regions (SearchRange_w, SearchRange_h) can be set to be proportional to the block size (BlkW, BlkH), for example, to have a fixed number of SAD comparisons per pixel. In the example, SearchRange_w = a BlkW and SearchRange_h = a BlkH, where ' ' is a constant that controls the gain / complexity tradeoff. In some examples, ' It can be equal to 5.

[0068] Intra-frame template matching can be enabled for CUs with a width and height less than or equal to 64. The maximum CU size used for intra-frame template matching can be configured.

[0069] Intra-frame template matching prediction modes can be signaled at the CU level via (e.g., dedicated) flags.

[0070] Although both IBC and IntraTMP modes use reference blocks from the current frame to create prediction blocks, IntraTMP does not require encoding the BV (Block Value) to create prediction blocks on the decoder side. However, the BV derived from IntraTMP can be used in IBC. The IntraTMP BV of stored neighboring blocks, as well as the IBC BV, can be used as spatial BV candidates in the IBC candidate list construction. Figure 10 As shown, the current IBC block can use the IBC BV and IntraTMP BV of neighboring blocks as BV candidates for the IBC BV candidate list. The IntraTMP block vector is added as a spatial candidate to the IBC block vector candidate list.

[0071] IBC mode can interact with other coding tools, such as Paired Merge Candidates, History-Based Motion Vector Predictors (HMVP), Combined Intra / Inter-Frame Prediction Mode (CIIP), Merge Mode with Motion Vector Difference (MMVD), and Geometric Partition Mode (GPM). For example, IBC can be used with Paired Merge Candidates and HMVP. New Paired IBC Merge Candidates can be generated, for example, by averaging (e.g., two) IBC Merge Candidates. IBC motion can be inserted into the history buffer for future reference, for example, for HMVP. In some examples, IBC may not be used in conjunction with affine motion inter-frame tools. IBC can be used in conjunction with Combined Inter-Intra-Frame Prediction (CIIP), MMVD, and / or Geometric Partition Mode (GPM). In some examples, IBC may not be allowed for chroma-coded blocks, for example, if / when using DUAL_TREE partitioning. In some examples, IBC can be used with an inter-frame prediction enhancement tool called LIC (Local Illumination Compensation).

[0072] For example, the current image may be included in the reference image list 0 for IBC prediction without being used as a reference image. The block vector derivation process for IBC modes can exclude (e.g., all) neighboring blocks of inter-frame modes, and vice versa.

[0073] The following IBC features can be applied. IBC can share the same process as in (e.g., regular) MV merging, including the use of pairwise merge candidates and history-based motion prediction, but may disallow temporal motion vector prediction (TMVP) and zero vectors, for example, because they are invalid for IBC mode. Separate HMVP buffers (e.g., 5 candidates each) can be used for (e.g., regular) MV and IBC. Block vector constraints can be implemented, for example, in the form of bitstream consistency constraints, such that the encoder ensures that invalid vectors do not exist in the bitstream. Merging mode may not be used, for example, if a merge candidate is invalid (e.g., out of range or zero (0)). Bitstream consistency constraints can be expressed, for example, in the form of virtual buffers.

[0074] IBC can be treated as an inter-frame mode, for example, for deblocking. AMVR may not use quarter pixels, for example, if the current block is encoded using IBC prediction mode. AMVR can be (e.g., only) signaled to indicate whether MV is an integer pixel or a 4-integer pixel, for example, if the current block is encoded using IBC prediction mode. The number of IBC merge candidates can be signaled in the stripe header, for example, separately from the number of regular, sub-block, and / or geometric merge candidates.

[0075] Bidirectional predictive IBC can be implemented, for example, to enhance the encoding performance of IBC for both natural and screen content. IBC can generate prediction samples using only one BV, i.e., unidirectional predictive IBC, but the prediction accuracy can still be improved. In some examples, in addition to unidirectional predictive IBC, IBC with multiple (e.g., two, i.e., bidirectional predictive IBC) BVs can also be used.

[0076] For example, the first approach (method 1) may include IBC BVP-merging patterns. The second approach (method 2) may include bidirectional prediction of IBC merging patterns.

[0077] Method 1 can derive two (e.g., desired) BVs (similar to the MV derivation of the AMVP-merge mode, which combines the AMVP motion vector predictor for the reference list and the inter-frame merge candidate for another reference list) from the IBC Block Vector Prediction (BVP) mode (also referred to in this paper as IBC AMVP) and the IBC Merge mode to form bidirectional predictive inter-frame CUs. The encoder can signal to the decoder two different indices (taken from the IBC AMVP candidate list and the IBC merge candidate list, respectively) for the IBC BVP mode and the IBC merge candidate (different from the AMVP-merge mode).

[0078] Method 2 can derive two (e.g., desired) BVs from the IBC merge candidate list, for example, by utilizing two different IBC merge indices. These two indices can be signaled from the encoder to the decoder. The target of the bidirectional predictive IBC merge mode can be, for example, IBC regular merge, IBC merge mode with block vector difference (IBC-MBVD), and IBC geometric partitioning mode (IBC-GPM), which can be enabled for screen content (e.g., by default). Bidirectional predictive IBC-MBVD can be enabled in both natural and screen content, while bidirectional predictive IBC-GPM can be enabled (e.g., only) in screen content.

[0079] These methods (e.g., Methods 1 and 2) can be implemented based on one or more of the following: These methods can reuse the IBC merging candidate list construction scheme used for unidirectional predictive IBC merging modes. These methods can use BV refinement. For example, these methods can enable IBC using template matching. These methods can use compensation. For example, these methods can use a simple (1:1) average of bidirectional predictive IBC samples to generate the final IBC predictive samples. These methods can store two BVs in the BV store, e.g., if / when bidirectional predictive IBC is enabled. These methods can use signaling. For example, control flags for bidirectional predictive IBC can be signaled at the stripe level in the I stripe (e.g., not signaled in the B and P stripes). Reconstruction reordering of IBC can be disabled, e.g., if / when bidirectional predictive IBC is enabled. These methods can be enabled in the chroma component blocks of a single tree.

[0080] like Figure 11 As shown, BV can be explicitly encoded, for example, if / when using the IBC AMVP mode. BV encoding can use a similar process specified for inter-frame prediction, but the rules can be constructed using a different list of predictor candidates than those used in inter-frame prediction. (For example, the optimal) IBC AMVP can be selected from competing candidates. Once the encoder has selected the final predictor, its index can be signaled. In the example, the encoder selects the Block Vector Predictor (BVP) and can also signal the Block Vector Difference (BVD) relative to the selected BVP. The absolute magnitudes of the non-zero BVD components (here, the components refer to the x and y projections of the vector in two-dimensional space) can be processed separately. For simplicity, BVDx and BVDy refer to the absolute values ​​of the vector components, and the notation is not considered in this paper. On the decoder side, the block vector data can be reconstructed. In the example, BV is equal to the sum of the BVP used for a given CU and its associated BVD.

[0081] In some examples, once the BVD to be encoded is obtained, a specific encoding scheme can be applied. The absolute values ​​BVDx and BVDy can be encoded using exponential Golomb codes. The pseudocode used on the encoder side for BVD encoding can be shown below.

[0082] BVDx = BVDx<0 ? -BVDx : BVDx; BVDy = BVDy<0 ? - BVDy : BVDy; Gr0x_flag = BVDx>0; Gr0y_flag = BVDy>0; if (rribcFlipType != 2) { m_BinEncoder.encodeBin(Gr0x_flag, Ctx::Bvd(HOR_BVD_CTX_OFFSET)); } if (rribcFlipType != 1) { m_BinEncoder.encodeBin(Gr0y_flag, Ctx::Bvd(VER_BVD_CTX_OFFSET)); } if( Gr0x_flag) { xWriteBvdContext(BVDx - 1, NUM_HOR_BVD_CTX, HOR_BVD_CTX_OFFSET, BVD_CODING_GOLOMB_ORDER); m_BinEncoder.encodeBinEP( (BVDx<0) ); } if( Gr0y_flag) { xWriteBvdContext(BVDy -1, NUM_VER_BVD_CTX, VER_BVD_CTX_OFFSET, BVD_CODING_GOLOMB_ORDER); m_BinEncoder.encodeBinEP( (BVDy<0) ); } In the pseudocode above, for each vector component, a flag indicating whether the absolute value is greater than 0 is first written (e.g., Gr0x_flag and Gr0y_flag). Only when it is true is the remaining absolute value minus 1 (e.g., BVDx-1 and BVDy-1) and the sign encoded.

[0083] `xWriteBvdContext` is a function used to write vector magnitudes minus one (e.g., BVDx-1 and BVDy-1) using exponential Golomb code. It first writes the prefix bin (binary bits), then the suffix bin. The first five prefix bins are context-coded, and the remaining bins (if present) can be encoded using equal probability (EP) mode. The suffix bin can also be encoded using EP mode. `BVD_CODING_GOLOMB_ORDER` can be set to 1 (first-order exponential Golomb code), and `NUM_HOR_BVD_CTX` and `NUM_VER_BVD_CTX` can be set to 5, meaning that up to five prefix bins can be context-coded. `HOR_BVD_CTX_OFFSET` and `VER_BVD_CTX_OFFSET` are offsets used to select the appropriate context model.

[0084] The decoder side reassembles the bin in the same operational order, but instead of encoding and writing the bin, it reads and decodes the bin. The pseudocode used for BVD decoding on the decoder side can be shown below.

[0085] BVDx = 0, BVDy = 0, Gr0x_flag = 0, Gr0y_flag = 0; if (rribcFlipType != 2) { Gr0x_flag = m_BinDecoder.decodeBin(Ctx::Bvd(HOR_BVD_CTX_OFFSET)); BVDx = Gr0x_flag; } if (rribcFlipType != 1) { Gr0y_flag = m_BinDecoder.decodeBin(Ctx::Bvd(VER_BVD_CTX_OFFSET)); BVDy = Gr0y_flag; } if( Gr0x_flag) { BVDx += xReadBvdContext(NUM_HOR_BVD_CTX, HOR_BVD_CTX_OFFSET, BVD_CODING_GOLOMB_ORDER); if (m_BinDecoder.decodeBinEP()) { BVDx = -BVDx; } } if( Gr0y_flag) { BVDy += xReadBvdContext(NUM_VER_BVD_CTX, VER_BVD_CTX_OFFSET, BVD_CODING_GOLOMB_ORDER); if (m_BinDecoder.decodeBinEP()) { BVDy = -BVDy; } } The k-th exponential Golomb code (EGk) represents a symbol as two parts: a prefix and a suffix. In EGk codes, the codeword size grows exponentially. For each bit in the prefix, the number of codes in the suffix can double. Each codeword contains a unary code representing the prefix, followed by a fixed-length code for the suffix.

[0086] The symbol L to be encoded is first associated with (e.g., mapped to) a set of indices represented by its index values, which can be determined by an index function. For example, The indexing function p(L) can be encoded using p(L) + 1 bits in unary code. The indexing function maps an integer L to an index that is also an integer. In this sense, integers are partitioned into sets (called index sets) according to the indices they map to, which are defined by the indexing function. Definition. Note that the indices in the set (not the prefix values ​​themselves) are encoded using unary codes that represent the first part of the EGk codeword. Prefix values ​​can be obtained from the index set, for example, using the following formula: .

[0087] Integers can then be encoded as their indices, plus a suffix indicating their rank in the index set. Therefore, the index set can be ordered, for example, using natural ordering in EGk codes. The rank (e.g., the suffix) can be determined from the following equation: Rankings (e.g., suffixes) can be encoded into a fixed-length representation of length p(L) + k, where k is the order of the exponential Golomb code. The fixed-length portion can be viewed as a binary representation using p(L) + k least significant bits. Therefore, the suffix value itself can be encoded using a fixed-length binary representation. It represents the second part of the EGk codeword, which is concatenated with the first part of the EGk codeword.

[0088] In the example, the order of the EG code is set to 1 (k=1). Table 1 provides examples of index set partitions and their prefix and suffix values.

[0089]

[0090] Finally, the symbol L is given by the following formula: L = prefix (L) + suffix (L).

[0091] BVD can be predicted at least partially (e.g., its suffix portion) without explicit encoding. In some examples, a template matching (TM) method can be used, which defines a set of hypotheses and selects the hypothesis that minimizes the cost function as the final output for further analysis. This is an encoder-decoder side derivation method that can be used to infer information about the current CU, for example, by finding the closest match between the template of the current CU (e.g., the top and / or left neighboring regions of the current CU) and the template of a block in a predefined reference region (also called the search region) (e.g., the same size as the current CU template). Costs can be measured (e.g., computed) for some (e.g., all) hypotheses. The decoder (and corresponding encoder) can search for the template with the minimum sum of absolute differences (SAD) and use its corresponding block as the prediction block. In the context of BVD encoding, the predicted BVD is taken from the hypothesis that minimizes the cost (from the prediction block). If the predicted BVD corresponds to the actual BVD (e.g., the one calculated via a regular encoder search process implemented in IBC mode), the prediction is marked as correct, and a (e.g., a dedicated) syntax element (e.g., a flag) can be set to 1 to indicate that the predicted BVD matches the actual BVD. If the predicted BVD (also known as the matching BVD) is the same as the current one, the prediction is true, and the encoder signals the use of this mode. In the latter case, the same prediction operation can be performed on the decoder side. Otherwise, the flag can be set to 0. The values ​​of the syntax elements are just examples. Other values ​​can be used. This flag can signal for each non-zero BVD at the CU level. In this paper, this flag is referred to as "valid_bvd_pred_flag".

[0092] In some examples, the encoder-side method may include the following steps: Obtain (e.g., find) the BVD (actual BVD) of the current CU using a standard encoder search procedure implemented in IBC mode. Obtain (e.g., find) a reference block using a TM in a (e.g., predefined) search region (e.g., search area), for example, if / when at least one vector component of the actual BVD is non-zero. Initialize the flag valid_bvd_pred_flag, for example, to 0. Obtain (e.g., acquire) the BVD (called the predicted BVD) from the reference block. In the example, obtain (e.g., derive) the BV from the reference block, and then obtain the predicted BVD by subtracting the BVP from the obtained (e.g., derived BV). The BVP is, for example, the same as the BVP used to obtain the actual BVD. The actual BVD and the predicted BVD are then compared, for example, by comparing the x and y components of the vector. In some examples, only the non-zero components of the actual BVD are considered (i.e., compared). This means that if the actual vector has one zero component and one non-zero component, only the non-zero component of the actual BVD is compared with the corresponding component of the predicted BVD. In this way, a correct prediction is obtained by comparing only the non-zero components of the actual BVD and the predicted BVD.

[0093] In the example, the comparison of components is further illustrated below. TrueBVD(BVDx-t, BVDy-t) is the actual BVD, where the horizontal and vertical vector components are given as BVDx-t and BVDy-t, respectively. PredBVD(BVDx-p, BVDy-p) is the predicted BVD, where the horizontal and vertical vector components are given as BVDx-p and BVDy-p, respectively.

[0094] If both BVDx-t and BVDy-t are nonzero, the comparison is marked as correct only if: BVDx-t == BVDx-p and BVDy-t == BVDy-p.

[0095] If BVDx-t is zero and BVDy-t is nonzero, the comparison is marked as correct only if: BVDy-t == BVDy-p.

[0096] If BVDx-t is nonzero and BVDy-t is zero, the comparison is marked as correct only if: BVDx-t == BVDx-p.

[0097] If both BVDx-t and BVDy-t are zero, no comparison is made.

[0098] If at least one component of the actual BVD is non-zero and the comparison is marked as correct (see the previous example, for example), set valid_bvd_pred_flag to 1 to indicate that the BVD suffix is ​​predicted; otherwise, set valid_bvd_pred_flag to 0 to indicate that the BVD suffix exists in the bitstream (e.g., where it is signaled / encoded).

[0099] Encode the prefix portions of the x and y components for the actual BVD, for example, only for non-zero vector components. Then encode valid_bvd_pred_flag, for example, only if at least one component of the actual BVD is non-zero. If valid_bvd_pred_flag is equal to 0, encode the suffix portions of the x and y components for the actual BVD, for example, only for non-zero vector components.

[0100] The decoder-side method can reassemble the BVD using the same sequence of operations. In some examples, the decoder-side method may include the following steps: First, decode the prefix portion for the x and y components, for example, only for the non-zero vector components. Initialize the valid_bvd_pred_flag, for example, to 0.

[0101] The syntax element `valid_bvd_pred_flag` indicates whether the BVD suffix is ​​predicted, e.g., if / when at least one vector component is nonzero. When `valid_bvd_pred_flag` equals 1, a TM is used in a (e.g., predefined) search region to obtain (e.g., find) a reference block. The BVD (predicted BVD) is obtained (e.g., retrieved) from the reference block, e.g., in the same manner as on the encoder side. The suffix portion is obtained from the predicted BVD for the x and y components, e.g., only for nonzero vector components. Otherwise (where `valid_bvd_pred_flag` equals 0), the suffix portion can be directly decoded from the bitstream for the x and y components, e.g., only for nonzero vector components.

[0102] In some examples, template matching can be used to search for (e.g., derive) the BVD suffix within a search range (also called the search region or search area) that depends on the BVD prefix (from the current prefix value to the next prefix value). This ensures that the actual BVD is within the search region. For example, if the BVD prefix is ​​6, then the next prefix value, as defined in Table 1, is 14, so the search range is from 6 to 14. However, there is no guarantee that template matching will correctly predict the actual BVD; for example, there is no guarantee that the minimum TM cost will correspond to the template corresponding to the actual BVD. Since the search range is defined based on the prefix, the suffix value may not exceed the range of existing suffix values; therefore, in a given example, the suffix value can range from 0 to 7. This is as follows: Figure 12As shown.

[0103] In some examples, prediction may be made only for non-zero magnitudes of vector components. Non-zero vector components are indicated by flags, such as Gr0x_flag and Gr0y_flag, which are encoded at the start of the BVD encoding process; see, for example, the BV encoding and decoding pseudocode. If / when the magnitude of a vector component is non-zero, this is indicated in the bitstream by encoding a flag greater than zero. This can be applied to some (e.g., all) examples disclosed below.

[0104] In the example above, a correct prediction is signaled via (e.g., a dedicated flag) `valid_bvd_pred_flag` only if at least one component is non-zero and the actual non-zero component of the BVD matches the prediction (whether the prediction is correct or not has been explained previously). In this case, `valid_bvd_pred_flag` is set to 1. Otherwise, if the non-zero BVD component is not correctly predicted or both components are zero, `valid_bvd_pred_flag` is set to 0. If `valid_bvd_pred_flag` equals 1, the suffix values ​​for the non-zero BVD components (suffix BVDx and suffix BVDy) are predicted but not encoded in the bitstream. In the example above, the vector components are not predicted individually. Predicting vector components individually can be beneficial. For this purpose, two new syntax elements (e.g., dedicated flags) could be signaled indicating whether a correct prediction was observed for each vector component. For example, suffix BVDx could be explicitly encoded into the bitstream, while suffix BVDy could be predicted via TM. Therefore, in this example, the vector components (projections onto the x and y axes) are predicted independently. The same template matching process can be used. Independent prediction can be signaled via two new flags, for example, `valid_bvd_x_pred_flag` and `valid_bvd_y_pred_flag`.

[0105] In the following examples, some (e.g., all) flags (e.g., valid_bvd_pred_flag, valid_bvd_x_pred_flag, valid_bvd_y_pred_flag) are initialized to 0 before the process begins. A flag is assumed to be 0 if / when it is not used or encoded into the bitstream. The flag values ​​are provided as examples only in the following examples. Different values ​​may be used in other examples.

[0106] Figure 13A flowchart illustrating the encoding method based on the first example is provided. In this example, global (e.g., joint) prediction and `valid_bvd_pred_flag` are omitted. Independent prediction of the vector components is performed. The same TM method as in the previous example can be used. The predicted BVD is compared to the actual BVD as follows. In some examples, this method may be performed only if the actual BVD magnitude is non-zero.

[0107] In S100, the BVD is obtained (e.g., found) for the image block (e.g., the current CU), for example by using a standard encoder search process implemented in IBC mode, such as... Figure 11 As shown. This BVD is referred to as the actual BVD in the following text.

[0108] In S102, a reference block is obtained (e.g., found), for example by using template matching (TM) in a (e.g., predefined) search area (also called a search region). In the example, S102 is performed only if at least one vector component of the actual BVD is nonzero (i.e., if / when BVDx or BVDy is nonzero or both are nonzero).

[0109] In S104, the BVD is obtained (e.g., derived or acquired) from the reference block obtained in S102, denoted as BVD. TM For example, only if at least one vector component of the actual BVD is nonzero. BVD TM This is referred to below as the predicted BVD. In the example, the BV is obtained (e.g., derived) from the reference block, and then the predicted BVD is obtained by subtracting the BVP from the obtained (e.g., derived) BV. TM The BVP is, for example, the same as the BVP used to obtain the actual BVD.

[0110] In S106, a prefix portion is encoded for each (e.g., each non-zero) component of the block vector difference (i.e., the actual BVD). In the example, the prefix portion of the x component (e.g., if non-zero) is encoded, and the prefix portion of the y component (e.g., if non-zero) is encoded.

[0111] In S108, based on actual BVD and BVD TM The comparison is used to encode information (e.g., flags) in the suffix portion of the BVD for each (e.g., each non-zero) component, indicating whether the component is predicted (i.e. not encoded) or not predicted (i.e. encoded in / signaled in the bitstream).

[0112] For example, if / when the actual BVDx is nonzero, compare the actual BVDx with the predicted BVDx. If they are equal (e.g., identical), set valid_bvd_x_pred_flag to 1, indicating that the BVDx suffix is ​​predicted (i.e., not encoded). Otherwise, set valid_bvd_x_pred_flag to 0, indicating that the BVDx suffix exists in the bitstream (e.g., encoded there, so no prediction is made). If / when BVDy is nonzero, compare the actual BVDy with the predicted BVDy. If they are equal (e.g., identical), set valid_bvd_y_pred_flag to 1, indicating that the BVDy suffix is ​​predicted. Otherwise, set valid_bvd_y_pred_flag to 0, indicating that the BVDy suffix exists in the bitstream (e.g., encoded there, so no prediction is made). The `valid_bvd_x_pred_flag` and `valid_bvd_y_pred_flag` are encoded in the bitstream, for example, only for each non-zero component. The values ​​of the flags can be different from those described above. For example, `valid_bvd_x_pred_flag` (and correspondingly, `valid_bvd_y_pred_flag`) can be set to 1 to indicate that the BVDx suffix (and correspondingly, the BVDy suffix) is encoded in the bitstream, while `valid_bvd_x_pred_flag` (and correspondingly, `valid_bvd_y_pred_flag`) can be set to 0 to indicate that the predicted BVDx suffix (and correspondingly, the BVDy suffix) is predicted.

[0113] In S110, the suffix part is encoded for each component (for which the information indicates that the suffix part is not predicted (i.e., it is encoded)). For example, if / when valid_bvd_x_pred_flag is equal to 0, the suffix part is encoded for the x component, and if / when valid_bvd_y_pred_flag is equal to 0, the suffix part is encoded for the y component.

[0114] In this embodiment, the valid_bvd_pred_flag is not used. Prediction is performed, and in some examples, the flags valid_bvd_x_pred_flag and valid_bvd_y_pred_flag are encoded only for non-zero components. Flags (e.g., Gr0x_flag and Gr0y_flag) can be encoded first (e.g., for each component) to indicate whether the component is non-zero. If the flag is 1, the component is non-zero, and the magnitude, such as its prefix and possible suffixes, should be encoded (if no prediction is performed). If the flag is 0, it means that the component (e.g., BVDx / BVDy) is zero, and no further encoding is required for that particular component.

[0115] Figure 14A A flowchart illustrating a decoding method based on an example is provided.

[0116] In S200, the prefix portion is decoded for each (e.g., each non-zero) component of the block vector difference.

[0117] In S202, information indicating whether a suffix portion is predicted for each (e.g., each non-zero) component is decoded. In other words, first information (e.g., a first flag) indicating whether a suffix portion is predicted for the first component is decoded, and second information (e.g., a second flag) indicating whether a suffix portion is predicted for the second component is decoded for the second component. In the example, `valid_bvd_x_pred_flag` (e.g., if / when actual BVDx is non-zero) indicates whether a BVDx suffix is ​​predicted, and `valid_bvd_y_pred_flag` (e.g., if / when actual BVDy is non-zero) indicates whether a BVDy suffix is ​​predicted. If the indication is not to predict, the suffix portion is signaled in the bitstream.

[0118] In S204, based on the information decoded for this component in S202, a suffix portion is obtained for each (e.g., each non-zero) component. In other words, if the information indicates that the suffix portion should not be predicted, the suffix portion is decoded from the bitstream (S204-1); otherwise, the suffix portion is predicted (S204-2 and S204-3), as shown below. Figure 14B This is detailed for a single component. The prediction of the suffix part can be performed as on the encoder side, and includes obtaining (S204-2) a reference block by using template matching (TM) in a (e.g., predefined) search region, and obtaining (e.g., deriving or acquiring) (S204-3) a BVD from the reference block, denoted as BVD. TM In the example, BV is obtained (e.g., derived) from the reference block, and then the predicted BVD is obtained by subtracting BVP from the obtained (e.g., derived) BV. BVP is, for example, the same BVP used to obtain the actual BVD. Finally, from BVD... TM Obtain (e.g., predict) the suffix portion used for the considered component (S204-4). In other words, BVD TM_x = prefix _BVD TM_x + suffix _BVD TM_x Therefore, the predicted suffix is ​​the suffix _BVD. TM_x The same applies to the y-component. If the suffix parts of both components are predicted, S204-2 operates only once because both components use the same BVD. TM .

[0119] For example, if / when valid_bvd_x_pred_flag == 1 or valid_bvd_y_pred_flag == 1 (or both), the reference block is obtained by using TM in the predefined search area. If / when valid_bvd_x_pred_flag == 1, the BVD is obtained from the reference block. TM (Predicted BVD). Finally, the x component is derived from the predicted BVD. TM Obtain (e.g., retrieve) the suffix portion. Otherwise (i.e., valid_bvd_x_pred_flag == 0), decode the suffix portion for the x component (e.g., directly) from the bitstream. In the example, the suffix portion for the x component is decoded only for non-zero BVDx magnitudes. If / when valid_bvd_y_pred_flag == 1, obtain the BVD from the reference block. TM (Predicted BVD). Finally, the y component is derived from the predicted BVD. TM Obtain (e.g., retrieve) the suffix portion. Otherwise (i.e., valid_bvd_y_pred_flag == 0), decode the suffix portion for the y component (e.g., directly) from the bitstream. In the example, the suffix portion for the y component is decoded only for non-zero BVDy amplitudes.

[0120] You can first decode flags (e.g., Gr0x_flag and Gr0y_flag) that indicate whether the component is non-zero, for example, for each component. If the flag is 1, the component is non-zero, and the amplitude, such as its prefix and possible suffixes (if no prediction is performed), is decoded. If the flag is 0, it means that the component (e.g., BVDx / BVDy) is zero, and no further decoding is needed for that particular component.

[0121] In S206, the BVD is obtained from the prefix and suffix portions obtained for each component.

[0122] Finally, BVD can be used to reconstruct image patches (e.g., the current CU). For example, the block vector BV is obtained by adding BVD to BVP. Thus, image patches are reconstructed from reference blocks identified by BV.

[0123] Figure 15 A flowchart illustrating the encoding method according to the second example is provided. In the second example, valid_bvd_pred_flag and two new (e.g., dedicated) flags are used for suffix prediction of individual BVD components. valid_bvd_pred_flag can be encoded first, followed by valid_bvd_x_pred_flag and valid_bvd_y_pred_flag, where applicable.

[0124] In S300, the BVD is obtained (e.g., found) for the image block (e.g., the current CU), for example, by using a standard encoder search process implemented in IBC mode, such as... Figure 11 As shown. This BVD is referred to as the actual BVD in the following text.

[0125] In S302, a reference block is obtained (e.g., found), for example, by using template matching (TM) in a (e.g., predefined) search area (also called a search region). In the example, S302 is performed only if at least one vector component of the actual BVD is nonzero (i.e., if BVDx or BVDy is nonzero or both are nonzero).

[0126] In S304, the BVD is obtained (e.g., derived or acquired) from the reference block obtained in S302, denoted as BVD. TM For example, only if at least one vector component of the actual BVD is nonzero. BVD TM This is referred to below as the predicted BVD. In the example, the BV is obtained (e.g., derived) from the reference block, and then the predicted BVD is obtained by subtracting the BVP from the obtained (e.g., derived) BV. TM The BVP is, for example, the same as the BVP used to obtain the actual BVD.

[0127] In S306, a prefix portion is encoded for each (e.g., each non-zero) component of the block vector difference (i.e., the actual BVD). In the example, the prefix portion of the x component (e.g., if non-zero) is encoded, and the prefix portion of the y component (e.g., if non-zero) is encoded.

[0128] In S308, when both BVDx and BVDy components are nonzero (S308-1), based on BVD and BVD... TM The comparison encodes information indicating whether the suffix portions of both components are predicted (e.g., a single flag). If the information indicates that the suffix portions of both components are predicted, then no more information is encoded for BVDx and BVDy.

[0129] In cases where both components are non-zero and the information in S308-1 indicates that at least one of the suffix parts of BVDx and BVDy cannot be correctly predicted (e.g., BVDx can be predicted from BVD...), the suffix part of BVDx and BVDy cannot be correctly predicted. TM Correct prediction, but BVDy cannot be obtained from BVD TM In the case of correct prediction (or vice versa), or when only one component is non-zero, information (e.g., a flag) indicating whether the non-zero component is a predicted suffix part is encoded (S308-2) for that component. The value of this information is based on the non-zero component of BVD and BVD. TM Comparison of the corresponding components.

[0130] For example, if / when the actual BVDx is non-zero and the actual BVDy is non-zero, compare the actual BVD with the predicted BVD, for example, compare the x and y components of the vector.

[0131] If / when both vector components are identical (e.g., the comparison / prediction is marked as correct), then valid_bvd_pred_flag is set to 1, indicating that the BVD suffix portion is predicted for both components. Otherwise, valid_bvd_pred_flag is set to 0, indicating that the BVD suffix may exist in the bitstream; for example, whether the suffix exists in the bitstream depends on further comparisons. If at least one vector component of the actual BVD is zero, valid_bvd_pred_flag is also set to 0, and a component-by-component comparison is performed in the next step.

[0132] If valid_bvd_pred_flag == 0 and the actual BVDx is non-zero, compare the actual BVDx with the predicted BVDx.

[0133] If the actual BVDx and the predicted BVDx are the same, then valid_bvd_x_pred_flag is set to 1, indicating that the BVDx suffix should be predicted; otherwise, valid_bvd_x_pred_flag is set to 0, indicating that the BVDx suffix exists in the bitstream, i.e., no prediction is performed. If the actual BVDx is zero, valid_bvd_x_pred_flag is also set to 0.

[0134] If valid_bvd_pred_flag == 0 and valid_bvd_x_pred_flag == 0, and if the actual BVDy is non-zero, compare the actual BVDy with the predicted BVDy. If the actual BVDy and the predicted BVDy are the same, set valid_bvd_y_pred_flag to 1 to indicate that the predicted BVDy suffix exists; otherwise, set valid_bvd_y_pred_flag to 0 to indicate that the BVDy suffix exists in the bitstream, i.e., no prediction is performed. If the actual BVDy is zero, valid_bvd_y_pred_flag is also set to 0.

[0135] If BVDx and BVDy are both non-zero, encode valid_bvd_pred_flag; otherwise (i.e., if at least one component is zero), do not encode valid_bvd_pred_flag (e.g., skip encoding valid_bvd_pred_flag).

[0136] If valid_bvd_pred_flag == 0 and if BVDx is non-zero, encode valid_bvd_x_pred_flag; otherwise (i.e., BVDx is zero), do not encode valid_bvd_x_pred_flag (e.g., skip encoding valid_bvd_x_pred_flag).

[0137] If valid_bvd_pred_flag == 0, and if valid_bvd_x_pred_flag == 0 and if BVDy is non-zero, encode valid_bvd_y_pred_flag; otherwise, do not encode valid_bvd_y_pred_flag (e.g., skip encoding valid_bvd_y_pred_flag).

[0138] In S310, the suffix part is encoded for each component (for which the information indicates that the suffix part is not predicted).

[0139] If BVDx is non-zero and valid_bvd_pred_flag == 0 and valid_bvd_x_pred_flag == 0, then the suffix part of the x component is encoded.

[0140] If BVDy is non-zero and valid_bvd_pred_flag == 0 and valid_bvd_y_pred_flag == 0, then the suffix part of the y component is encoded.

[0141] In this example, therefore, valid_bvd_pred_flag is only encoded if both BVDx and BVDy are nonzero. If at least one component has a zero amplitude, valid_bvd_pred_flag can be skipped (e.g., not encoded into the bitstream), and the nonzero component suffix prediction can be handled with other flags, valid_bvd_x_pred_flag or valid_bvd_y_pred_flag, depending on the component being considered.

[0142] In this example, valid_bvd_x_pred_flag and valid_bvd_y_pred_flag can only be encoded if / if valid_bvd_pred_flag equals 0. valid_bvd_pred_flag == 1 means that the suffix parts of BVDx and BVDy are correctly predicted. Signaling only valid_bvd_pred_flag (when equal to 1) represents optimal signaling overhead because there is no need to send the additional flags valid_bvd_x_pred_flag and valid_bvd_y_pred_flag. In fact, they provide no additional information in this case.

[0143] When flags are not encoded (e.g., valid_bvd_pred_flag, valid_bvd_x_pred_flag, or valid_bvd_y_pred_flag), their value is assumed to be 0.

[0144] Table 2 illustrates examples of possible combinations of flags, where "X" indicates that the flag is not encoded (and therefore not decoded).

[0145]

[0146] In the example above, `valid_bvd_x_pred_flag` is encoded before `valid_bvd_y_pred_flag`. However, `valid_bvd_y_pred_flag` is only encoded if `valid_bvd_x_pred_flag` equals 0. In fact, the case where both flags are 1 is equivalent to the case where `valid_bvd_pred_flag` equals 1. Since `valid_bvd_pred_flag` is encoded first (and therefore decoded first), the case where two other flags are both 1 is not allowed. Therefore, bits are saved. So, if `valid_bvd_x_pred_flag == 1`, it implicitly implies that the other flag, `valid_bvd_y_pred_flag`, is 0. However, this is convention, and in another example, valid_bvd_y_pred_flag can be encoded before valid_bvd_x_pred_flag, in which case valid_bvd_x_pred_flag is only encoded if valid_bvd_y_pred_flag == 0.

[0147] It should be noted that the examples in the table above (Table 2) illustrate the case where both components are non-zero. There are other examples where at least one component is zero.

[0148] Figure 16 A flowchart illustrating a decoding method based on an example is provided.

[0149] In S400, the prefix portion is decoded for each component of the block vector difference, for example, only for non-zero BVD magnitudes.

[0150] In S402, if both BVDx and BVDy components are non-zero (S402-1), the decoding process indicates whether to predict the suffix portion of both components (e.g., a single flag such as valid_bvd_pred_flag). If the information indicates that the suffix portion of both components is predicted (e.g., valid_bvd_pred_flag == 1), the method continues in S404.

[0151] In the case where both components are non-zero but the information in S402-1 indicates that at least one of the suffix parts of BVDx and BVDy cannot be correctly predicted (e.g., valid_bvd_pred_flag == 0), or in the case where only one component (e.g., only one component) is non-zero, the decoding (S402-2) for that non-zero component indicates whether to predict the information (e.g., the flag) of the suffix part for that component.

[0152] In S404, based on the information decoded in S402, a suffix portion is obtained for each (e.g., each non-zero) component. In other words, if the information indicates that no prediction of the suffix portion is to be made, the suffix portion is decoded from the bitstream; otherwise, it is as follows. Figure 14B The predicted suffix portion is shown. The predicted suffix portion is identical to the encoder side and includes obtaining a reference block by using template matching (TM) in a (e.g., predefined) search region, and obtaining (e.g., deriving or acquiring) the BVD from the reference block, denoted as BVD. TM In the example, BV is obtained (e.g., derived) from the reference block, and then the predicted BVD is obtained by subtracting BVP from the obtained (e.g., derived) BV. BVP is, for example, the same BVP used to obtain the actual BVD. Finally, from BVD... TM Obtain the suffix.

[0153] In the example, if / when both the actual BVDx and the actual BVDy are nonzero, the valid_bvd_pred_flag is decoded. This flag indicates whether both the BVDx and BVDy suffixes are predicted (if not, further steps are needed to obtain information about whether they are predicted or exist in the bitstream). Otherwise, valid_bvd_pred_flag is set to 0 (only one component is nonzero, or both are zero, so valid_bvd_pred_flag is not decoded).

[0154] If valid_bvd_pred_flag equals 1, the reference block is obtained by using TM in the predefined search region. The BVD is then obtained from the reference block. TM (Predicted BVD). Finally, the x and y components are both derived from the predicted BVD. TM To obtain (e.g., to acquire) the suffix part.

[0155] Otherwise (i.e., if / when valid_bvd_pred_flag is equal to 0 and at least one component is non-zero), further steps are required.

[0156] If BVDx is non-zero, decode valid_bvd_x_pred_flag, which indicates whether the BVDx suffix exists (and is therefore predicted). Otherwise, set valid_bvd_x_pred_flag=0.

[0157] If BVDy is non-zero and valid_bvd_x_pred_flag is equal to 0, then decode valid_bvd_y_pred_flag, which indicates whether the BVDy suffix exists (and is therefore predicted). Otherwise, set valid_bvd_y_pred_flag = 0.

[0158] If BVDx is non-zero and valid_bvd_pred_flag == 0 and valid_bvd_x_pred_flag == 0, then decode the suffix portion for the x component. Otherwise, obtain a reference block by using a TM in a (e.g., predefined) search region. Obtain the BVD from the reference block. TM (Predicted BVD). Finally, the x component is derived from the predicted BVD. TM To obtain (e.g., to acquire) the suffix part.

[0159] If BVDy is non-zero and valid_bvd_pred_flag == 0 and valid_bvd_y_pred_flag == 0, then decode the suffix portion of the y component. Otherwise, obtain a reference block by using a TM in a (e.g., predefined) search region. Obtain the BVD from the reference block. TM (Predicted BVD). Finally, the y component is derived from the predicted BVD. TM To obtain (e.g., to acquire) the suffix part.

[0160] In S406, the BVD is obtained from the prefix and suffix portions obtained for each component.

[0161] Finally, BVD can be used to reconstruct image patches (e.g., the current CU). For example, the block vector BV is obtained by adding BVD to BVP. Thus, image patches are reconstructed from reference blocks identified by BV.

[0162] exist Figure 15 and 16 In other examples where this method applies, using the valid_bvd_pred_flag can be subject to additional constraints. In the example, this flag can be used to impose conditional restrictions based on the prefix values ​​of the BVDx and BVDy components.

[0163] In one variant, valid_bvd_pred_flag can be used (e.g., for encoding / decoding) when (e.g., only if) both prefix_BVDx and prefix_BVDy are greater than or equal to the predefined value BVD_PREFIX_MIN_SIZE (e.g., BVD_PREFIX_MIN_SIZE can be set to any prefix value indicated in Table 1).

[0164] In the second variant, valid_bvd_pred_flag can be used (e.g., encoding / decoding) when (for example, only when) at least one of prefix_BVDx or prefix_BVDy is greater than or equal to the predefined value BVD_PREFIX_MIN_SIZE.

[0165] In the third variant, the valid_bvd_pred_flag can be used (e.g., for encoding / decoding) when (for example, only if) prefix_BVDx is greater than or equal to the predefined value BVD_PREFIX_MIN_SIZE_X and prefix_BVDy is greater than or equal to the predefined value BVD_PREFIX_MIN_SIZE_Y. In the example, for instance, BVD_PREFIX_MIN_SIZE_X and BVD_PREFIX_MIN_SIZE_Y can be set to any prefix value indicated in Table 1, and BVD_PREFIX_MIN_SIZE_X may or may not be equal to BVD_PREFIX_MIN_SIZE_Y.

[0166] In the fourth variant, valid_bvd_pred_flag can be used (e.g., for encoding / decoding) when (e.g., only when) the sum of two prefixes (e.g., prefix_BVDx and prefix_BVDy) is greater than or equal to the predefined value BVD_PREFIX_MIN_SIZE.

[0167] exist Figures 13 to 16 In other examples where the method applies, the use of valid_bvd_x_pred_flag and valid_bvd_y_pred_flag may be subject to additional constraints.

[0168] In one variant, valid_bvd_x_pred_flag can be used (e.g., encoding / decoding) when (for example, only if) prefix_BVDx is greater than or equal to the predefined value BVD_PREFIX_MIN_SIZE_X.

[0169] In another variant, valid_bvd_y_pred_flag can be used (e.g., for encoding / decoding) when (for example, only if) prefix_BVDy is greater than or equal to the predefined value BVD_PREFIX_MIN_SIZE_Y.

[0170] This document is not limited to ECM, VVC, or HEVC, and can be applied to, for example, other standards and recommendations, as well as any extensions of such standards and recommendations. Unless otherwise stated or technically impractical, the aspects described in this application may be used alone or in combination.

[0171] Various numerical values ​​are used in this application. Specific values ​​are used for illustrative purposes, and the aspects described are not limited to these specific values.

[0172] It should be noted that the syntax elements used in this article (such as terms in equations and algorithms, signal (e.g., flag) labels / names, such as valid_bvd_pred_flag, valid_bvd_x_pred_flag, valid_bvd_y_pred_flag, etc.) are descriptive terms. As descriptive terms, they do not preclude the use of other syntax element names.

[0173] Various implementations involve decoding. As used herein, "decoding" can encompass all or part of the process performed on a received encoded sequence to produce a final output suitable for display. In various embodiments, such processes include one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also include, or alternatively include, processes performed by the decoder in the various implementations described herein, such as decoding block vector differences.

[0174] As a further example, in one embodiment, "decoding" refers only to entropy decoding; in another embodiment, "decoding" refers only to differential decoding; in yet another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding; and in yet another embodiment, "decoding" refers to the entire image reconstruction process, including entropy decoding. Whether the phrase "decoding process" is intended to specifically refer to a subset of operations or to refer to the broader decoding process will be clear based on the specific context of the description and is believed to be well understood by those skilled in the art.

[0175] Various implementations involve encoding. Similar to the discussion of “decoding” above, the term “encoding” as used herein can encompass all or part of the processes performed on an input video sequence to produce an encoded bitstream. In various embodiments, such processes include one or more processes typically performed by an encoder, such as partitioning, differential coding, transform, quantization, and entropy coding. In various embodiments, such processes also include, or alternatively include, processes performed by the encoder in the various implementations described herein, such as coded block vector difference.

[0176] As a further example, in one embodiment, “encoding” refers only to entropy encoding; in another embodiment, “encoding” refers only to differential encoding; and in yet another embodiment, “encoding” refers to a combination of differential and entropy encoding. Whether the phrase “encoding process” is intended to specifically refer to a subset of operations or to refer to a broader encoding process will be clear based on the specific context of the description and is believed to be well understood by those skilled in the art.

[0177] This disclosure describes various types of information that can be transmitted or stored, such as syntax. This information can be packaged or arranged in various ways, including those common in video standards, such as placing the information in SPS, PPS, NAL units, headers (e.g., NAL unit headers or stripe headers), or SEI messages. Other methods are also available, including those common in system-level or application-level standards, such as placing the information in one or more of the following: a. SDP (Session Description Protocol), a format for describing multimedia communication sessions for the purposes of session announcement and session invitation, such as as described in RFCs and used in conjunction with RTP (Real-Time Transport Protocol) transmission.

[0178] b. DASH MPD (Media Presentation Description) descriptor (e.g., used in DASH and transmitted via HTTP), which is associated with a representation or set of representations to provide additional features of the content representation.

[0179] c. RTP header extensions, for example, used during RTP streaming.

[0180] d. ISI Basic Media File Format, used in, for example, OMAF, and using boxes as object-oriented building blocks defined by unique type identifiers and lengths, also referred to as 'atoms' in some specifications.

[0181] e. HLS (HTTP Live Streaming) manifests, transmitted via HTTP. A manifest can, for example, be associated with a version or set of versions of content to provide characteristics of that version or set of versions.

[0182] When a diagram is presented as a flowchart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a diagram is presented as a block diagram, it should be understood that it also provides a flowchart of the corresponding method / process.

[0183] Some implementations may involve rate-distortion optimization. In particular, during encoding, a balance or trade-off between rate and distortion is typically considered under constraints of computational complexity. Rate-distortion optimization is generally formulated as minimizing a rate-distortion function that is a weighted sum of rate and distortion. Different approaches exist to address the rate-distortion optimization problem. For example, these approaches may be based on extensive testing of all encoding options, including all considered modes or encoding parameter values, and a complete evaluation of their encoding costs and the associated distortion of the reconstructed signal after encoding and decoding. Faster methods can also be used to save encoding complexity, particularly by calculating approximate distortion based on the predicted or predicted residual signal (rather than the reconstructed signal). A hybrid of these approaches can also be used, for example, using approximate distortion only for some possible encoding options and full distortion for others. Other methods evaluate only a subset of possible encoding options. More generally, many methods employ any of a variety of techniques to perform optimization, but optimization is not necessarily a complete evaluation of encoding costs and associated distortion.

[0184] The embodiments and aspects described herein can be implemented, for example, in methods or processes, apparatuses, software programs, data streams, or signals. Even if discussed only in the context of a single implementation (e.g., discussed only as a method), implementations of the discussed features can be implemented in other forms (e.g., apparatuses or programs). Apparatuses can be implemented, for example, in suitable hardware, software, and firmware. Methods can be implemented, for example, in a processor, where processor refers generally to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as computers, mobile phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0185] The references to "an embodiment," "an embodiment," "an implementation," or "an implementation," and other variations, mean that a particular feature, structure, characteristic, etc., described in connection with that embodiment is included in at least one embodiment. Therefore, the phrases "in an embodiment," "in an embodiment," "in an implementation," or "in an implementation," and any other variations appearing in various places in this application do not necessarily all refer to the same embodiment.

[0186] Furthermore, this application may involve "determining" various types of information. Determining information may include one or more of, for example, estimation information, calculation information, prediction information, or information retrieved from memory.

[0187] Furthermore, this application may involve "accessing" various types of information. Accessing information may include one or more of the following: receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0188] Furthermore, this application may relate to "receiving" various types of information. "Receiving," like "accessing," is intended to be a broad term. Receiving information may include, for example, accessing information or retrieving information (e.g., from memory). Moreover, "receiving" generally involves in some way operations such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0189] It should be understood that the use of any of the following “ / ”, “and / or”, and “…at least one of”, such as in the cases of “A / B”, “A and / or B”, and “at least one of A and B”, is intended to cover selecting only the first listed option (A), or only the second listed option (B), or selecting both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C”, this wording is intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or selecting all three options (A, B, and C). This will be clear to those skilled in the art and related fields and can be extended to any number of listed items.

[0190] Similarly, as used herein, the term "signal" refers to, among other things, instructing the corresponding decoder to do something. For example, in some embodiments, the encoder signals the presence of a suffix portion of the BVD in the bitstream, and therefore whether it must be decoded or predicted. Thus, in one embodiment, the same parameter is used on both the encoder and decoder sides. Therefore, for example, the encoder can transmit a specific parameter (explicit signal) to the decoder so that the decoder can use the same specific parameter. Conversely, if the decoder already has that specific parameter as well as other parameters, it can simply inform the decoder to know and select that specific parameter without transmitting it (implicit signal). Bit savings are achieved in many embodiments by avoiding the transmission of any actual function. It should be understood that signal can be implemented in various ways. For example, in many embodiments, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the foregoing refers to the verb form of the term "signal," the term "signal" can also be used as a noun herein.

[0191] As will be apparent to those skilled in the art, implementations can generate signals in various formats to carry information that can be stored or transmitted. This information may include, for example, instructions for performing a method, or data generated by one of the implementations. For example, the signal may be formatted to carry a bitstream of the embodiments described. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal can be transmitted via a variety of different wired or wireless links. The signal may be stored on a processor-readable medium.

[0192] This document describes numerous examples. Features of the examples may be provided individually or in any combination across multiple claim classes and types. Furthermore, examples may include one or more features, apparatuses, or aspects described herein, individually or in any combination, across multiple claim classes and types. For example, features described herein may be implemented in a bitstream or signal that includes information generated as described herein. This information may allow a decoder to decode the bitstream according to any described embodiment, and / or an encoder to encode the bitstream. For example, features described herein may be implemented by creating and / or transmitting and / or receiving and / or decoding a bitstream or signal. For example, features described herein may be implemented by a method, process, apparatus, medium storing instructions, medium storing data, or signal. For example, features described herein may be implemented by a television, set-top box, mobile phone, tablet computer, or other electronic device performing decoding (and correspondingly encoding). The television, set-top box, mobile phone, tablet computer, or other electronic device may display (e.g., using a monitor, screen, or other type of display) the resulting image (e.g., an image reconstructed from the residual of the video bitstream). The television, set-top box, mobile phone, tablet computer, or other electronic device may receive a signal including the encoded image and perform decoding.

[0193] Many embodiments have been described. The features of these embodiments may be provided individually or in any combination across a variety of claim classes and types.

[0194] A decoding method is disclosed, which includes: Decode the prefix portion for each (e.g., each non-zero) component of the vector difference; For each (e.g., each non-zero) component, the decoding indicator determines whether to predict the suffix information for that component; For each (e.g., each non-zero) component, the suffix portion is obtained based on the information decoded for that component; and Obtain the vector difference from the prefix and suffix parts.

[0195] A decoding method is disclosed, which includes: Decode the prefix portion for each (e.g., each non-zero) component of the vector difference; In response to determining that both components are non-zero, the decoding indicates whether to predict the suffix portion of both components; In response to determining that a component (e.g., only one) is nonzero or that the decoded information indicates that at least one suffix part cannot be predicted, for each component (e.g., each nonzero component), the decoding indicates whether to predict the information of the suffix part for that component; and The suffix portion for each (e.g., each non-zero) component is obtained based on the decoded information; Obtain the vector difference from the prefix and suffix parts.

[0196] A decoding method is disclosed, which includes: Decode the prefix portion for each (e.g., each non-zero) component of the vector difference; If both components are non-zero, the decoding indicates whether to predict the suffix portion of both components. If a component (e.g., only one) is non-zero or the decoded information indicates that at least one suffix part is not predicted, then for each (e.g., each non-zero) component, the decoding indicates whether the information for the suffix part is predicted for that component; Based on the decoded information, obtain the suffix portion for that component (e.g., each non-zero component); and Obtain the vector difference from the prefix and suffix parts.

[0197] An encoding method is disclosed, which includes: Obtain the vector difference for the image patch; Obtain the reference block through template matching; Obtain the prediction vector difference from the reference block; Encode a prefix portion for each (e.g., each non-zero) component of the vector difference; For each (e.g., each non-zero) component, information indicating whether it is a predicted suffix part of that component is encoded based on a comparison between the vector difference and the predicted vector difference; and For each component whose information indicates that the suffix part is not predicted, the suffix part is encoded.

[0198] An encoding method is disclosed, which includes: Obtain the vector difference for the image patch; Obtain the reference block through template matching; Obtain the prediction vector difference from the reference block; Encode the prefix portion for each component of the vector difference; In response to determining that both components are non-zero, information indicating whether to predict the suffix portion of both components is encoded. In response to determining that a component (e.g., only one) is nonzero or that the information indicates that at least one suffix part should not be predicted, for each component (e.g., each nonzero component), information indicating whether a suffix part should be predicted for that component is encoded; and For each component whose information indicates that the suffix part is not predicted, the suffix part is encoded.

[0199] An encoding method is disclosed, which includes: Obtain the vector difference for the image patch; Obtain the reference block through template matching; Obtain the prediction vector difference from the reference block; Encode the prefix portion for each component of the vector difference; If both components are non-zero, the code (S308-1) indicates whether the suffix part of both components is predicted. If a component (e.g., only one) is nonzero or the information indicates that at least one suffix part is not predicted, then for each component (e.g., each nonzero component), information indicating whether a suffix part is predicted for that component is encoded; and For each component whose information indicates that the suffix part is not predicted, the suffix part is encoded.

[0200] In the example, the vector difference is the motion vector difference or the block vector difference.

[0201] In the example, obtaining the suffix based on the decoded information includes, for components: Predict the suffix portion if the decoded information indicates that the suffix portion has not been signaled; and Otherwise, decode the suffix.

[0202] In the example, the predicted suffix includes: Obtain the reference block through template matching; Obtain the prediction vector difference from the reference block; and The suffix is ​​obtained from the difference in the predicted vectors.

[0203] In the example, obtaining the predicted vector difference from the reference block involves subtracting the vector predictor from the vector identifying the reference block.

[0204] In the example, the vector predictor is obtained by decoding (correspondingly, encoding) the index of the vector predictor in the candidate vector list.

[0205] In the example, the information indicating whether to predict the suffix of both components is decoded (and correspondingly encoded) only if the prefix of both components is greater than a value.

[0206] In the example, the information indicating whether to predict the suffixes of both components is decoded (correspondingly encoded) only if at least one prefix portion of the component is greater than a value.

[0207] In the example, the information indicating whether to predict the suffixes of both components is decoded (and correspondingly encoded) only if the prefix of the first component is greater than the first value and the prefix of the second component is greater than the second value.

[0208] In the example, the information indicating whether to predict the suffix of both components is decoded (and correspondingly encoded) only if the sum of the prefixes of the two components is greater than a value.

[0209] In the example, information indicating whether the suffix part of the component is predicted is only decoded (and correspondingly encoded) if the prefix part of the component is greater than a value.

[0210] A decoding apparatus is disclosed, comprising one or more processors and at least one memory coupled to the one or more processors, wherein the one or more processors are configured to perform any decoding method.

[0211] An encoding apparatus is disclosed, comprising one or more processors and at least one memory coupled to the one or more processors, wherein the one or more processors are configured to perform any encoding method.

[0212] A computer program is disclosed, comprising program code instructions for implementing any encoding or decoding method when executed by a processor.

Claims

1. A decoding method, comprising: Decode the prefix portion (S200) for each non-zero component of the vector difference; For each non-zero component, decoding (S202) indicates whether it is information of the predicted suffix part of the component; For each non-zero component, the suffix portion is obtained based on the information decoded for the component (S204); as well as (S206) The vector difference is obtained from the prefix portion and the suffix portion.

2. A decoding method, comprising: Decode the prefix portion (S400) for each non-zero component of the vector difference; If both components are non-zero, the decoding (S402-1) indicates whether to predict the suffix portion of both components. If a component is non-zero or the decoded information indicates that at least one suffix part is not predicted, then for each non-zero component, the decoding (S402-2) indicates whether the information of the suffix part is predicted for said component; as well as Based on the decoded information, obtain (S404) the suffix portion for the component; as well as The vector difference is obtained from the prefix portion and the suffix portion (S406).

3. The method as described in claim 1 or 2, wherein the vector difference is a motion vector difference or a block vector difference.

4. The method of any one of claims 1 to 3, wherein obtaining the suffix portion based on the decoded information comprises, for components: If the decoded information indicates that the suffix portion has not been signaled, predict the suffix portion; and Otherwise, decode the suffix portion.

5. The method of claim 4, wherein predicting the suffix portion comprises: Obtain the reference block through template matching; The prediction vector difference is obtained from the reference block; as well as The suffix portion is obtained from the difference in the prediction vectors.

6. The method of claim 5, wherein obtaining the prediction vector difference from the reference block comprises: Subtract the vector predictor from the vector that identifies the reference block.

7. The method of claim 6, wherein the vector predictor is obtained by decoding the index of the vector predictor in the candidate vector list.

8. The method of claim 2, wherein the information indicating whether to predict the suffix portion of both components is decoded only if the prefix portions of both components are greater than a value.

9. The method of claim 2, wherein the information indicating whether to predict the suffix portions of both components is decoded only if at least one prefix portion of the components is greater than a value.

10. The method of claim 2, wherein the information indicating whether to predict the suffix portions of both components is decoded only if the prefix portion of the first component is greater than the first value and the prefix portion of the second component is greater than the second value.

11. The method of claim 2, wherein the information indicating whether to predict the suffix portion of both components is decoded only if the sum of the prefix portions of the two components is greater than a value.

12. The method of claim 1 or 2, wherein information indicating whether it is a component prediction suffix is ​​decoded only if the prefix portion of the component is greater than a value.

13. An encoding method, comprising: Obtain the (S100) vector difference for the image patch; The reference block (S102) is obtained through template matching; The prediction vector difference is obtained from the reference block (S104); Encode a prefix portion (S106) for each non-zero component of the vector difference; For each non-zero component, information indicating whether it is a predicted suffix part of the component is encoded (S108) based on the comparison between the vector difference and the predicted vector difference; as well as For each component whose information indicates that the suffix portion is not predicted, the suffix portion is encoded (S110).

14. An encoding method, comprising: Obtain the (S300) vector difference for the image patch; The reference block (S302) is obtained through template matching; The prediction vector difference is obtained from the reference block (S304); Encode the prefix portion (S306) for each component of the vector difference; If both components are non-zero, the code (S308-1) indicates whether the suffix part of both components is predicted. If a component is non-zero or the information indicates that at least one suffix part is not predicted, then for each non-zero component, the encoding (S308-2) indicates whether the information for predicting the suffix part is for that component; as well as For each component whose information indicates that the suffix portion is not predicted, the suffix portion is encoded (S310).

15. The method of claim 13 or 14, wherein the vector difference is a motion vector difference or a block vector difference.

16. The method of claim 13 or 14, wherein obtaining the prediction vector difference from the reference block comprises: Subtract the vector predictor from the vector that identifies the reference block.

17. The method of claim 14, wherein the information indicating whether to predict the suffix portion of both components is encoded only if the prefix portions of both components are greater than a value.

18. The method of claim 14, wherein the information indicating whether to predict the suffix portions of both components is encoded only if at least one prefix portion of the components is greater than a value.

19. The method of claim 14, wherein the information indicating whether to predict the suffix portions of both components is encoded only if the prefix portion of the first component is greater than the first value and the prefix portion of the second component is greater than the second value.

20. The method of claim 14, wherein the information indicating whether to predict the suffix portion of both components is encoded only if the sum of the prefix portions of the two components is greater than a value.

21. The method of claim 13 or 14, wherein information indicating whether it is a predicted suffix portion of the component is encoded only if the prefix portion of the component is greater than a value.

22. A decoding apparatus comprising one or more processors and at least one memory coupled to the one or more processors, wherein the one or more processors are configured to perform the method as claimed in any one of claims 1-12.

23. An encoding apparatus comprising one or more processors and at least one memory coupled to the one or more processors, wherein the one or more processors are configured to perform the method as claimed in any one of claims 13-21.

24. A computer program comprising program code instructions for implementing the method according to any one of claims 1-12 when executed by a processor.

25. A computer-readable storage medium having stored thereon instructions for implementing the method as described in any one of claims 13-21.

Images