Method and apparatus for intra block copy and intra template matching

Abstract

Methods, apparatus, and non-transitory computer-readable storage media for video decoding and encoding are provided. A method for video decoding includes: obtaining, by a decoder, a current block encoded in a geometric partition mode (GPM); obtaining, by a decoder, a first intra block copy (IBC) prediction and a second IBC prediction based on the partition using GPM for the current block; receiving, by a decoder, a coding mode of the current block; and obtaining, by a decoder, a final prediction of the current block using the first IBC prediction, the second IBC prediction, and a coding mode of the current block.

Description

Cross Reference to Related Applications

[0001] This application is based on and claims priority to U.S. Provisional Application No. 63 / 472,250, filed June 9, 2023, entitled “Methods and Devices for Intra Block Copy and Intra Template Matching,” the entire disclosure of which is incorporated by reference. TECHNICAL FIELD

[0002] The present disclosure relates to video coding and compression, and in particular, but not exclusively, to methods and apparatuses to improve coding efficiency of Intra Block Copy (IBC) and Intra Template Matching Prediction (Intra TMP). BACKGROUND

[0003] Various electronic devices (e.g., digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video gaming devices, smart phones, video teleconferencing devices, video streaming devices, etc.) support digital video. Electronic devices transmit and receive, or otherwise communicate, digital video data over communication networks, and / or store digital video data on storage devices. Because the bandwidth capacity of communication networks and the memory storage capacity of storage devices are limited, video data can be compressed prior to being communicated or stored using a video coding standard. For example, video coding standards include Versatile Video Coding (VVC), Joint Exploration Test Model (JEM), High Efficiency Video Coding (HEVC / H.265), Advanced Video Coding (AVC / H.264), Moving Picture Experts Group (MPEG) codecs, etc. Video coding typically employs prediction methods (e.g., inter-prediction, intra-prediction, etc.) that take advantage of redundancy inherent in video data. Video coding aims to compress video data into a form that uses a lower bit rate, while avoiding or minimizing losses in video quality. SUMMARY

[0004] The present disclosure provides examples of techniques related to improving Intra Block Copy methods in video encoding or decoding processes.

[0005] According to a first aspect of the disclosure, a video decoding method is provided. The method comprises: obtaining, by a decoder, a current block encoded with a geometric partition mode (GPM); obtaining, by the decoder, a first intra block copy (IBC) prediction and a second IBC prediction based on a partition of the current block using the GPM; receiving, by the decoder, an encoding mode of the current block; and obtaining, by the decoder, a final prediction of the current block using the first IBC prediction, the second IBC prediction, and the encoding mode of the current block.

[0006] According to a second aspect of the disclosure, a video encoding method is provided. The method comprises: obtaining, by an encoder, a current block encoded with a geometric partition mode (GPM); obtaining, by the encoder, a first intra block copy (IBC) prediction and a second IBC prediction based on a partition of the current block using the GPM; setting, by the encoder, an encoding mode of the current block; obtaining, by the encoder, a final prediction of the current block using the first IBC prediction, the second IBC prediction, and the encoding mode of the current block; and generating, by the encoder, a bitstream based on the final prediction.

[0007] According to a third aspect of the disclosure, a video decoding method is provided. The method comprises: for applying an intra block copy (IBC) merge mode or an IBC adaptive motion vector prediction (AMVP) mode to a current block, obtaining, by a decoder, an IBC candidate from non-adjacent neighboring blocks according to a scanning rule corresponding to the IBC merge mode or the IBC AMVP mode, wherein the non-adjacent neighboring blocks are at least one block distance away from the current block and are located in multiple directions relative to the current block, and the scanning rule is obtained based on the distance and the directions; and obtaining, by the decoder, a prediction of the current block by applying the IBC merge mode or the IBC AMVP mode based on the IBC candidate.

[0008] According to a fourth aspect of the disclosure, a video encoding method is provided. The method comprises: for applying an intra block copy (IBC) merge mode or an IBC adaptive motion vector prediction (AMVP) mode to a current block, obtaining, by an encoder, an IBC candidate from non-adjacent neighboring blocks according to a scanning rule corresponding to the IBC merge mode or the IBC AMVP mode, wherein the non-adjacent neighboring blocks are at least one block distance away from the current block and are located in multiple directions relative to the current block, and the scanning rule is obtained based on the distance and the directions; obtaining, by the encoder, a prediction of the current block by applying the IBC merge mode or the IBC AMVP mode based on the IBC candidate; and generating, by the encoder, a bitstream based on the prediction.

[0009] According to a fifth aspect of the present disclosure, a method of video decoding is provided. The method comprises: obtaining, by a decoder, an ordered list of intra block copy (IBC) candidates for applying an IBC merge mode or an IBC adaptive motion vector prediction (AMVP) mode to a current block; inserting, by the decoder, a spatial non-adjacent candidate into the ordered list of IBC candidates after a spatial adjacent candidate or a history-based block vector prediction (BVP) candidate; and obtaining, by the decoder, a prediction of the current block by applying the IBC merge mode or the IBC AMVP mode based on the ordered list of IBC candidates.

[0010] According to a sixth aspect of the present disclosure, a method of video encoding is provided. The method comprises: obtaining, by an encoder, an ordered list of intra block copy (IBC) candidates for applying an IBC merge mode or an IBC adaptive motion vector prediction (AMVP) mode to a current block; inserting, by the encoder, a spatial non-adjacent candidate into the ordered list of IBC candidates after a spatial adjacent candidate or a history-based block vector prediction (BVP) candidate; obtaining, by the encoder, a prediction of the current block by applying the IBC merge mode or the IBC AMVP mode based on the ordered list of IBC candidates; and generating, by the encoder, a bitstream based on the prediction.

[0011] According to a seventh aspect of the present disclosure, an apparatus for video decoding is provided. The apparatus can include one or more processors and a memory coupled to the one or more processors and configured to store instructions executable by the one or more processors. Moreover, the one or more processors, upon execution of the instructions, are configured to perform a method according to the first aspect, the third aspect, or the fifth aspect.

[0012] According to an eighth aspect of the present disclosure, an apparatus for video encoding is provided. The apparatus can include one or more processors and a memory coupled to the one or more processors and configured to store instructions executable by the one or more processors. Moreover, the one or more processors, upon execution of the instructions, are configured to perform a method according to the second aspect, the fourth aspect, or the sixth aspect.

[0013] According to a ninth aspect of the present disclosure, a non-transitory computer- readable storage medium for storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform a method according to the first aspect, the third aspect, or the fifth aspect.

[0014] According to a tenth aspect of the disclosure, there is provided a non-transitory computer-readable storage medium for storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform a method according to the second, fourth or sixth aspect.

[0015] According to an eleventh aspect of the disclosure, there is provided a non-transitory computer-readable storage medium for storing a bitstream to be decoded by a method according to the first, third or fifth aspect.

[0016] According to a twelfth aspect of the disclosure, there is provided a non-transitory computer-readable storage medium for storing a bitstream generated by a method according to the second, fourth or sixth aspect. BRIEF DESCRIPTION OF DRAWINGS

[0017] More specific descriptions of examples of the disclosure will be presented by referring to specific examples illustrated in the attached drawings. In view of the fact that these drawings only depict some examples and therefore should not be considered as limiting the scope, these examples will be described and explained in more detail by using the attached drawings.

[0018] Figure 1 is a block diagram illustrating an exemplary system for encoding and decoding video blocks according to some examples of the disclosure.

[0019] Figure 2 is a block diagram illustrating an exemplary video encoder according to some examples of the disclosure.

[0020] Figure 3 is a block diagram illustrating an exemplary video decoder according to some examples of the disclosure.

[0021] Figures 4A to 4E is a block diagram illustrating how a frame is recursively partitioned into multiple video blocks having different sizes and shapes according to some examples of the disclosure.

[0022] Figure 5 is a diagram illustrating the position of spatial candidates according to some examples of the disclosure.

[0023] Figure 6 is a diagram illustrating pairs of candidates that are considered for redundancy checking of spatial candidates according to some examples of the disclosure.

[0024] Figure 7 is a diagram illustrating scaling of motion vectors of temporal candidates according to some examples of the disclosure.

[0025] Figure 8 is a diagram illustrating candidate positions of temporal candidates according to some examples of the disclosure.

[0026] Figure 9 A diagram illustrating a merge mode (MMVD) search point with motion vector difference is shown.

[0027] Figure 10 A diagram illustrating uni-prediction motion vector selection for geometric partition mode (GPM) is shown.

[0028] Figure 11 A diagram illustrating top and left neighboring blocks used in CIIP weight derivation is shown.

[0029] Figure 12 A diagram illustrating current CTU processing order and its available reference samples in current CTU and left CTU is shown.

[0030] Figure 13 A diagram illustrating padding candidates for replacing zero vectors in IBC list is shown.

[0031] Figure 14 A diagram illustrating reference regions for IBC when CTU (m, n) is coded. According to some examples of the disclosure, the block (m, n) with dotted shading represents the current CTU; the block with " / " shading represents the reference region; and the block without shading represents the invalid reference region.

[0032] Figure 15 A diagram illustrating IBC reference region for camera captured content is shown.

[0033] Figures 16A to 16B A diagram illustrating partitioning method for angular mode is shown.

[0034] Figures 17A to 17D A diagram illustrating GPM with inter and intra prediction is shown. Figures 17A to 17C An available IPM candidate is shown. Figure 17D An example of GPM with intra and inter prediction is shown according to some examples of the disclosure.

[0035] Figure 18 A diagram illustrating edges on a template is shown according to some examples of the disclosure.

[0036] Figure 19 A diagram illustrating intra template matching search region used according to some examples of the disclosure is shown.

[0037] Figure 20 A diagram illustrating template for template matching based OBMC is shown according to some examples of the disclosure.

[0038] Figure 21Methods of partitioning and corresponding weights for intra coded blocks of angular and planar modes are illustrated in accordance with some examples of the present disclosure.

[0039] Figure 22 Templates and their reference samples for intra coded blocks are illustrated in accordance with some examples of the present disclosure.

[0040] Figure 23 Templates and their reference samples for IBC coded blocks are illustrated in accordance with some examples of the present disclosure.

[0041] Figure 24 Exemplary non-adjacent neighboring blocks for IBC AMVP candidates or IBC merge candidates are illustrated in accordance with some examples of the present disclosure.

[0042] Figures 25A to 25B Non-adjacent neighboring blocks of different sizes are illustrated: Figure 25A Neighboring blocks with the same size as the current block are illustrated in accordance with some examples of the present disclosure, and Figure 25B Neighboring blocks with different sizes (e.g., 4 x 4 or 8 x 8) than the current block are illustrated in accordance with some examples of the present disclosure.

[0043] Figure 26 One example of spatial neighboring blocks for deriving spatial non-adjacent candidates for IBC mode is illustrated in accordance with some examples of the present disclosure, where the numbers in the non-adjacent neighboring blocks represent the scan order.

[0044] Figure 27 Another example of spatial neighboring blocks for deriving spatial non-adjacent candidates for IBC mode is illustrated in accordance with some examples of the present disclosure, where the numbers in the non-adjacent neighboring blocks represent the scan order, and the numbers after the arrows represent the degree value of the angle.

[0045] Figure 28 An example where the spatial non-adjacent region is limited within half CTU size above and left of the current CTU is illustrated in accordance with some examples of the present disclosure.

[0046] Figures 29A to 29B Motion storage for spatial non-adjacent neighbors (IBC neighboring CUs or non-IBC neighboring CUs) is illustrated: Figure 29A Permissible spatial non-adjacent regions beyond the current CTU are illustrated in accordance with some examples of the present disclosure, and Figure 29B Motion storage in the line buffer (2902 is an IBC CU; 2903 is a non-IBC CU) is illustrated in accordance with some examples of the present disclosure.

[0047] Figure 30Figures illustrate non-adjacent neighboring positions projected / clipped when the scanned non-adjacent neighboring positions exceed the allowable spatial region, according to some examples of the present disclosure.

[0048] Figure 31 Figures illustrate another example of non-adjacent neighboring positions projected / clipped when the scanned non-adjacent neighboring positions exceed the allowable spatial region (e.g., beyond the current CTU and available row buffer), according to some examples of the present disclosure.

[0049] Figure 32 Figures illustrate that the granularity of IBC motion storage is different from the minimum IBC block size, according to some examples of the present disclosure.

[0050] Figure 33 Figures illustrate one example of subblock-based IBC mode, according to some examples of the present disclosure, where the BVs of subblocks in the current block are obtained by reusing the BVs of subblocks of collocated blocks in the collocated picture.

[0051] Figure 34 Figures illustrate one example of subblock-based IBC mode, where the BVs of left or top subblocks in the current block are obtained by refining the BVs of the current block with a template matching method.

[0052] Figure 35 is a diagram illustrating a computing environment coupled with a user interface, according to some examples of the present disclosure.

[0053] Figure 36 is a diagram illustrating a ramp function of the weight of GPM blending based on the displacement (d) from the prediction sample position to the GPM partition boundary and the blending region size (τ), according to some examples of the present disclosure.

[0054] Figures 37A to 37C is a diagram illustrating a spatial GPM candidate, according to some examples of the present disclosure.

[0055] Figure 38 is a diagram illustrating a GPM template, according to some examples of the present disclosure.

[0056] Figure 39 is a diagram illustrating GPM blending, according to some examples of the present disclosure.

[0057] Figure 40 Figures illustrate additional directions (positions 4010 for anchor points) along k x π / 8 diagonal line angles, according to some examples of the present disclosure.

[0058] Figure 41 Figures illustrate another example of spatial neighboring blocks for deriving spatial non-adjacent candidates for IBC mode, according to some examples of the present disclosure, where the numbers in the non-adjacent neighboring blocks represent the scan order.

[0059] Figure 42A and Figure 42B Figures respectively illustrate horizontal flipping and vertical flipping of BV adjustment according to some embodiments of the disclosure.

[0060] Figure 43 Figure illustrates an example of GPM application block according to some embodiments of the disclosure.

[0061] Figure 44 Figure illustrates another example of spatial neighboring blocks for deriving spatial non-adjacent candidates of IBC mode, where the numbers in the non-adjacent neighboring blocks represent the scanning order. The example is according to some embodiments of the disclosure.

[0062] Figure 45 Figure illustrates another example of spatial neighboring blocks for deriving spatial non-adjacent candidates of IBC mode, where the numbers in the non-adjacent neighboring blocks represent the scanning order. The example is according to some embodiments of the disclosure.

[0063] Figure 46 Figure illustrates another example of spatial neighboring blocks for deriving spatial non-adjacent candidates of IBC mode, where the numbers in the non-adjacent neighboring blocks represent the scanning order. The example is according to some embodiments of the disclosure.

[0064] Figure 47 Figure illustrates another example of spatial neighboring blocks for deriving spatial non-adjacent candidates of IBC mode, where the numbers in the non-adjacent neighboring blocks represent the scanning order. The example is according to some embodiments of the disclosure.

[0065] Figure 48 Figure illustrates another example of spatial neighboring blocks for deriving spatial non-adjacent candidates of IBC mode, where the numbers in the non-adjacent neighboring blocks represent the scanning order. The example is according to some embodiments of the disclosure.

[0066] Figure 49 Figure illustrates another example of spatial neighboring blocks for deriving spatial non-adjacent candidates of IBC mode, where the numbers in the non-adjacent neighboring blocks represent the scanning order. The example is according to some embodiments of the disclosure.

[0067] Figure 50 Figure is a flowchart illustrating a video decoding method according to some examples of the disclosure.

[0068] Figure 51 Figure is a flowchart illustrating a video encoding method according to some examples of the disclosure corresponding to the video decoding method as shown in Figure 44

[0069] Figure 52 ​is a flowchart illustrating a video decoding method according to some examples of the present disclosure.

[0070] Figure 53 is a flowchart illustrating a video encoding method according to some examples of the present disclosure corresponding to the video decoding method as shown in Figure 46

[0071] Figure 54 is a flowchart illustrating a video decoding method according to some examples of the present disclosure.

[0072] Figure 55 is a flowchart illustrating a video encoding method according to some examples of the present disclosure corresponding to the video decoding method as shown in Figure 48 DETAILED DESCRIPTION

[0073] Reference will now be made in detail to specific implementations, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth in order to assist in understanding the subject matter presented herein. But various alternatives can be used without departing from the scope of the claims, and the subject matter can be practiced without these specific details. For example, the subject matter presented herein can be implemented on many classes of electronic devices with digital video capabilities.

[0074] The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. The use of the singular herein includes the plural unless the context clearly dictates otherwise. The use of the term “and / or” in the context of a list of items, such as “A and / or B” or “A / B” is intended to refer to one or all of the items in the list, for example, A or B or both A and B. The use of the term “one or more” in the context of a list of items, such as “A and / or B” or “A / B” is intended to refer to one or all of the items in the list, for example, A or B or both A and B.

[0075] Reference throughout this specification to “one embodiment,” “an embodiment,” “example,” “some embodiments,” “some examples,” or similar language means that a particular feature, structure, or characteristic described follows in at least one embodiment or example. The particular feature, structure, element, or characteristic follows in at least one embodiment, unless specifically stated otherwise.

[0076] Throughout this disclosure the terms “first,” “second,” “third,” etc. are used as a nomenclature, merely to refer to a relevant element, such as a device, component, constituent, step, etc., and do not imply any spatial or chronological order, unless explicitly stated otherwise. For example, “a first device” and “a second device” can refer to two separately formed devices, or two parts, components, or working states of the same device, and can be arbitrarily named.

[0077] ​​The terms "module," "sub-module," "circuit," "sub-circuit," "circuitry," "sub-circuitry," "unit," or "sub-unit" can include a memory (shared, dedicated, or group) that stores code or instructions that can be executed by one or more processors. A module can include one or more circuits with or without stored code or instructions. A module or circuit can include one or more components connected directly or indirectly to each other or to each other adjacent.

[0078] As used herein, the terms "if' or "when" can be understood to mean "upon" or "in response to," depending on the context. These terms, if present in a claim, can not indicate that the related limitation or feature is conditional or optional. For example, a method can include the steps of: i) performing function or action X' when or if condition X is present, and ii) performing function or action Y' when or if condition Y is present. The method can simultaneously possess the ability to perform function or action X' and the ability to perform function or action Y'. Thus, functions X' and Y' can be performed in different times in multiple executions of the method.

[0079] A unit or module can be implemented in pure software, in pure hardware, or in a combination of hardware and software. For example, in a pure software implementation, a unit or module can include functionally related code blocks or software components that are directly or indirectly linked together to perform a particular function.

[0080] Figure 1 is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel, in accordance with some embodiments of the present disclosure. As shown in Figure 1 As shown in FIG. 1, system 10 includes a source device 12 that generates and encodes video data to be decoded later by a destination device 14. Source device 12 and destination device 14 can comprise any of a wide variety of electronic devices including, for example, a cloud server, a server computer, a desktop or laptop computer, a tablet computer, a smart phone, a set-top box, a digital television, a camera, a display device, a digital media player, a video gaming console, a video streaming device, etc. In some embodiments, source device 12 and destination device 14 are equipped with wireless communication capabilities.

[0081] In some embodiments, destination device 14 can receive, via link 16, encoded video data to be decoded. Link 16 can comprise any type of communication medium or device capable of moving the encoded video data from source device 12 to destination device 14. In one example, link 16 can comprise a communication medium to enable source device 12 to transmit encoded video data directly to destination device 14 in real-time. The encoded video data can be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to destination device 14. The communication medium can comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium can form part of a packet-based network, such as a local area network, a wide-area network, or a global network such as the Internet. The communication medium can include routers, switches, base stations, or any other equipment that can be useful to facilitate communication from source device 12 to destination device 14.

[0082] In other embodiments, encoded video data can be transmitted from output interface 22 to storage device 32. Subsequently, encoded video data in storage device 32 can be accessed by destination device 14 via input interface 28. Storage device 32 can include any of a variety of distributed or locally accessed data storage media such as a hard drive, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded video data. In a further example, storage device 32 can correspond to a file server or another intermediate storage device that can hold the encoded video data generated by source device 12. Destination device 14 can access stored video data from storage device 32 via streaming or download. The file server can be any type of computer

[0083] As Figure 1As shown in FIG. 1, source device 12 includes video source 18, video encoder 20, and output interface 22. Video source 18 can include a source such as a video capture device, e.g., a video camera, a video archive containing previously captured video, a video feed interface to receive video from a video content provider, and / or a computer graphics system for generating computer graphics data as the source video, or a combination of such sources. As one example, if video source 18 is a video camera of a security surveillance system, source device 12 and destination device 14 can form a camera phone or video phone. However, the implementations described in this application can be applied to video coding in general, and can have application to wireless and / or wired applications.

[0084] The captured, pre-captured, or computer-generated video can be encoded by video encoder 20. The encoded video data can be transmitted directly to destination device 14 via output interface 22 of source device 12. The encoded video data can also (or alternatively) be stored onto storage device 32 for later access by destination device 14 or other devices, for decoding and / or playback. Output interface 22 can further include a modem and / or a transmitter.

[0085] Destination device 14 includes input interface 28, video decoder 30, and display device 34. Input interface 28 can include a receiver and / or modem and receives encoded video data over link 16. The encoded video data communicated over link 16, or provided on storage device 32, can include a variety of syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements can be included within the encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0086] In some implementations, destination device 14 can include display device 34, which can be an integrated display device and an external display device configured to communicate with destination device 14. Display device 34 displays the decoded video data to a user, and can include any of a variety of display devices such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0087] Video encoder 20 and video decoder 30 can operate according to a proprietary standard or industry standard, such as VVC, HEVC, MPEG-4, Part 10, AVC, or extensions of such standards. It should be understood that the application is not limited to a specific video coding / decoding standard and can be applicable to other video coding / decoding standards. It is generally contemplated that video encoder 20 of source device 12 can be configured to encode video data according to any of these current or future standards. Similarly, it is also generally contemplated that video decoder 30 of destination device 14 can be configured to decode video data according to any of these current or future standards.

[0088] Video encoder 20 and video decoder 30 can be implemented as any of a variety of suitable encoder and / or decoder circuitry, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic circuitry, software, hardware, firmware or any combinations thereof. When implemented partially in software, an electronic device can store instructions for the software in a suitable, non- transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the video coding / decoding operations disclosed in the present disclosure. Each of video encoder 20 and video decoder 30 can be included in one or more encoders or decoders, either of which can be integrated as part of a combined encoder / decoder (CODEC) in a respective device.

[0089] In some implementations, at least a portion of the components of source device 12 (e.g., video source 18, video encoder 20, or the components of video encoder 20 described below with reference to FIG. 2, and output interface 22), and / or the components of destination device 14 (e.g., input interface 28, video decoder 30, or the components of video decoder 30 described below with reference to FIG. 3) can be implemented as one or more integrated circuits. Figure 1 In some implementations, at least a portion of the components of source device 12 (e.g., video source 18, video encoder 20, or the components of video encoder 20 described below with reference to FIG. 2, and output interface 22), and / or the components of destination device 14 (e.g., input interface 28, video decoder 30, or the components of video decoder 30 described below with reference to FIG. 3) can be implemented as one or more integrated circuits. Figure 3At least a portion of the components described as included in video decoder 30, and display device 34, can operate in a cloud computing services network, such as a software as a service (SaaS), platform as a service (PaaS), or infrastructure as a service (IaaS), which can provide software, platforms, and / or infrastructure. In some implementations, one or more components of source device 12 and / or destination device 14 that are not included in the cloud computing services network can be disposed in one or more client devices, and the one or more client devices can communicate with server computers in the cloud computing services network through a wireless communication network (e.g., a cellular communication network, a short-range wireless communication network, or a global navigation satellite system (GNSS) communication network) or a wired communication network (e.g., a local area network (LAN) communication network or a power line communication (PLC) network). In embodiments, at least a portion of the operations described herein can be implemented as a cloud-based service provided by one or more server computers implemented by at least a portion of the components of source device 12 and / or at least a portion of the components of destination device 14 in the cloud computing services network; and one or more other operations described herein can be implemented by one or more client devices. In some implementations, the cloud computing services network can be a private cloud, a public cloud, or a hybrid cloud. The terms such as “cloud,” “cloud computing,” “cloud-based,” and the like can be used interchangeably herein as appropriate without departing from the scope of the disclosure. It should be understood that the disclosure is not limited to implementation in the above-described cloud computing services network. Rather, the disclosure can also be implemented in any other type of computing environment currently known or developed in the future.

[0090] Figure 2 is a block diagram illustrating another exemplary video encoder 20 in accordance with some embodiments described in the present application. Video encoder 20 can perform intra-prediction encoding and inter-prediction encoding on video blocks within a video frame. Intra-prediction encoding relies on spatial prediction to reduce or remove spatial redundancy in video data within a given video frame or picture. Inter-prediction encoding relies on temporal prediction to reduce or remove temporal redundancy in video data within neighboring video frames or pictures of a video sequence. It should be noted that in the field of video coding, the term “frame” can be used as a synonym for the term “image” or “picture.”

[0091] As Figure 2As shown in FIG. 1, video encoder 20 includes video data memory 40, prediction processing unit 41, decoded picture buffer (DPB) 64, summer 50, transform processing unit 52, quantization unit 54, and entropy encoding unit 56. Prediction processing unit 41 further includes motion estimation unit 42, motion compensation unit 44, partition unit 45, intra-prediction processing unit 46, and intra-block copy (BC) unit 48. In some implementations, video encoder 20 also includes inverse quantization unit 58, inverse transform processing unit 60, and summer 62 for video block reconstruction. A loop filter 63, such as a deblocking filter, can be located between summer 62 and DPB 64 to filter block boundaries to remove blockiness artifacts from reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can be used to filter the output of summer 62. It should be noted that for CCSAO techniques, the present application is not limited to the embodiments described herein, but can also be applied to cases where one offset is selected for any one of a luma component, a Cb chroma component, and a Cr chroma component to select one offset for any other one of the luma component, the Cb chroma component, and the Cr chroma component to modify the any other one based on the selected offset. Furthermore, it should also be noted that the first component referred to herein can be any one of the luma component, the Cb chroma component, and the Cr chroma component, the second component referred to herein can be any other one of the luma component, the Cb chroma component, and the Cr chroma component, and the third component referred to herein can be the remaining one of the luma component, the Cb chroma component, and the Cr chroma component. In some examples, the loop filter can be omitted, and the decoded video block can be provided directly by summer 62 to DPB 64. Video encoder 20 can take the form of a fixed or programmable hardware encoder, or can be dispersed in one or more of the illustrated fixed or programmable hardware encoders.

[0092] Video data memory 40 can store video data to be encoded by the components of video encoder 20. The video data in video data memory 40 can be obtained, for example, from video source 18, as shown in FIG. 1. DPB 64 is a buffer that stores reference video data (e.g., reference frames or pictures) for use in encoding video data by video encoder 20 (e.g., in intra- or inter-coding modes). Video data memory 40 and DPB 64 can be formed by a variety of memory devices, including a Figure 1 suggested in FIG. 1. DPB 64 is a buffer that stores reference video data (e.g., reference frames or pictures) for use in encoding video data by video encoder 20 (e.g., in intra- or inter-coding modes). Video data memory 40 and DPB 64 can be formed by a variety of memory devices, including a

[0093] As Figure 2As shown in FIG. 1, after receiving video data, partitioning unit 45 within prediction processing unit 41 partitions the video data into video blocks. This partitioning can also include partitioning of a video frame into slices, tiles (e.g., a collection of video blocks), or other larger coding units (CUs) according to a predefined splitting structure (e.g., a quadtree (QT) structure) associated with the video data. A video frame is or can be considered as a two-dimensional array or matrix of sample values. Samples in the array can also be referred to as pixels or pels. The number of samples in the horizontal and vertical directions (or axes) of the array or picture defines the size and / or resolution of the video frame. For example, a video frame can be divided into a plurality of video blocks by using QT partitioning. A video block is or can be considered as a two-dimensional array or matrix of sample values as well, but with smaller dimensions than the video frame. The number of samples in the horizontal and vertical directions (or axes) of the video block defines the size of the video block. A video block can be further partitioned into one or more block partitions or sub-blocks (which can form blocks again) by, for example, iteratively using QT partitioning, binary tree (BT) partitioning, or ternary tree (TT) partitioning, or any combination thereof. It should be noted that the term “block” or “video block” as used herein can be a portion of a frame or picture, in particular a rectangular (square or non-square) portion. With reference to, for example, HEVC and VVC, a block or video block can be or correspond to a coding tree unit (CTU), a CU, a prediction unit (PU), or a transform unit (TU) and / or can be or correspond to a respective block (e.g., a coding tree block (CTB), a coding block (CB), a prediction block (PB), or a transform block (TB)) and / or a sub-block.

[0094] Prediction processing unit 41 can select one of a plurality of possible predictive encoding modes, such as one of a plurality of intra-predictive encoding modes or one of a plurality of inter-predictive encoding modes, for the current video block based on error results (e.g., rate and distortion levels). Prediction processing unit 41 can provide the resulting intra- or inter-predicted block to summer 50 to generate a residual block, and to summer 62 to reconstruct the encoded block for use as part of a reference frame at a later time. Prediction processing unit 41 also provides syntax elements, such as motion vectors, intra-mode indicators, partitioning information, and other such syntax information, to entropy encoding unit 56.

[0095] To select a suitable intra-prediction coding mode for the current video block, intra-prediction processing unit 46 within prediction processing unit 41 can perform intra-prediction coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block being coded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 perform inter-prediction coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. Video encoder 20 can perform multiple coding passes, e.g., to select a suitable coding mode for each block of video data.

[0096] In some implementations, motion estimation unit 42 determines an inter-prediction mode for a current video frame by generating motion vectors according to a predetermined pattern within a sequence of video frames, the motion vectors indicating displacement of video blocks within the current video frame relative to prediction blocks within a reference video frame. Motion estimation performed by motion estimation unit 42 is a process of generating motion vectors that estimate motion for video blocks. For example, a motion vector can indicate displacement of a video block within a current video frame or picture relative to a prediction block within a reference frame that is related to a current block being coded within the current frame. The predetermined pattern can designate video frames in the sequence as P-frames or B-frames. Intra-BC unit 48 can determine vectors for intra-BC coding (e.g., block vectors) in a similar manner as motion vectors determined by motion estimation unit 42 for inter-prediction, or can utilize block vectors determined by motion estimation unit 42.

[0097] In terms of pixel difference, a prediction block for a video block can be or can correspond to a block or reference block of a reference frame that is deemed to closely match the video block being coded, the pixel difference can be determined by sum of absolute difference (SAD), sum of squared difference (SSD), or other difference metric. In some implementations, video encoder 20 can calculate values for sub-integer pixel positions of reference frames stored in DPB 64. For example, video encoder 20 can interpolate values for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of a reference frame. Thus, motion estimation unit 42 can perform a motion search relative to full-pixel positions and fractional-pixel positions and output motion vectors with fractional-pixel precision.

[0098] Motion estimation unit 42 calculates a motion vector for a video block in an inter-prediction coded frame by comparing a location of the video block to a location of a prediction block of a reference frame selected from a first reference frame list (List 0) or a second reference frame list (List 1), each of the first and second reference frame lists identifying one or more reference frames stored in DPB 64. Motion estimation unit 42 sends the calculated motion vector to motion compensation unit 44, which then sends to entropy encoding unit 56.

[0099] Motion compensation performed by motion compensation unit 44 can involve fetching or generating a prediction block based on a motion vector determined by motion estimation unit 42. After receiving a motion vector for a current video block, motion compensation unit 44 can locate the prediction block pointed to by the motion vector in one of the reference frame lists, retrieve the prediction block from DPB 64, and forward the prediction block to summer 50. Summer 50 then forms a residual video block of pixel difference values by subtracting the pixel values of the prediction block provided by motion compensation unit 44 from the pixel values of the current video block being encoded. The pixel difference values forming the residual video block can include luma component differences or chroma component differences or both. Motion compensation unit 44 can also generate syntax elements associated with a video block of a video frame for use by video decoder 30 when decoding the video block of the video frame. The syntax elements can include, for example, syntax elements defining motion vectors used to identify a prediction block, any flags indicating a prediction mode, or any other syntax information described herein. It is noted that motion estimation unit 42 and motion compensation unit 44 can be highly integrated, but are illustrated separately for conceptual purposes.

[0100] In some implementations, intra BC unit 48 can generate vectors and fetch prediction blocks in a manner similar to that described above in connection with motion estimation unit 42 and motion compensation unit 44, but the prediction blocks are in the same frame as the current block being encoded, and the vectors are referred to as block vectors rather than motion vectors. Specifically, intra BC unit 48 can determine an intra prediction mode to use for encoding the current block. In some examples, intra BC unit 48 can encode the current block using various intra prediction modes, e.g., during a separate encoding pass, and test their performance through rate-distortion analysis. Next, intra BC unit 48 can select an appropriate intra prediction mode to use among the various tested intra prediction modes and generate an intra mode indicator accordingly. For example, intra BC unit 48 can compute rate-distortion values for the various tested intra prediction modes using rate-distortion analysis, and select the intra prediction mode with the best rate-distortion characteristics among the tested modes as the appropriate intra prediction mode to use. Rate-distortion analysis generally determines an amount of distortion (or error) between an encoded block and the original, unencoded block that was encoded to produce the encoded block, and a bit rate (i.e., number of bits) used to produce the encoded block. Intra BC unit 48 can compute a ratio from the distortion and rate for various encoded blocks to determine which intra prediction mode exhibits the best rate-distortion values for the block.

[0101] In other examples, intra BC unit 48 can perform such functions for intra BC prediction according to the implementations described herein using motion estimation unit 42 and motion compensation unit 44 in whole or in part. In either case, for intra block copy, in terms of pixel difference, the prediction block can be a block that is deemed to closely match the block to be encoded, the pixel difference can be determined by SAD, SSD, or other difference metrics, and identifying the prediction block can include calculating values for sub-integer pixel positions.

[0102] Whether the prediction block is from the same frame according to intra prediction or a different frame according to inter prediction, video encoder 20 can form pixel difference values by subtracting the pixel values of the prediction block from the pixel values of the current video block being encoded, thereby forming a residual video block. The pixel difference values forming the residual video block can include both luma component differences and chroma component differences.

[0103] As an alternative to inter prediction performed by motion estimation unit 42 and motion compensation unit 44 or intra block copy prediction performed by intra BC unit 48 as described above, intra prediction processing unit 46 can intra predict the current video block. In particular, intra prediction processing unit 46 can determine an intra prediction mode to use for encoding the current block. To this end, intra prediction processing unit 46 can encode the current block using various intra prediction modes, e.g., during a separate encoding pass, and intra prediction processing unit 46 (or, in some examples, a mode selection unit) can select an appropriate intra prediction mode to use from among the tested intra prediction modes. Intra prediction processing unit 46 can provide information indicative of the selected intra prediction mode for the block to entropy encoding unit 56. Entropy encoding unit 56 can encode the information indicative of the selected intra prediction mode into the bitstream.

[0104] After prediction processing unit 41 determines a prediction block for the current video block via inter prediction or intra prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block can be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform, e.g., a discrete cosine transform (DCT) or a conceptually similar transform.

[0105] Transform processing unit 52 can send the resulting transform coefficients to quantization unit 54. Quantization unit 54 quantizes the transform coefficients to further reduce bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting a quantization parameter. In some examples, quantization unit 54 can then perform a scan on the matrix including the quantized transform coefficients. Alternatively, entropy encoding unit 56 can perform the scan.

[0106] Following quantization, entropy encoding unit 56 entropy encodes the quantized transform coefficients into a video bitstream using, for example, context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), Probability Interval Partitioning Entropy (PIPE) coding, or another entropy encoding methodology or technique. The encoded bitstream can then be transmitted to video decoder 30 as shown in FIG. 3, or archived, such as to storage device 32 as shown in FIG. 3 for later transmission to or retrieval by video decoder 30. Entropy encoding unit 56 can also entropy encode motion vectors and other syntax elements for the current video frame that is being encoded. Figure 1 Figure 1

[0107] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transforms, respectively, to reconstruct the residual video block in the pixel domain for use in generating reference blocks for predicting other video blocks. As noted above, motion compensation unit 44 can generate a motion compensated prediction block from one or more reference blocks of frames stored in DPB 64. Motion compensation unit 44 can also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values for use in motion estimation.

[0108] Adder 62 adds the reconstructed residual block to the motion compensated prediction block produced by motion compensation unit 44 to produce a reference block to be stored in DPB 64. The reference block can then be used by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 as a prediction block to inter predict another video block in a subsequent video frame.

[0109] Figure 3 FIG. 3 is a block diagram illustrating an example video decoder 30, in accordance with some embodiments of the present application. Video decoder 30 includes video data memory 79, entropy decoding unit 80, prediction processing unit 81, inverse quantization unit 86, inverse transform processing unit 88, adder 90, and DPB 92. Prediction processing unit 81 further includes motion compensation unit 82, intra prediction unit 84, and intra BC unit 85. Video decoder 30 can perform a decoding process generally reciprocal to the encoding process described above in connection with video encoder 20. Figure 2 The decoding process described in connection with video encoder 20 is substantially reciprocal. For example, motion compensation unit 82 can generate prediction data based on motion vectors received from entropy decoding unit 80, while intra prediction unit 84 can generate prediction data based on intra prediction mode indicators received from entropy decoding unit 80.

[0110] ​​In some examples, the components of video decoder 30 can be tasked to perform the implementations of the present application. Moreover, in some examples, the implementations of the present disclosure can be distributed among one or more of the components of video decoder 30. For example, intra BC unit 85 can perform the implementations of the present application, alone or in combination with other units of video decoder 30, such as motion compensation unit 82, intra prediction unit 84, and entropy decoding unit 80. In some examples, video decoder 30 can not include intra BC unit 85, and the functionality of intra BC unit 85 can be performed by other components of prediction processing unit 81, such as motion compensation unit 82.

[0111] Video data memory 79 can store video data, such as an encoded video bitstream, to be decoded by the other components of video decoder 30. The video data stored in video data memory 79 can be obtained, for example, from storage device 32, from a local video source, such as a camera, via wired or wireless network communication of video data, or by accessing physical data storage media (e.g., a flash drive or hard disk). Video data memory 79 can include a coded picture buffer (CPB) that stores encoded video data from an encoded video bitstream. DPB 92 of video decoder 30 stores reference video data for use in decoding video data by video decoder 30 (e.g., in intra- or inter-coding modes). Video data memory 79 and DPB 92 can be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magneto resistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For Figure 3 illustrative purposes, video data memory 79 and DPB 92 are depicted as two distinct components of video decoder 30. However, it will be readily apparent to one of ordinary skill in the art that video data memory 79 and DPB 92 can be provided by one memory device or separate memory devices. In some examples, video data memory 79 can be on-chip with other components of video decoder 30, or off-chip relative to those components.

[0112] During the decoding process, video decoder 30 receives an encoded video bitstream that represents encoded video frames and associated syntax elements. Video decoder 30 can receive the syntax elements at the video frame level and / or video block level. Entropy decoding unit 80 of video decoder 30 entropy decodes the bitstream to generate quantized coefficients, motion vectors, or intra-prediction mode indicators, among other syntax elements. Entropy decoding unit 80 then forwards the motion vectors, or intra-prediction mode indicators, among other syntax elements, to prediction processing unit 81.

[0113] When a video frame is encoded as an intra-predicted (I) frame or an intra-coded prediction block in another type of frame, intra-prediction unit 84 of prediction processing unit 81 can generate prediction data for a video block of the current video frame based on the intra-prediction mode signaled and reference data from previously decoded blocks of the current frame.

[0114] When a video frame is encoded as an inter-predicted (i.e., B or P) frame, motion compensation unit 82 of prediction processing unit 81 produces one or more prediction blocks for a video block of the current video frame based on the motion vectors and other syntax elements received from entropy decoding unit 80. Each of the prediction blocks can be produced from a reference frame within one of the reference frame lists. Video decoder 30 can construct the reference frame lists, i.e., List 0 and List 1, using default construction techniques based on reference frames stored in DPB 92.

[0115] In some examples, when a video block is encoded according to the intra BC modes described herein, intra BC unit 85 of prediction processing unit 81 produces a prediction block for the current video block based on the block vectors and other syntax elements received from entropy decoding unit 80. The prediction block can be within a reconstructed region of the same picture as the current video block, as defined by video encoder 20.

[0116] Motion compensation unit 82 and / or intra BC unit 85 determine the prediction information for a video block of the current video frame by parsing the motion vectors and other syntax elements, and then use the prediction information to produce a prediction block for the current video block being decoded. For example, motion compensation unit 82 uses some of the received syntax elements to determine the prediction mode used to encode the video block of the video frame (e.g., intra-prediction or inter-prediction), the inter-prediction frame type (e.g., B or P), the construction information for one or more of the reference frame lists for the frame, the motion vectors for each inter-predicted video block of the frame, the inter-prediction status for each inter-predicted video block of the frame, and other information used to decode the video block in the current video frame.

[0117] Similarly, intra BC unit 85 can use some of the received syntax elements, such as flags, to determine that the current video block is predicted using the intra BC mode, the construction information for which video blocks of the frame are within the reconstructed region and should be stored in DPB 92, the block vectors for each intra BC predicted video block of the frame, the intra BC prediction status for each intra BC predicted video block of the frame, and other information used to decode the video block in the current video frame.

[0118] Motion compensation unit 82 can also perform interpolation using interpolation filters as used by video encoder 20 during encoding of the video blocks to calculate interpolated values for sub-integer pixels of reference blocks. In this case, motion compensation unit 82 can determine the interpolation filters used by video encoder 20 from the syntax elements received and use these interpolation filters when generating the prediction blocks.

[0119] Inverse quantization unit 86 inverse quantizes quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80 using the same quantization parameter calculated by video encoder 20 for each video block in the video frame to determine a degree of quantization. Inverse transform processing unit 88 applies an inverse transform, e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to the transform coefficients in order to reconstruct the residual blocks in the pixel domain.

[0120] After motion compensation unit 82 or intra BC unit 85 generates the prediction block for the current video block based on the vectors and other syntax elements, adder 90 reconstructs the decoded video block for the current video block by adding the residual block from inverse transform processing unit 88 to the corresponding prediction block generated by motion compensation unit 82 and intra BC unit 85. In-loop filter 91, e.g., a de-blocking filter, a SAO filter, a CCSAO filter, and / or an ALF, can be located between adder 90 and DPB 92 to further process the decoded video block. In some examples, in-loop filter 91 can be omitted, and the decoded video block can be directly provided by adder 90 to DPB 92. The decoded video blocks in a given frame are then stored in DPB 92, which stores reference frames for subsequent motion compensation of video blocks. DPB 92 or a separate memory device from DPB 92 can also store decoded video for later presentation on a display device (e.g., display device 34 of FIG. 1). Figure 1

[0121] In a typical video coding process, a video sequence generally includes an ordered set of frames or pictures. Each frame can include three sample arrays denoted as SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of Cb chroma samples. SCr is a two-dimensional array of Cr chroma samples. In other instances, a frame can be monochrome, and thus include only one two-dimensional array of luma samples.

[0122] As Figure 4A ​As shown in FIG. 1, video encoder 20 (or more specifically, a partition unit in a prediction processing unit of video encoder 20) generates an encoded representation of a frame by first partitioning the frame into a set of CTUs. A video frame can include an integer number of CTUs ordered consecutively in the raster scan order from left to right and from top to bottom. Each CTU is the largest logical coding unit and the width and height of the CTU are signaled by video encoder 20 in a sequence parameter set such that all CTUs in a video sequence have the same size of one of 128x128, 64x64, 32x32, and 16x16. But it should be noted that the present application is not necessarily limited to a particular size. As Figure 4B As shown in FIG. 3, each CTU can include one CTB of luma samples, two corresponding coding tree blocks of chroma samples, and syntax elements used to code the samples of the coding tree blocks. The syntax elements describe the properties of different types of units of the coded pixel blocks and how the video sequence, including inter-prediction or intra-prediction, intra-prediction modes, motion vectors, and other parameters can be reconstructed at video decoder 30. In monochrome pictures or pictures with three separate color planes, a CTU can include a single coding tree block and syntax elements used to code the samples of the coding tree block. A coding tree block can be an NxN block of samples.

[0123] To achieve better performance, video encoder 20 can recursively perform tree partitioning, such as binary tree partitioning, ternary tree partitioning, quad-tree partitioning, or a combination thereof, on the coding tree blocks of a CTU and divide the CTU into smaller CUs. FIGS. 4B-4E are block diagrams illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes according to some embodiments of the present disclosure. As Figure 4C As depicted in FIG. 4B, a 64x64 CTU 400 is first divided into four smaller CUs, each having a block size of 32x32. Of the four smaller CUs, CUs 410 and 420 are each divided into four CUs having a block size of 16x16. Two 16x16 CUs 430 and 440 are each further divided into four CUs having a block size of 8x8. Figure 4D The final result of the partitioning process of CTU 400 is depicted in a quad-tree data structure illustrating the hierarchy of the CUs in FIG. 4E. Each leaf node of the quad-tree corresponds to one CU of a respective size ranging from 32x32 to 8x8. Similar to the binary tree data structure depicted in FIG. 4D, the root node of the quad-tree corresponds to the 64x64 CTU 400. Figure 4C As depicted in FIG. 4B, a 64x64 CTU 400 is first divided into four smaller CUs, each having a block size of 32x32. Of the four smaller CUs, CUs 410 and 420 are each divided into four CUs having a block size of 16x16. Two 16x16 CUs 430 and 440 are each further divided into four CUs having a block size of 8x8. Figure 4B As depicted in FIG. 3, each CTU can include one CTB of luma samples, two corresponding coding tree blocks of chroma samples, and syntax elements used to code the samples of the coding tree blocks. The syntax elements describe the properties of different types of units of the coded pixel blocks and how the video sequence, including inter-prediction or intra-prediction, intra-prediction modes, motion vectors, and other parameters can be reconstructed at video decoder 30. In monochrome pictures or pictures with three separate color planes, a CTU can include a single coding tree block and syntax elements used to code the samples of the coding tree block. A coding tree block can be an NxN block of samples. Figure 4C and Figure 4DThe quad-tree partitioning depicted in the middle is for illustrative purposes only, and one CTU can be split into multiple CUs based on quad-tree partitioning / triple-tree partitioning / binary-tree partitioning to adapt to varying local characteristics. In the multi-type tree structure, one CTU is partitioned according to a quad-tree structure, and each quad-tree leaf CU can be further partitioned according to binary and triple tree structures. As shown in Figure 4E The coding block with width W and height H has five possible partition types, i.e., quad partition, horizontal binary partition, vertical binary partition, horizontal ternary partition, and vertical ternary partition.

[0124] In some implementations, video encoder 20 can further partition the coding block of a CU into one or more (MxN) PBs. A PB is a rectangular (square or non-square) block of samples to which the same prediction (inter or intra) is applied. A PU of a CU can include a PB of luma samples, two corresponding PBs of chroma samples, and syntax elements used to predict the PBs. In a monochrome picture or a picture having three separate color planes, a PU can include a single PB and syntax structures used to predict the PB. Video encoder 20 can generate a predicted luma block, a predicted Cb block, and a predicted Cr block for the luma PB, the Cb PB, and the Cr PB of each PU of a CU.

[0125] Video encoder 20 can use intra prediction or inter prediction to generate the predicted blocks of a PU. If video encoder 20 uses intra prediction to generate the predicted blocks of a PU, video encoder 20 can generate the predicted blocks of the PU based on decoded samples of the frame associated with the PU. If video encoder 20 uses inter prediction to generate the predicted blocks of a PU, video encoder 20 can generate the predicted blocks of the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0126] After video encoder 20 generates the predicted luma blocks, the predicted Cb blocks, and the predicted Cr blocks for one or more PUs of a CU, video encoder 20 can generate luma residual blocks for the CU by subtracting the predicted luma blocks of the CU from the original luma coding block of the CU, such that each sample in a luma residual block of the CU indicates a difference between a luma sample in one of the predicted luma blocks of the CU and a corresponding sample in the original luma coding block of the CU. Similarly, video encoder 20 can generate Cb residual blocks and Cr residual blocks for the CU, respectively, such that each sample in a Cb residual block of the CU indicates a difference between a Cb sample in one of the predicted Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and each sample in a Cr residual block of the CU can indicate a difference between a Cr sample in one of the predicted Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0127] Furthermore, as Figure 4CAs shown, the video encoder 20 can use quadtree partitioning to decompose the luminance residual block, Cb residual block, and Cr residual block of the CU into one or more luminance transform blocks, Cb transform blocks, and Cr transform blocks, respectively. A transform block is a rectangular (square or non-square) sample block to which the same transform is applied. A TU of the CU can include a transform block of the luminance samples, two corresponding transform blocks of the chrominance samples, and syntax elements for transforming the transform block samples. Therefore, each TU of the CU can be associated with a luminance transform block, a Cb transform block, and a Cr transform block. In some examples, the luminance transform block associated with a TU can be a sub-block of the CU's luminance residual block. A Cb transform block can be a sub-block of the CU's Cb residual block. A Cr transform block can be a sub-block of the CU's Cr residual block. In a monochrome image or an image with three separate color planes, a TU can include a single transform block and syntax structures for transforming the samples of that transform block.

[0128] The video encoder 20 can apply one or more transforms to the luminance transform block of the TU to generate a luminance coefficient block for the TU. The coefficient block can be a two-dimensional array of transform coefficients. The transform coefficients can be scalars. The video encoder 20 can apply one or more transforms to the Cb transform block of the TU to generate a Cb coefficient block for the TU. The video encoder 20 can apply one or more transforms to the Cr transform block of the TU to generate a Cr coefficient block for the TU.

[0129] After generating coefficient blocks (e.g., luminance coefficient blocks, Cb coefficient blocks, or Cr coefficient blocks), video encoder 20 can quantize the coefficient blocks. Quantization typically refers to the process of quantizing transform coefficients to potentially reduce the amount of data used to represent the transform coefficients, thereby providing further compression. After quantizing the coefficient blocks, video encoder 20 can entropy encode the syntax elements indicating the quantized transform coefficients. For example, video encoder 20 can perform CABAC on the syntax elements indicating the quantized transform coefficients. Finally, video encoder 20 can output a bitstream comprising a bit sequence that forms a representation of coded frames and associated data; the bitstream is stored in storage device 32 or transmitted to target device 14.

[0130] After receiving the bitstream generated by video encoder 20, video decoder 30 can parse the bitstream to obtain syntax elements from the bitstream. Video decoder 30 can reconstruct the frames of the video data based at least in part on the syntax elements obtained from the bitstream. The process of reconstructing the video data is generally reciprocal to the encoding process performed by video encoder 20. For example, video decoder 30 can perform inverse transforms on the coefficient blocks associated with the TUs of the current CU to reconstruct the residual blocks associated with the TUs of the current CU. Video decoder 30 also reconstructs the coding blocks of the current CU by adding the samples of the prediction blocks for the PUs of the current CU to corresponding samples of the transform blocks of the TUs of the current CU. After reconstructing the coding blocks for each CU of a frame, video decoder 30 can reconstruct the frame.

[0131] As described above, video coding primarily uses two modes (i.e., intra-frame prediction (or intra prediction) and inter-frame prediction (or inter prediction)) to achieve video compression. It should be noted that IBC can be considered as intra prediction or a third mode. Between the two modes, inter prediction contributes more to coding efficiency than intra prediction due to the use of motion vectors to predict the current video block from reference video blocks.

[0132] However, as video data capturing technology is constantly improving and finer video block sizes are used to preserve details in the video data, the amount of data required to represent the motion vectors for the current frame also increases substantially. One way to overcome this challenge benefits from the fact that not only a group of neighboring CUs in both spatial and temporal domains have similar video data for prediction purposes, but also the motion vectors between these neighboring CUs are similar. Therefore, the motion information (e.g., motion vectors) of the spatial neighboring CUs and / or temporal collocated CUs can be used as an approximation of the motion information (e.g., motion vectors) of the current CU (which is also referred to as the “motion vector predictor” (MVP) of the current CU) by exploring their spatial and temporal correlations.

[0133] Instead of encoding the actual motion vectors of the current CUs determined by motion estimation unit 42 into the video bitstream as described above in connection with Figure 2 Subtracting the motion vector predictor of the current CU from the actual motion vector of the current CU to generate a motion vector difference (MVD) for the current CU. By doing so, the motion vectors determined by motion estimation unit 42 for each CU of the frame do not need to be encoded into the video bitstream, and the amount of data used to represent the motion information in the video bitstream can be reduced significantly.

[0134] Similar to the process of selecting a prediction block in a reference frame during inter prediction of a coding block, both video encoder 20 and video decoder 30 need to employ a set of rules for constructing a motion vector candidate list (also referred to as a "merge list") for a current CU using those potential candidate motion vectors associated with spatially neighboring CUs and / or temporally collocated CUs of the current CU, and then selecting one member from the motion vector candidate list as the motion vector predictor for the current CU. By doing so, there is no need to transmit the motion vector candidate list itself from video encoder 20 to video decoder 30, and the index of the selected motion vector predictor within the motion vector candidate list is sufficient for video encoder 20 and video decoder 30 to use the same motion vector predictor within the motion vector candidate list to encode and decode the current CU.

[0135] In general, the basic inter prediction scheme applied in VVC is almost the same as HEVC, except that several prediction tools are further extended, added and / or improved, such as extended merge prediction, MMVD and GPM.

[0136] Extended merge prediction With ever-improving video data capture technology and finer video block sizes for preserving details in video data, the amount of data needed to represent the motion vectors of a current picture also increases substantially. One approach to overcome this challenge is to use the motion information (e.g., motion vectors) of spatially neighboring CUs, temporally collocated CUs, etc. of the current CU as an approximation (e.g., prediction) of the motion information of the current CU, also referred to as the "motion vector predictor (MVP)" of the current CU.

[0137] Similar to the process of selecting a prediction block in a reference picture during inter prediction of a coding block, both video encoder 20 and video decoder 30 need to employ a set of rules for constructing a MVP candidate list for a current CU, and then selecting one MVP candidate from the MVP candidate list as the MVP of the current CU. By doing so, there is no need to transmit the MVP candidate list itself between video encoder 20 and video decoder 30, and the index of the selected MVP candidate from the MVP candidate list is sufficient for video encoder 20 and video decoder 30 to use the same MVP candidate selected from the MVP candidate list to encode and decode the current CU.

[0138] In VVC, the MVP candidate list is constructed by including the following five types of MVPs in order: — spatial MVPs from spatially neighboring CUs (i.e., spatial candidates); — temporal MVPs from temporally collocated CUs (i.e., temporal candidates); — history-based MVPs (HMVPs) from a first-in-first-out (FIFO) table; —Paired average MVP; and —Zero MVP.

[0139] The size of the MVP candidate list is signaled in the sequence parameter set header, and the maximum allowed size of the MVP candidate list is 6. For each CU encoded in merge mode, the index of the best MVP candidate is encoded using truncated unary binarization. The first binary bit of the index is encoded and decoded using the context, and the remaining binary bits of the index are encoded and decoded using bypass.

[0140] The derivation process for each type of MVP is provided below. Like HEVC, VVC also supports parallel derivation of the MVP candidate list for all CUs within a certain size region.

[0141] Deriving MVP from Spatial Candidates In addition to swapping the positions of the first two spatial candidates, VVC also selects from spatial candidates (e.g., Figure 5 The MVP derivation for the CU (which is adjacent to the current CU101) is the same as that in HEVC. From the location... Figure 5 Up to four spatial candidates are selected from the spatial candidates for the positions depicted (i.e., top position B0, left position A0, top right position B1, bottom left position A1, and top left position B2). The derivation is performed in the order of the CUs at positions B0, A0, B1, A1, and B2. The CU at position B2 is considered only if one or more CUs at positions B0, A0, B1, and A1 are unavailable (e.g., because said one or more CUs belong to other stripes or tiles) or are intra-coded.

[0142] After adding the CU at position B0 as a candidate to the merged candidate list, a redundancy check is needed for adding the remaining candidates to the merged candidate list. This ensures that candidates with the same motion information are excluded from the merged candidate list, thereby improving encoding and decoding efficiency. To reduce computational complexity, not all possible candidate pairs are considered in the redundancy check. Instead, only... Figure 6 Pairs are linked using arrowed lines, and a candidate is added to the merged candidate list only if the motion information of the candidate in the pair used for redundancy checking is different from the motion information of the candidate to be added. The spatial MVP derived from the candidates in the merged candidate list is added to the MVP candidate list.

[0143] MVP Derivation from Time Candidates During the MVP derivation from the time candidate, only one time candidate is added to the merge candidate list. Specifically, when deriving the MVP from that time candidate, it is based on whether it belongs to the current CU (e.g., Figure 7 The corresponding image of curr_CU 303 in (e.g., Figure 7The time candidate CU (e.g., col_pic 302) in the col_pic 302) is the same as the time candidate CU (e.g., Figure 7 The scaled motion vector is derived using col_CU 301 and added as a temporal MVP candidate to the MVP candidate list. The list of reference images and their indices used to derive the co-located CUs are explicitly transmitted via signals in the strip header. Figure 7 As shown, the scaled motion vector is obtained (i.e., scaled) from the motion vector of the co-located CU using the picture order count (POC) distance (i.e., tb and td), where tb is defined as the current picture (e.g., Figure 7 Reference image for curr_ref 304 (e.g., Figure 7 The difference between curr_ref305 in the image and the current image in the POC, and td is defined as the reference image of the co-located image (e.g., Figure 7 The POC difference between col_ref 306 and the corresponding image. The reference image index for the time candidate is set to zero.

[0144] Select the position of the current time candidate (i.e., the corresponding CU) in CU 401 between positions C0 and C1, such as... Figure 8 As described. If the CU at position C0 in the co-occurring frame is unavailable, intra-coded, or outside the current CTU line, then the CU at position C1 is used as the co-occurring CU to derive the temporal MVP candidate. Otherwise, the CU at position C0 is used as the co-occurring CU to derive the temporal MVP candidate.

[0145] Derivation of HMVP Candidates HMVP candidates are added to the MVP candidate list after the spatial MVP and temporal MVP. Motion information of previously encoded blocks is stored in the HMVP table and used as the MVP of the current CU. The table with multiple HMVP candidates is maintained during the encoding / decoding process. The table is reset (cleared) when a new CTU row is encountered. Whenever a non-sub-block inter-coded CU exists, the associated motion information is added to the last entry of the HMVP table as a new HMVP candidate.

[0146] The size of the HMVP table is set to 6. When a new HMVP candidate is inserted into the HMVP table, a constrained FIFO rule is used, where a redundancy check is first applied to find if a duplicate HMVP already exists in the HMVP table. If found, the duplicate HMVP is removed from the HMVP table, all subsequent HMVP candidates are shifted forward, and the duplicate HMVP is added to the last entry in the HMVP table.

[0147] HMVP candidates can be used in the MVP candidate list construction process. The latest few HMVP candidates in the HMVP table are checked in order and inserted in the MVP candidate list after the temporal MVP candidates. Redundancy check is applied to the HMVP candidates with respect to the spatial candidates and / or the temporal MVP candidates.

[0148] To reduce the number of redundancy check operations, the following simplifications are introduced: - The last two entries in the HMVP table are checked for redundancy with respect to the spatial MVP candidates derived from the spatial candidates at positions Al and Bl, respectively; and - The MVP candidate list construction process from HMVP candidates is terminated once the total number of available MVP candidates reaches the maximum allowed size of the MVP candidate list minus 1.

[0149] Derivation of pair-wise average MVP candidate The pair-wise average MVP candidate is generated by averaging the MVPs derived from a predefined pair of the first two merge candidates in the existing merge candidate list. The first merge candidate in the predefined pair can be defined as p0Cand and the second merge candidate in the predefined pair can be defined as plCand. For each reference picture list, the average motion vector is calculated separately according to the availability of the motion vectors of p0Cand and plCand. If both motion vectors are available for one reference picture list, the two motion vectors are averaged even when they point to different reference pictures, and the reference picture of the average motion vector is set to the reference picture of p0Cand; if only one motion vector is available for one reference picture list, the motion vector is used directly; if no motion vector is available for one reference picture list, the motion vector and the reference picture index for that reference picture list remain invalid.

[0150] Zero MVP When the MVP candidate list is not full after adding the pair-wise average MVP candidate, a zero MVP is inserted at the end of the MVP candidate list until the maximum allowed size of the MVP candidate list is reached.

[0151] MMVD As described above, in merge mode, motion information (i.e., MVP candidates) is implicitly derived from the MVP candidate list constructed for the current CU and directly used as the MV of the current CU to generate predicted samples for the current CU. This may result in a certain error between the actual MV of the current CU and the implicitly derived MVP. To improve the accuracy of the MV of the current CU, MMVD is introduced in VVC, where the motion vector difference (MVD) of the current CU is added to the implicitly derived MVP to obtain the MV of the current CU. After transmitting the regular merge flag, the MMVD flag is signaled to specify whether the MMVD mode is used for the current CU.

[0152] In MMVD mode, after selecting an MVP candidate from the first two MVP candidates in the MVP candidate list, MMVD information is transmitted by signaling. The MMVD information includes an MMVD candidate flag for specifying which of the first two MVP candidates is selected as the basis for the MV, a distance index for indicating the motion amplitude information of the MVD, and a direction index for indicating the motion direction information of the MVD.

[0153] The distance index indicating the motion amplitude information of the specified MVD is compared with the reference image of the current CU pointed to by the selected MVP candidate (e.g., Figure 9 The starting point in the L0 reference image 501 or L1 reference image 503 (e.g., by) Figure 9 The dashed circles in the table represent predefined offsets, and MVD can be derived from these offsets and added to selected MVP candidates. Table 1 below specifies the relationship between distance indices and predefined offsets.

[0154]

[0155] Table 1 The direction index specifies the sign of the MVD, which indicates the direction of the MVD relative to the starting point. Table 2 specifies the relationship between the direction index and the predefined sign. In some examples, the meaning of the sign of the MVD can vary depending on the information of the selected MVP candidate. When the selected MVP candidate is a non-predicted MV or a bi-predicted MV where both MVs point to the same side of the current picture (i.e., the POCs of both reference pictures of the current picture (e.g., the reference pictures of List 0 and List 1, which are also referred to as L0 reference picture and L1 reference picture, respectively) are both greater than the POC of the current picture or both are smaller than the POC of the current picture), the sign in Table 2 specifies the sign of the MVD added to the selected MVP candidate. When the selected MVP candidate is a bi-predicted MV where both MVs point to different sides of the current picture (i.e., the POC of one reference picture of the current picture is greater than the POC of the current picture and the POC of the other reference picture of the current picture is smaller than the POC of the current picture), if the POC distance of the L0 reference picture (i.e., the POC distance between the L0 reference picture and the current picture) is greater than the POC distance of the L1 reference picture (i.e., the POC distance between the L1 reference picture and the current picture), the sign in Table 2 specifies the sign of the List 0 MVD (MVD0) added to the List 0 MVP (MVP0) of the selected MVP candidate and the sign of the List 1 MVD (MVD1) added to the List 1 MVP (MVP1) of the selected MVP candidate is opposite to the sign in Table 2; otherwise, if the POC distance of the L1 reference picture is greater than the POC distance of the L0 reference picture, the sign in Table 2 specifies the sign of the MVD1 added to the MVP1 and the sign of the MVD0 added to the MVP0 is opposite to the sign in Table 2.

[0156]

[0157] Table 2 The MVD is scaled according to the POC distance. If the POC distances of the L0 reference picture and the L1 reference picture are the same, the MVD does not need to be scaled. Otherwise, if the POC distance of the L0 reference picture is greater than the POC distance of the L1 reference picture, the MVD1 is scaled. If the POC distance of the L1 reference picture is greater than the POC distance of the L0 reference picture, the MVD0 is scaled.

[0158] GPM In VVC, GPM is supported for inter prediction. GPM is signaled using a CU-level flag as one of the merge modes, other merge modes include regular merge mode, MMVD mode, CIIP mode, and subblock merge mode. For each possible CU size ( wherein, ) (excluding 8 64 and 64 8), GPM supports 64 partitions in total.

[0159] When GPM is used, a CU is split into two parts by a straight line positioned geometrically. The position of the split line is mathematically derived according to the angle and offset parameters of the specific partition. Each part of the CU obtained by geometric partitioning is inter predicted using its own motion; and each partition is only allowed uni-directional prediction, i.e., each part has one motion vector and one reference index. The uni-directional prediction motion constraint is applied to ensure that each CU only needs two kinds of motion-compensated prediction as in the traditional bi-directional prediction.

[0160] If GPM is used for the current CU, the geometry partition index and two merge indices (one for each partition) indicating the partition mode of the geometry partition (indicating the angle and offset of the geometry partition) are further signaled.

[0161] The uni-prediction candidate list is derived directly from the merge candidate list constructed according to the extended merge prediction process described above. Let n denote the index of a uni-prediction motion vector in the uni-prediction candidate list. The LX motion vector (X equals the parity of n) of the nth merge candidate in the merge candidate list is used as the nth uni-prediction motion vector of GPM. These motion vectors are marked with “x” in Figure 10 . In the case where the corresponding LX motion vector of the nth merge candidate in the merge candidate list does not exist, the L(1 - X) motion vector of the same merge candidate is used as the uni-prediction motion vector of GPM.

[0162] CIIP In VVC, when a CU is coded in merge mode, an additional flag is signaled to indicate whether CIIP mode is applied to the current CU if the CU contains at least 64 luma samples (i.e., the width of the CU multiplied by the height of the CU is equal to or greater than 64) and if both the width of the CU and the height of the CU are less than 128 luma samples. In CIIP mode, the prediction signal is obtained by combining the inter prediction signal with the intra prediction signal. The inter prediction signal in CIIP mode is derived using the same inter prediction process applied in the regular merge mode; and the intra prediction signal in CIIP mode is derived after the regular intra prediction process with the planar mode. Then, the intra prediction signal and the inter prediction signal are combined using a weighted average, where the weight values are calculated according to the coding modes of the top and left neighboring blocks of the current CU 1601 as shown in Figure 11 — If the top neighboring block is available and has been intra coded, isIntraTop is set to 1, otherwise it is set to 0; ​— If the left neighboring block is available and has been intra coded, then isIntraLeft is set to 1, otherwise, isIntraLeft is set to 0; — If (isIntraLeft + isIntraTop) is equal to 2, then the weight value is set to 3; — Otherwise, if (isIntraLeft + isIntraTop) is equal to 1, then the weight value is set to 2; — Otherwise, the weight value is set to 1.

[0163] — Prediction signal in CIIP mode is derived as follows:

[0164] where, is the inter prediction signal in CIIP mode, is the intra prediction signal in CIIP mode, is the weight value, and » denotes a right shift operation.

[0165] Intra block copy in Versatile Video Coding (VVC) Intra block copy (IBC) is a tool adopted in the HEVC extension on SCC. IBC significantly improves the coding efficiency of screen content material. Since IBC mode is implemented as a block-level coding mode, block matching (BM) is performed at the encoder to find the best block vector (or motion vector) for each CU. Here, the block vector is used to indicate the displacement from the current block to the reference block that has been reconstructed within the current picture. The luma block vector of an IBC coded CU has integer precision. The chroma block vector is also rounded to integer precision. When combined with AMVR, the IBC mode can switch between 1-pixel motion vector precision and 4-pixel motion vector precision. An IBC coded CU is considered as a third prediction mode different from the intra prediction mode or the inter prediction mode. IBC mode is applicable to a CU whose width and height are both less than or equal to 64 luma samples.

[0166] At the encoder side, hash-based motion estimation is performed for IBC. The encoder performs RD check for blocks whose width or height is not larger than 16 luma samples. For non-merge mode, block vector search is first performed using hash-based search. If hash search does not return a valid candidate, a local search based on block matching will be performed.

[0167] In hash-based search, the hash key match (32-bit CRC) between the current block and the reference block is extended to all allowed block sizes. The hash key calculation for each position in the current picture is based on 4x4 sub-blocks. For larger sizes of the current block, the hash key is determined to match the hash key of the reference block when all hash keys of all 4x4 sub-blocks match the hash keys in the corresponding reference positions. If multiple reference blocks are found whose hash keys match the hash key of the current block, the block vector cost of each matching reference is calculated and the reference with the smallest cost is selected.

[0168] In block matching search, the search range is set to cover both the previous CTU and the current CTU.

[0169] At CU level, the IBC mode is signaled using a flag, and it can be signaled as IBC AMVP mode or IBC skip / merge mode as follows: IBC skip / merge mode: The merge candidate index is used to indicate which block vectors from the list of neighboring candidate IBC coded blocks are used to predict the current block. The merge list consists of spatial candidates, HMVP candidates, and pairwise candidates.

[0170] IBC AMVP mode: The block vector difference is coded in the same way as the motion vector difference. The block vector prediction method uses two candidates as the predictor, one from the left neighboring block and one from the above neighboring block (if IBC coded). When neither neighboring block is available, the default block vector will be used as the predictor. A flag is signaled to indicate the block vector predictor index.

[0171] IBC reference region To reduce memory consumption and decoder complexity, IBC in VVC only allows the reconstructed part of the pre-defined region that includes the region of the current CTU and a certain region of the left CTU. Figure 12 The reference region of IBC mode is illustrated, where each block represents 64x64 luma sample unit.

[0172] Depending on the location of the current coded CU position within the current CTU, the following applies: If the current block falls into the top-left 64x64 block of the current CTU, in addition to the samples in the current CTU that have already been reconstructed, the current block can also refer to the reference samples in the bottom-right 64x64 block of the left CTU. Using the CPR mode, the current block can also refer to the reference samples in the bottom-left 64x64 block of the left CTU and the reference samples in the top-right 64x64 block of the left CTU.

[0173] If the current block falls into the top-left 64 x 64 block of the current CTU, in addition to the already reconstructed samples in the current CTU, the current block can also refer to the reference samples in the bottom-left 64 x 64 block of the left CTU using the CPR mode in case the luma position (0, 64) is not yet reconstructed with respect to the current CTU; otherwise, the current block can also refer to the reference samples in the bottom-right 64 x 64 block of the left CTU using the CPR mode.

[0174] If the current block falls into the bottom-left 64 x 64 block of the current CTU, in addition to the already reconstructed samples in the current CTU, the current block can also refer to the reference samples in the top-right 64 x 64 block and the bottom-right 64 x 64 block of the left CTU using the CPR mode in case the luma position (64, 0) is not yet reconstructed with respect to the current CTU; otherwise, the current block can also refer to the reference samples in the bottom-right 64 x 64 block of the left CTU using the CPR mode.

[0175] If the current block falls into the bottom-right 64 x 64 block of the current CTU, the current block can only refer to the already reconstructed samples in the current CTU using the CPR mode.

[0176] This restriction allows the IBC mode to be implemented using the local on-chip memory of hardware implementations.

[0177] Interaction of IBC with other coding tools The interaction of the IBC mode with other inter coding tools in VVC (such as paired merge candidate, history-based motion vector predictor (HMVP), combined intra / inter prediction mode (CIIP), merge mode with motion vector difference (MMVD), and geometric partition mode (GPM)) is as follows: IBC can be used together with paired merge candidate and HMVP. A new paired IBC merge candidate can be generated by averaging the two IBC merge candidates. For HMVP, the IBC motion is inserted into the history buffer for future reference.

[0178] IBC cannot be used in combination with the following inter tools: affine motion, CIIP, MMVD, and GPM.

[0179] When DUAL_TREE partitioning is used, IBC is not allowed for chroma coding blocks.

[0180] Unlike in the HEVC screen content coding extension, the current picture is no longer included as one of the reference pictures in the reference picture list 0 for IBC prediction. The derivation process of the motion vector of the IBC mode excludes all neighboring blocks in inter mode and vice versa. The following IBC design aspects are applied: IBC shares the same process as regular MV merge, including pair-wise merge candidates and history-based motion predictor values, but does not allow TMVP and zero vector because they are not valid for IBC mode.

[0181] A separate HMVP buffer (5 candidates each) is used for regular MV and IBC.

[0182] The block vector constraints are enforced in the form of bitstream conformance constraints, the encoder needs to make sure that no invalid vector exists in the bitstream, and merge should not be used if the merge candidate is invalid (out of range or 0). This bitstream conformance constraint is expressed in terms of a virtual buffer, as described below.

[0183] For deblocking, IBC is treated as an inter mode.

[0184] If the current block is coded using IBC prediction mode, AMVR does not use quarter-pel; instead, AMVR is signaled to indicate only if the MV is inter-pel or 4 integer-pel.

[0185] The number of IBC merge candidates can be signaled in the slice header separately from the number of regular merge candidates, subblock merge candidates, and geometric merge candidates.

[0186] The virtual buffer concept is used to describe the allowable reference region and valid block vectors for IBC prediction mode. Denote the CTU size as ctbSize, the virtual buffer ibcBuf has a width of wIbcBuf = 128 x 128 / ctbSize and a height of hIbcBuf = ctbSize. For example, for a CTU size of 128 x 128, the size of ibcBuf is also 128 x 128; for a CTU size of 64 x 64, the size of ibcBuf is 256 x 64; and for a CTU size of 32 x 32, the size of ibcBuf is 512 x 32.

[0187] The size of VPDU is min(ctbSize, 64) in each dimension, W v = min(ctbSize, 64).

[0188] The virtual IBC buffer ibcBuf is maintained as follows.

[0189] When starting to decode each CTU row, the entire ibcBuf is flushed with invalid values 1. The entire ibcBuf is flushed with invalid values.

[0190] ibcBuf[ x ][ y ] = recSample[ x ][ y ] 1, where x = xVPDU%wIbcBuf,..., xVPDU% wIbcBuf + W v 1; y = yVPDU%ctbSize,..., yVPDU%ctbSize + W v 1.

[0191] After decoding, the CU contains (x, y) relative to the picture top-left corner, set ibcBuf[ x % wIbcBuf ][ y % ctbSize ] = recSample[ x ][ y ] For a block covering coordinates (x, y), the block is valid if the following holds for the block vector bv = (bv[0], bv[1]) ibcBuf[ (x + bv[0])% wIbcBuf][ (y + bv[1]) % ctbSize ] should not be equal to 1.

[0192] Intra block copy in enhanced compression mode (ECM) In ECM, IBC is improved in several ways.

[0193] IBC merge / AMVP list construction The IBC merge / AMVP list construction is modified as follows: An IBC merge / AMVP candidate can only be inserted into the IBC merge / AMVP candidate list if it is valid.

[0194] The top-right, bottom-left and top-left spatial candidates as well as one pair-wise average candidate can be added to the IBC merge / AMVP candidate list.

[0195] Template-based adaptive reordering (ARMC-TM) is applied to the IBC merge list.

[0196] The HMVP table size for IBC is increased to 25. After deriving up to 20 IBC merge candidates by full pruning, they are reordered together. After reordering, the first 6 candidates with the lowest template matching cost are selected as the final candidates in the IBC merge list.

[0197] ​Zero vector candidates for filling the IBC merge / AMVP list are replaced by a set of BVP candidates located in the IBC reference region. The zero vector is invalid as a block vector in IBC merge mode and thus, it is discarded as a BVP in the IBC candidate list.

[0198] Three candidates are located on the nearest corners of the reference region and three additional candidates are determined in the middle of the three sub-regions (A, B and C) with coordinates determined by the width and height of the current block and the ΔX and ΔY parameters as depicted in Figure 13

[0199] Block vector candidates for IBC intra-TMP derivation In this method, block vectors (BVs) derived from intra template matching prediction (IntraTMP) are used for intra block copy (IBC). The stored IntraTMP BVs of neighboring blocks are used as spatial BV candidates in the IBC BV candidate list construction along with the IBC BVs.

[0200] IntraTMP block vectors are stored in the IBC block vector buffer and the current IBC block can use both the IBC BV and the IntraTMP BV of a neighboring block as BV candidates for the IBC BV candidate list. The IntraTMP block vector is added as a spatial candidate to the IBC block vector candidate list.

[0201] IBC with template matching Template matching is used in IBC for both IBC merge mode and IBC AMVP mode.

[0202] The IBC-TM merge list is modified compared to the list used by the regular IBC merge mode so that the candidates are selected according to a pruning method and the motion distance between the candidates is the same as for the regular TM merge mode. The zero motion at the end is replaced by the motion vectors of the left side (-W, 0), the top (0, -H) and the top-left (-W, -H) where W is the width of the current CU and H is the height of the current CU.

[0203] In IBC-TM merge mode, the selected candidates are refined using the template matching method before the RDO or the decoding process. The IBC-TM merge mode competes with the regular IBC merge mode and the TM-merge flag is signaled.

[0204] In IBC-TM AMVP mode, up to 3 candidates are selected from the IBC-TM merge list. Each of the 3 selected candidates is refined using the template matching method and ordered according to their resulting template matching cost. Then, only the first 2 are considered as usual in the motion estimation process.

[0205] ​The template matching refinements for IBC-TM merge mode and AMVP mode are very simple, because IBC motion vectors are constrained (i) to be integer, and (ii) to be within the reference region, as shown in Figure 12 Thus, in IBC-TM merge mode, all refinements are performed with integer precision, while in IBC-TM AMVP mode, they are performed with integer or 4-pel precision depending on the AMVR value. This refinement only accesses samples without interpolation. In both cases, the refinement motion vector and the used template in each refinement step must obey the constraints of the reference region.

[0206] IBC reference region The reference region of IBC is extended to two CTU rows above. Figure 14 The reference region for coding a CTU (m, n) is illustrated. Specifically, for a CTU (m, n) to be coded, the reference region includes CTUs with indices (m - 2, n - 2)... (W, n - 2), (0, n - 1)... (W, n - 1), (0, n)... (m, n), where W denotes the maximum horizontal index within the current tile, slice, or picture. This setup ensures that IBC does not require extra memory on the current ETM platform when the CTU size is 128. The per-sample block vector search (or local search) range is limited to [–(C « 1), C » 2] in the horizontal direction and [–C, C » 2] in the vertical direction to accommodate the reference region extension, where C denotes the CTU size.

[0207] IBC merge mode with block vector difference The IBC merge mode with block vector difference is adopted in ECM. The distance set is {1 pixel, 2 pixel, 4 pixel, 8 pixel, 12 pixel, 16 pixel, 24 pixel, 32 pixel, 40 pixel, 48 pixel, 56 pixel, 64 pixel, 72 pixel, 80 pixel, 88 pixel, 96 pixel, 104 pixel, 112 pixel, 120 pixel, 128 pixel}, and the BVD directions are two horizontal directions and two vertical directions.

[0208] The base candidate is selected from the top five candidates in the reordered IBC merge list. And all possible MBVD refinement positions (20 x 4) for each base candidate are reordered based on the SAD cost between the template (one row above the current block and one column left of the current block) and its reference for each refinement position. Finally, the top 8 refinement positions with the lowest template SAD cost are kept as available positions, thus used for MBVD index coding.

[0209] IBC adaptation for camera captured content When IBC is adapted for camera-captured content, the IBC reference range is reduced from 2 CTU rows to 2 x 128 rows, as shown in Figure 15 At the encoder side, to reduce complexity, the local search range is set to be horizontal [-8, 8] and vertical [-8, 8] centered at the first block vector predictor of the current CU. This encoder modification does not apply to SCC sequences.

[0210] Reconstruction Reordering IBC (RR-IBC) IBC coded blocks allow a reconstruction reordering IBC (RR-IBC) mode. When RR-IBC is applied, the samples in the reconstructed block are flipped according to the flipping type of the current block. At the encoder side, the original block is flipped before motion search and residual calculation, while the predicted block is derived without flipping. At the decoder side, the reconstructed block is flipped back to recover the original block.

[0211] For RR-IBC coded blocks, two flipping methods are supported, i.e., horizontal flipping and vertical flipping. First, a syntax flag of IBC AMVP coded blocks is signaled to indicate whether the reconstruction is flipped, and if the reconstruction is flipped, another flag specifying the flipping type is further signaled. For IBC merge, the flipping type is inherited from the neighboring block without syntax signaling. Considering the horizontal or vertical symmetry, the current block and the reference block are usually horizontally or vertically aligned. Therefore, when horizontal flipping is applied, the vertical component of BV is not signaled and it is inferred to be equal to 0. Similarly, when vertical flipping is applied, the horizontal component of BV is not signaled and it is inferred to be equal to 0.

[0212] To better exploit the symmetry property, a flipping-aware BV adjustment method is applied to refine the block vector candidates. For example, as shown in Figure 42A or Figure 42B , (xnbr, ynbr) and (xcur, ycur) denote the coordinates of the center sample of the neighboring block and the current block, respectively, and BVnbr and BVcur denote the BVs of the neighboring block and the current block, respectively. Instead of directly inheriting the BV from the neighboring block, in the case that the neighboring block is coded with horizontal flipping, the horizontal component of BVcur is calculated by adding a motion shift to the horizontal component of BVnbr (denoted as BVnbrh), i.e., BVcurh = 2(xnbr - xcur) + BVnbrh. Similarly, in the case that the neighboring block is coded with vertical flipping, the vertical component of BVcur is calculated by adding a motion shift to the vertical component of BVnbr (denoted as BVnbrv), i.e., BVcurv = 2(ynbr - ycur) + BVnbrv.

[0213] IBC with geometric partitioning Intra block copy with geometric partition mode (IBC-GPM) is a coding tool that partitions a CU geometry into two sub-partitions. IBC and intra prediction are used to generate the prediction signal for the two sub-partitions. IBC-GPM can be applied to regular IBC merge mode or IBC TM merge mode. For intra prediction, the same method as GPM with inter and intra prediction is used to construct the intra prediction mode (IPM) candidate list, and the IPM candidate list size is predefined to 3. There are 48 geometric partition modes in total, which are divided into two geometric partition mode sets as follows: Table 3: Geometric partition modes in the first geometric partition mode set

[0214] Table 4: Geometric partition modes in the second geometric partition mode set

[0215] When IBC-GPM is used, IBC-GPM geometric partition mode set flag is signaled to indicate whether the first geometric partition mode set or the second geometric partition mode set is selected, followed by the geometric partition mode index. IBC-GPM intra flag is signaled to indicate whether intra prediction is used for the first sub-partition. When intra prediction is used for the sub-partition, intra prediction mode index is signaled. When IBC is used for the sub-partition, merge index is signaled.

[0216] Combination of CIIP with TIMD and TM merge In CIIP mode, the prediction samples are generated by weighting the inter prediction signal predicted using the CIIP-TM merge candidate and the intra prediction signal predicted using the intra prediction mode derived using TIMD. This method is applied only to coding blocks with an area smaller than or equal to 1024.

[0217] TIMD derivation method is used to derive the intra prediction mode in CIIP. Specifically, the intra prediction mode with the smallest SATD value in the TIMD mode list is selected and mapped to one of the 67 regular intra prediction modes.

[0218] In addition, it is proposed to modify the weights (wIntra, wInter) of the two tests if the derived intra prediction mode is an angular mode. For near horizontal modes (2 <= angle mode index < 34), the current block is vertically divided as shown in Figure 16A For near vertical modes (34 <= angle mode index <= 66), the current block is horizontally divided as shown in Figure 16B

[0219] The (wIntra, wInter) for different sub-blocks are shown in Table 5.​

[0220]

[0221] Table 5. Modified weights for angle mode.

[0222] Using CIIP-TM, a CIIP-TM merge candidate list is established for the CIIP-TM pattern. The merge candidates are refined through template matching. The CIIP-TM merge candidates are also reordered into regular merge candidates using the ARMC method. The maximum number of CIIP-TM merge candidates is two.

[0223] Multiple Hypothesis Prediction (MHP) In multi-hypothesis inter-frame prediction mode, in addition to the traditional bidirectional prediction signal, one or more additional motion-compensated prediction signals are transmitted via signal transmission. The resulting overall prediction signal is obtained by sample-by-sample weighted superposition. The bidirectional prediction signal is utilized. and the first additional inter-frame prediction signal / hypothesis The predicted signal is obtained as follows. : (2) Weighting factor The new syntax element add_hyp_weight_idx specifies the weight based on the mapping presented in Table 6:

[0224] Table 6. add_hyp_weight_idx and The mapping between them.

[0225] Similar to the above, more than one additional prediction signal can be used. The resulting overall prediction signal is iteratively accumulated with each additional prediction signal.

[0226] (3) The resulting overall prediction signal is obtained as the final (i.e., has the largest index) n of Within this mode, up to two additional prediction signals can be used (i.e., n Limited to 2).

[0227] The motion parameters for each additional prediction hypothesis can be explicitly signaled by specifying a reference index, a motion vector prediction value index, and a motion vector difference, or implicitly signaled by specifying a merging index. A separate multiple hypothesis merging flag distinguishes between these two signaling modes.

[0228] For inter AMVP mode, MHP is applied only when unequal weights in BCW are selected in bi-prediction mode.

[0229] A combination of MHP and BDOF is possible, however BDOF is applied only to the bi-predicted signal part of the prediction signal (i.e. the first two hypotheses in normal).

[0230] Geometric partition mode (GPM) in ECM GPM with merge motion vector difference (MMVD) GPM in VVC is extended by applying motion vector refinement on top of the existing GPM uni-directional MV. First a flag is signaled for the GPM CU to specify whether the mode is used or not. If the mode is used, each geometric partition of the GPM CU can further decide whether to signal MVD or not. If MVD is signaled for a geometric partition, the motion of the partition is further refined by the signaled MVD information after the GPM merge candidate is selected. All other procedures remain the same as in GPM.

[0231] Similar to in MMVD, MVD is signaled as a pair of distance and direction. Nine candidate distances (¼ pixel, ½ pixel, 1 pixel, 2 pixel, 3 pixel, 4 pixel, 6 pixel, 8 pixel, 16 pixel) and eight candidate directions (four horizontal / vertical directions and four diagonal directions) are involved in GPM with MMVD (GPM-MMVD). In addition, when pic_fpel_mmvd_enabled_flag is equal to 1, the MVD is left shifted by 2 bits as in MMVD.

[0232] GPM with template matching (TM) Template matching is applied to GPM. When GPM mode is enabled for a CU, a CU-level flag is signaled to indicate whether TM is applied to both geometric partitions or not. The motion information of each geometric partition is refined using TM. When TM is selected, the template is constructed using the left neighboring sample, the above neighboring sample, or both the left neighboring sample and the above neighboring sample according to the partition angle as shown in Table 7. Then the motion is refined by minimizing the difference between the current template and the template in the reference picture using the same search pattern of the merge mode with the semi-pixel interpolation filter disabled.

[0233]

[0234] Table 7. Template for the 1st geometric partition and the 2nd geometric partition, where A denotes using the above sample, L denotes using the left sample, and L+A denotes using both the left sample and the above sample.

[0235] GPM candidate list is constructed as follows: 1. Interleaved List 0 and List 1 MV candidates are derived directly from the regular merge candidate list, where List 0 MV candidates have higher priority than List 1 MV candidates. A pruning method is applied where an adaptive threshold based on the current CU size is used to remove redundant MV candidates.

[0236] 2. Interleaved List 1 and List 0 MV candidates are further derived directly from the regular merge candidate list, where List 1 MV candidates have higher priority than List 0 MV candidates. The same pruning method with adaptive threshold is also applied to remove redundant MV candidates.

[0237] 3. Zero MV candidates are padded until the GPM candidate list is full.

[0238] GPM-MMVD and GPM-TM are enabled only for one GPM CU. This is achieved by first signaling a GPM-MMVD syntax. When both GPM-MMVD control flags are equal to false (i.e., GPM-MMVD is disabled for both GPM partitions), a GPM-TM flag is signaled to indicate whether template matching is applied to both GPM partitions. Otherwise (at least one GPM-MMVD flag is equal to true), the value of the GPM-TM flag is inferred to be false.

[0239] GPM with inter and intra prediction In GPM with inter and intra prediction, the final prediction samples are generated by weighting the inter and intra prediction samples for each GPM sub-region. The inter prediction samples are derived by inter GPM, while the intra prediction samples are derived by an intra prediction mode (IPM) candidate list and an index signaled from the encoder. The IPM candidate list size is pre-defined to be 3. The available IPM candidates are the parallel angular mode (parallel mode) for GPM block boundary, the vertical angular mode (vertical mode) for GPM block boundary, and the planar mode as shown in Figures 17A to 17D In addition, GPM with inter and intra prediction as shown in Figure 17D is restricted to reduce the signaling overhead of IPM and avoid the increase of intra prediction circuit size on hardware decoders. Furthermore, direct motion vector and IPM storage are introduced in the GPM mixed region to further improve the coding performance.

[0240] In IPM derivation based on DIMD and neighboring mode, parallel mode is registered first. Therefore, at most two IPM candidates derived from decoder-side intra mode derivation (DIMD) method and / or neighboring block can be registered if there is no identical IPM candidate in the list. As for the derivation of neighboring mode, there are at most five positions of neighboring blocks available, but these positions are limited by the angle of GPM block boundary (as shown in Table 8), which has been used for GPM with template matching (GPM-TM).

[0241]

[0242] Table 8. Positions of neighboring blocks available for IPM candidate derivation based on the angle of GPM block boundary. A and L denote the top and left side of the prediction block.

[0243] GPM-Intra can be combined with GPM with merge using motion vector difference (GPM-MMVD). TIMD is used on the IPM candidates of GPM-Intra to further improve the coding performance. The parallel mode can be registered first, and then the IPM candidates of TIMD, DIMD and neighboring block are registered.

[0244] GPM partition mode reordering based on template matching In GPM partition mode reordering based on template matching, given the motion information of the current GPM block, the corresponding TM cost value of GPM partition mode is calculated. Then, all GPM partition modes are reordered in ascending order based on the TM cost value. Instead of sending GPM partition mode, the index indicating where the exact GPM partition mode is located in the reordered list is signaled using Golomb-Rice code.

[0245] The reordering method of GPM partition mode is a two-step process performed after the generation of the corresponding reference templates of two GPM partitions in the coding unit, as follows: • The GPM partition edge is extended into the reference templates of two GPM partitions, resulting in 64 reference templates, and the corresponding TM cost of each of the 64 reference templates is calculated; • The GPM partition modes are reordered in ascending order based on their TM cost values, and the best 32 are marked as available partition modes.

[0246] As shown in Figure 18 The edge on the template is extended from the edge of the current CU, but the GPM blending process is not used for the template area on this edge.

[0247] After the ascending reordering using TM cost, the index is signaled.

[0248] Geometric partition mode (GPM) with adaptive blending In VVC, the final prediction samples are generated by blending the predictions of the two prediction signals using a weighted average. Two integer blending matrices (W0 and W1) are used. The weights in the GPM blending matrix are derived from a ramp function based on the displacement from the prediction sample position to the GPM partition boundary. The blending region size is fixed to two (2 samples on each side of the GPM partition split boundary).

[0249] The blending process in ECM is improved by adding four additional blending region sizes (one quarter, one half, two times, and four times of the existing region size), as shown in Figure 36 A CU-level flag is signaled to represent the selected blending region size. In addition, an extended weighting precision is utilized, where the maximum value of the weight is changed from 8 (in VVC) to 32 to accommodate the extended blending region size.

[0250] Spatial Geometry Partition Mode (SGPM) SGPM is an intra mode of inter coding tool similar to GPM, where two prediction parts are generated according to an intra prediction process. In this mode, a candidate list is established, where each entry contains one partition split mode and two intra prediction modes, as shown in Figures 37A to 37C The length of the candidate list is set to be equal to 16. The selected candidate index is signaled.

[0251] The list is reordered using a template (RANKING OF CANDIDATE LIST Figure 38 ), where the SAD between the prediction and reconstruction of the template is used for the ranking. In one example, the template size is fixed to 1. In some other examples, the template size can be set differently.

[0252] For each partition mode, the same intra-inter GPM list derivation is used to derive an IPM list for each part. The IPM list size is set to 3. In this list, the TIMD derivation mode is replaced by 2 derivation modes with horizontal and vertical orientation.

[0253] The SGPM mode is applied to restricted block sizes: 4 <= width <= 64, 4 <= height <= 64, width < height x 8, height < width x 8, width x height >= 32.

[0254] Adaptive blending is also used for spatial GPM, where the blending depth τ as shown in Figure 39 is derived as follows: If min(width, height) == 4, then 1 / 2τ is selected; Otherwise, if min(width, height) == 8, then τ is selected; Otherwise, if min(width, height) == 16, select 2T; Otherwise, if min(width, height) == 32, select 4T; Otherwise, select 8T.

[0255] Intra template matching Intra template matching prediction (Intra-TMP) is a special intra prediction mode that copies from the reconstructed part of the current frame the best prediction block that matches the current template. For a predefined search range, the encoder searches in the reconstructed part of the current frame for the template that is most similar to the current template and uses the corresponding block as the prediction block. The encoder then signals the use of this mode and the same prediction operation is performed at the decoder side.

[0256] The prediction signal is generated by matching the L-shaped causal neighbor of the current block with another block in a predefined search area in Figure 19 consisting of: R1: the current CTU R2: the top-left CTU R3: the top CTU R4: the left CTU The sum of absolute differences (SAD) is used as cost function.

[0257] Within each region, the decoder searches for the template that has the smallest SAD with respect to the current template and uses its corresponding block as the prediction block.

[0258] The dimensions of all regions (SearchRange_w, SearchRange_h) are set proportionally to the block dimensions (BlkW, BlkH) so that each pixel has a fixed number of SAD comparisons. That is: SearchRange_w = a x BlkW SearchRange_h = a x BlkH where ‘a’ is a constant that controls the gain / complexity trade-off. In practice, ‘a’ is equal to 5.

[0259] The Intra Template Matching tool is enabled for CUs with width and height dimensions smaller than or equal to 64. This maximum CU size for Intra Template Matching is configurable.

[0260] When DIMD is not used for the current CU, the Intra Template Matching prediction mode is signaled at the CU level by a dedicated flag.

[0261] ​​Fusion of template-based intra mode derivation (TIMD) For each intra prediction mode in the MPMs, the SATD between the prediction samples of the template and the reconstructed samples is calculated. The first two intra prediction modes with the smallest SATD are selected as the TIMD modes. After applying the PDPC process, the two TIMD modes are fused with weights and the current CU is coded using this weighted intra prediction. Position dependent intra prediction combination (PDPC) is included in the derivation of the TIMD modes.

[0262] The cost of the two selected modes is compared with a threshold, in the test, the cost factor 2 is applied as follows: costMode2 < 2 x costMode1.

[0263] If this condition is true, fusion is applied, otherwise only mode 1 is used.

[0264] The weights of the modes are calculated according to their SATD cost as follows: weight1 = costMode2 / (costMode1+ costMode2) weight2 = 1 - weight1 The division operation is done using the same look-up table (LUT) based integerization scheme as used for CCLM.

[0265] Local illumination compensation (LIC) LIC is an inter prediction technique to model the local illumination variation between the current block and its prediction block as a function of the local illumination variation between the current block template and the reference block template. The parameters of the function can be expressed in a scale a and an offset b, which form a linear equation (i.e., a x p[x] + b) to compensate the illumination variation, where p[x] is the MV pointed reference sample at position x on the reference picture. When wraparound motion compensation is enabled, the wraparound offset should be considered to clip the MV. Since a and b can be derived based on the current block template and the reference block template, they do not require signaling overhead except for the LIC flag to signal the indication of using LIC for AMVP mode.

[0266] The local illumination compensation proposed in JVET-O0066 is used for uni-prediction inter CUs by the following modifications.

[0267] Intra neighboring samples can be used for LIC parameter derivation; LIC is disabled for blocks with less than 32 luma samples; For both non-subblock mode and affine mode, the LIC parameter derivation is performed based on the template block samples corresponding to the current CU, instead of the partial template block samples corresponding to the first top-left 16 × 16 unit. The sample of the reference block template is generated using MC and the block MV without rounding it to integer pixel precision.

[0268] TM-based MMVD and affine MMVD reordering MMVD offsets are extended for MMVD mode and affine MMVD mode. Figure 40 The additional refinement positions along k × π / 8 diagonal angles are shown in Fig. 1, thus increasing the number of directions from 4 to 16. Second, all possible MMVD refinement positions (16 × 6) for each base candidate are reordered based on the SAD cost between the template (one row above the current block and one column left of the current block) and its reference for each refinement position. Finally, the top 1 / 8 refinement position with the minimum template SAD cost is kept as the available position, thus used for MMVD index coding. The MMVD index is rice code binarized with a parameter equal to 2. The affine MMVD reordering is extended, where additional refinement positions along k × π / 4 diagonal angles are added. After reordering, the top 1 / 2 refinement position with the minimum template SAD cost is kept.

[0269] The top N motion candidates in the candidate list are used as the base candidates for MMVD and affine MMVD before being reordered. For MMVD, N is equal to 3, and for affine MMVD, N is [1, 3] depending on the neighboring block affine flag. Two ways of adding MMVD offsets are allowed, including ‘both sides’ and ‘one side’, depending on whether the offset of the other reference picture list is mirrored or directly set to zero. Which way is applied to a block depends on the TM cost.

[0270] OBMC When OBMC is applied, the top and left boundary pixels of the CU are refined with weighted prediction using the motion information of the neighboring blocks as described in JVET-L0101.

[0271] The condition for not applying OBMC is as follows: When OBMC is disabled at SPS level When the current block has Intra mode or IBC mode When the current block applies LIC When the current luma block area is smaller than or equal to 32 Subblock boundary OBMC is performed by applying the same blending to the top subblock boundary pixels, left subblock boundary pixels, bottom subblock boundary pixels and right subblock boundary pixels using the motion information of the neighboring subblocks. It is enabled for the following subblock-based coding tools: Affine AMVP mode; Affine merge mode and subblock-based temporal motion vector prediction (SbTMVP); Subblock-based bilateral matching.

[0272] When OBMC mode is used in CIIP mode with LMCS, inter-frame blending is performed before LMCS mapping of inter-frame samples. LMCS is applied to the blended inter-frame samples which are combined with intra-frame samples to which LMCS has been applied in CIIP mode,

[0273]

[0274] wherein, represents a sample predicted by motion prediction of the current block in the original domain, represents a sample predicted in the mapping domain, represents a sample predicted by motion prediction of a neighboring block in the original domain, and and are weights.

[0275] Template matching based OBMC In the template matching based OBMC scheme, instead of directly using weighted prediction, the prediction value of the CU boundary sample derivation method is determined according to the template matching cost, including using only the motion information of the current block, or using the motion information of the neighboring block and using one of the blending modes.

[0276] In this scheme, for each block of size 4 x 4 at the top CU boundary, the top template size is equal to 4 x 1. If N the neighboring blocks have the same motion information, the top template size is enlarged to 4 N x 1 because the MC operation can be processed at one time. For each left block of size 4 x 4 at the left CU boundary, the left template size is equal to 1 x 4 or 1 x 4 N Figure 20 .

[0277] For each 4 x 4 top block (or N group of 4 x 4 blocks), the prediction value of the boundary sample is derived according to the following steps.

[0278] ​For example, take block A as the current block and take its above neighbor block AboveNeighbor_A. The operation of the left block is done in the same way.

[0279] First, three template matching costs (Costl, Cost2, Cost3) are measured by SAD between the template reconstructed samples from MC process and their corresponding reference samples according to the following three categories of motion information: Costl is calculated according to the motion information of A.

[0280] Cost2 is calculated according to the motion information of AboveNeighbor_A.

[0281] Cost3 is calculated according to the weighted prediction of the motion information of A and AboveNeighbor_A, with the weighting factors being 3 4 and 1 4, respectively.

[0282] Second, by comparing Costl, Cost2 and Cost3, select one method to calculate the final prediction result of the boundary samples.

[0283] The original MC result using the current block ring motion information is denoted as Pixell, and the MC result using the motion information of the neighboring block is denoted as Pixel2. The final prediction result is denoted as NewPixel.

[0284] If Costl is the smallest, then NewPixel(i,j) = Pixell(i,j).

[0285] If (Cost2 + (Cost2 » 2) + (Cost2 » 3)) <= Costl, then use the mixed mode 1.

[0286] For luma blocks, the number of mixed pixel rows is 4.

[0287] NewPixel(i,0) = (26*Pixell(i,0) + 6*Pixel2(i,0) + 16) 5 NewPixel(i,1) = (7*Pixell(i,1) + Pixel2(i,1) + 4) 3 NewPixel(i,2) = (15*Pixell(i,2) + Pixel2(i,2) + 8) 4 NewPixel(i,3) = (31*Pixell(i,3) + Pixel2(i,3) + 16) 5 For chroma blocks, the number of mixed pixel rows is 1.

[0288] NewPixel(i, 0) = (26 x Pixel1(i, 0) + 6 x Pixel2(i, 0) + 16) 5 If Cost1<= Cost2, then use blend mode 2.

[0289] For luma blocks, the number of mixed pixel rows is 2.

[0290] NewPixel(i, 1) = (31 x Pixel1(i, 1) + Pixel2(i, 1) + 16) 4 NewPixel(i, 1) = (31 x Pixel1(i, 1) + Pixel2(i, 1) + 16) 5 For chroma blocks, the number of mixed pixel rows / columns is 1.

[0291] NewPixel(i, 0) = (15 x Pixel1(i, 0) + Pixel2(i, 0) + 8) 4 Otherwise, use blend mode 3.

[0292] For luma blocks, the number of mixed pixel rows is 4.

[0293] NewPixel(i, 1) = (7 x Pixel1(i, 1) + Pixel2(i, 1) + 4) 3 NewPixel(i, 2) = (15 x Pixel1(i, 2) + Pixel2(i, 2) + 8) 4 NewPixel(i, 3) = (31 x Pixel1(i, 3) + Pixel2(i, 3) + 16) 5 For chroma blocks, the number of mixed pixel rows is 1.

[0294] NewPixel(i, 0) = (7 x Pixel1(i, 0) + Pixel2(i, 0) + 4) 3 Currently, the IBC tool is not combined with the GPM tool. Therefore, the present disclosure provides examples of combining them together, which can improve prediction accuracy and improve coding performance.

[0295] Currently, the coding blocks coded with IBC mode are not combined with the coding blocks coded with intra or inter mode. Therefore, the disclosure provides examples of combining them together, which can improve prediction accuracy and improve coding performance.

[0296] Currently, the weights of the intra coded blocks and the inter coded blocks in CIIP are predefined in a fixed way. Therefore, the disclosure provides examples of adaptively deciding the weights based on template matching method, which can improve prediction accuracy and improve coding performance.

[0297] Currently, the number of block vectors (BVs) in IBC tool is odd. Therefore, the disclosure provides examples for increasing the number of block vectors (BVs), and the prediction results can be combined, which can improve prediction accuracy and improve coding performance.

[0298] Currently, the coding blocks coded with intra TMP mode are not combined with the coding blocks coded with intra or inter mode. Therefore, the disclosure provides examples of combining them together, which can improve prediction accuracy and improve coding performance.

[0299] Currently, the intra TMP tool is not combined with the GPM tool. Therefore, the disclosure provides examples of combining them together, which can improve prediction accuracy and improve coding performance.

[0300] Currently, the IBC tool is not combined with the TIMD tool. Therefore, the disclosure provides examples of combining them together, which can improve prediction accuracy and improve coding performance.

[0301] Currently, the intra TMP tool is not combined with the TIMD tool. Therefore, the disclosure provides examples of combining them together, which can improve prediction accuracy and improve coding performance.

[0302] Currently, the intra TMP tool is not combined with the LIC tool. Therefore, the disclosure provides examples of combining them together, which can improve prediction accuracy and improve coding performance.

[0303] Currently, the IBC tool is not combined with the OBMC tool. Therefore, the disclosure provides examples of combining them together, which can improve prediction accuracy and improve coding performance.

[0304] Currently, the intra TMP tool is not combined with the OBMC tool. Therefore, the disclosure provides examples of combining them together, which can improve prediction accuracy and improve coding performance.

[0305] Currently, the candidate derivation process of IBC merge mode and IBC AMVP mode only uses neighboring neighboring blocks in the top-left region and side non-neighboring blocks, therefore further extending more non-neighboring blocks is provided in the present disclosure, which can improve prediction accuracy and improve coding performance.

[0306] Currently, IBC mode generally uses BV on block level for motion compensation, therefore further introducing sub-block based IBC mode is provided in the present disclosure, which can improve prediction accuracy and improve coding performance.

[0307] Currently, TM IBC mode and TM regular inter mode use both left and top templates for motion refinement, therefore further extending template mode is provided in the present disclosure, which can improve prediction accuracy and improve coding performance.

[0308] Currently, when obtaining boundary strength, the deblocking filter treats blocks coded with IBC mode and blocks coded with intra TM P mode differently. Therefore, unifying the two modes is provided in the present disclosure, which can improve coding performance.

[0309] Currently, intra TM P tool is not combined with IBC tool, therefore combining the two tools in GPM or CIIP manner is provided in the present disclosure, which can improve coding performance.

[0310] Currently, MMVD candidate of GPM utilizing MMVD mode is signaled similar to MMVD design in VVC, therefore reordering MMVD candidate of GPM utilizing MMVD mode based on template matching method is provided in the present disclosure, which can improve coding performance.

[0311] Currently, IBC / regular inter / affine inter HMVP candidate is utilized only based on the coding order of HMVP candidate, therefore utilizing HMVP candidate based on the relative position of HMVP candidate to the current coding block is provided in the present disclosure, which can improve coding performance.

[0312] Currently, the intra prediction mode pointed by the block vector of IBC or intra TM P is saved in memory, therefore the saved intra prediction mode is directly used for intra MPM list construction.

[0313] In the present disclosure, in order to solve the problems pointed out above, methods for further improving the existing design of IBC are provided. Generally, the main features of the technology proposed in the present disclosure are summarized as follows.

[0314] IBC tool is combined with GPM tool, the combined form can be GPM utilizing IBC and IBC prediction, GPM utilizing IBC and intra prediction or GPM utilizing IBC and inter prediction.

[0315] As a simplified version of the combination of IBC tool with GPM tool, for predefined directions (e.g. 45 degrees), the top-left part is predicted with intra mode, the bottom-right part is predicted with IBC mode, and then they are averaged with weighted to obtain the final prediction signal.

[0316] IBC tool combined with CIIP tool, where IBC prediction is combined with intra prediction mode or IBC prediction is combined with inter prediction mode.

[0317] Adaptively decide the weights of intra-coded blocks and inter-coded blocks in CIIP based on template matching method.

[0318] IBC tool combined with MHP tool, where more than one BV prediction is obtained and they are averaged with weighted to obtain the final prediction signal.

[0319] Intra TMP tool combined with CIIP tool, where intra TMP is combined with intra prediction mode or intra TMP is combined with inter prediction mode.

[0320] Intra TMP tool combined with GPM tool, which can be GPM with intra TMP and intra TMP prediction, GPM with intra TMP and intra prediction or GPM with intra TMP and inter prediction.

[0321] As a simplified version of the combination of intra TMP tool with GPM tool, for predefined directions (e.g. 45 degrees), the top-left part is predicted with intra mode, the bottom-right part is predicted with intra TMP mode, and then they are averaged with weighted to obtain the final prediction signal.

[0322] IBC tool combined with TIMD tool, where IBC mode is used together with intra prediction modes in MPM to do TIMD fusion.

[0323] Intra TMP tool combined with TIMD tool, where intra TMP mode is used together with intra prediction modes in MPM to do TIMD fusion.

[0324] Intra TMP tool combined with LIC tool, where local illumination changes between current block and its intra TMP predicted block are compensated with LIC tool.

[0325] IBC tool combined with OBMC tool, where top and left boundary pixels of current block predicted with IBC are refined with OBMC tool.

[0326] Intra-TMP tool is combined with OBMC tool, where the top and left boundary pixels of the current block predicted with Intra-TMP are refined with OBMC tool.

[0327] The candidate derivation process of IBC merge mode or IBC AMVP mode is extended by using not only neighboring neighboring blocks but also non-neighboring neighboring blocks.

[0328] IBC mode is extended to subblock level, where a subblock in the current block has its own BV for motion compensation.

[0329] The template mode of TM IBC mode and TM regular inter mode is extended, where only left template, only top template, etc. are used for motion refinement.

[0330] When obtaining boundary strength, the deblocking filter treats the blocks coded with IBC mode and the blocks coded with Intra-TMP mode equally.

[0331] IBC tool is combined with Intra-TMP tool, where the combined form can be GPM with IBC and Intra-TMP prediction, CIIP with IBC and Intra-TMP prediction.

[0332] MMVD candidates of GPM with MMVD mode are reordered based on template matching method.

[0333] IBC / regular inter / affine inter HMVP candidates are utilized based on the relative position of the HMVP candidate to the current coding block.

[0334] During the construction of the Intra MPM list of the current block, when one neighboring block is IBC coded or Intra-TMP coded, the Intra prediction mode pointed by the block vector of IBC or Intra-TMP is used to construct the Intra MPM list of the current block.

[0335] In some examples, the disclosed methods can be applied independently or jointly.

[0336] GPM with IBC and IBC prediction According to one or more embodiments of the present disclosure, IBC tool is combined with GPM tool in the form of GPM with IBC and IBC prediction. Different methods can be used to achieve this goal.

[0337] In the first method, both of the two “inter” parts of GPM with inter and inter prediction method in VVC are replaced by IBC. This means that the two IBC merge prediction results are weighted average with each other according to the partition line in the coding block. The weights can be obtained by referring to GPM with inter and inter prediction method in VVC.

[0338] In the second method, both of the two "inter" parts of the GPM in the ECM that utilize inter and inter prediction methods are replaced by IBC, where some template matching tools can be utilized to further improve the coding performance.

[0339] Source of IBC prediction results When the IBC tool is combined with the GPM tool in the form of GPM with IBC and IBC prediction, the IBC prediction results can come from regular merge candidates, TM refined merge candidates or merge candidates with block vector difference (MBVD).

[0340] In some examples, regular merge candidates, TM refined merge candidates or MBVD candidates can exist in the ECM. In the first method, both of the two IBC prediction results come from the same kind of merge candidates. For example, both of the two IBC prediction results come from regular merge candidates, or both come from TM refined merge candidates, or both come from MBVD candidates, where the merge indices of the two IBC prediction results are different. When both of the two IBC prediction results come from regular merge candidates or both come from TM refined merge candidates, one flag is signaled in the bitstream to indicate whether regular merge candidates or TM refined merge candidates are utilized. This flag can be signaled in different ways. In the first method, a TM merge flag is signaled in the bitstream in the ECM to indicate whether the current block is coded in IBC TM merge mode, which can be reused for GPM with IBC and IBC prediction. Specifically, after the TM merge flag is signaled, a flag is further signaled to indicate whether the current block is coded in IBC-GPM. Then, if the TM merge flag is true, both of the two IBC prediction results come from TM refined merge candidates; if the TM merge flag is false, both of the two IBC prediction results come from regular merge candidates. In this method, if the flag to indicate whether the current block is coded in IBC-GPM is true, the merge indices of the two IBC merge candidates are signaled in the IBC-GPM syntax structure, which means that the merge indices of the two IBC merge candidates do not need to be signaled again elsewhere. In the second method, after the TM merge flag is signaled in the bitstream to indicate whether the current block is coded in IBC TM merge mode, the flag to indicate whether the current block is coded in IBC-GPM is further signaled only when the TM merge flag is false. If the flag to indicate whether the current block is coded in IBC-GPM is true, another flag is further signaled to indicate whether both of the two IBC prediction results come from TM refined merge candidates. If this flag is true, it indicates that both of the two IBC prediction results come from TM refined merge candidates; if this flag is false, it indicates that both of the two IBC prediction results come from regular merge candidates.

[0341] In the second method, the two IBC prediction results come from different kinds of merge candidates. For example, one IBC prediction result comes from a regular merge candidate and the other IBC prediction result comes from a TM refined merge candidate. In this example, a flag is signaled in the bitstream to indicate whether the IBC prediction result of the first GPM partition comes from a regular merge candidate. If the flag is true, the IBC prediction result of the first GPM partition comes from a regular merge candidate and the IBC prediction result of the second GPM partition comes from a TM refined merge candidate. If the flag is false, the IBC prediction result of the first GPM partition comes from a TM refined merge candidate and the IBC prediction result of the second GPM partition comes from a regular merge candidate. In this example, the merge indices of the two IBC prediction results can be the same or different.

[0342] In the third method, the two IBC prediction results can come from the same kind of merge candidate or from different kinds of merge candidates. For example, one IBC prediction result can come from a regular merge candidate or a TM refined merge candidate and the other IBC prediction result can also come from a regular merge candidate or a TM refined merge candidate. In this example, for the first GPM partition, a flag is first signaled to indicate whether the IBC prediction result of this GPM partition comes from a regular merge candidate. For the second GPM partition, a flag is also signaled to indicate whether the IBC prediction result of this GPM partition comes from a regular merge candidate. If both flags are true or both flags are false, this indicates that both IBC prediction results come from regular merge candidates or both IBC prediction results come from TM refined merge candidates, then the merge indices of the two IBC prediction results are different. If the first flag is true and the second flag is false or the first flag is false and the second flag is true, this indicates that the IBC prediction result of the first GPM partition comes from a regular merge candidate and the IBC prediction result of the second GPM partition comes from a TM refined merge candidate, or the IBC prediction result of the first GPM partition comes from a TM refined merge candidate and the IBC prediction result of the second GPM partition comes from a regular merge candidate, then the merge indices of the two IBC prediction results can be the same or different.

[0343] Interaction between IBC and GPM with IBC prediction and RR-IBC For the interaction between the GPM tool with IBC and IBC prediction and the reordering-IBC (RR-IBC) tool, different approaches can be utilized. In the first approach, both IBC prediction results cannot come from merge candidates coded with RR-IBC mode. To implement this, different approaches can be utilized. In the first example, scan the IBC merge candidate list from the beginning to the end, if one merge candidate is coded with RR-IBC mode, move it to the end of the IBC merge candidate list, and the following merge candidate is moved forward accordingly. Terminate the scan after reordering all merge candidates or reaching the maximum allowable number of merge candidates that are not coded with RR-IBC mode. Then, utilize the reordered merge candidate list to generate the IBC prediction results for the GPM with IBC and IBC prediction. In the second example, the IBC merge candidate list is unchanged when generating the IBC prediction results for the GPM with IBC and IBC prediction, and only the merge candidates coded with RR-IBC mode are skipped. In the second approach, both IBC prediction results can come from merge candidates coded with RR-IBC mode or merge candidates not coded with RR-IBC mode, there is no restriction when generating the IBC prediction results for the GPM with IBC and IBC prediction.

[0344] GPM with IBC and IBC prediction using TM When TM refined merge candidates are used for the GPM with IBC and IBC prediction, the TM refined merge candidates can be directly reused for the GPM with IBC and IBC prediction, or similar to the GPM with TM, different templates can be utilized for different parts of the GPM partition for a predefined GPM partition mode. For example, for one GPM partition, use the left neighboring sample, the above neighboring sample, or the left neighboring sample and the above neighboring sample to construct the template according to the GPM partition angle as shown in Table 7. Then, refine the block vector by minimizing the difference between the current template and the template in the reference region using the same search pattern of the TM refined merge mode with the semi-pixel interpolation filter disabled. In some embodiments, the provided methods presented in this section can be freely combined with the methods presented in the section of “Source of IBC prediction results”. For example, both IBC prediction results come from regular merge candidates or both come from TM refined merge candidates, when both IBC prediction results come from TM refined merge candidates, the TM refined merge candidates can be directly reused, or different templates can be utilized for different parts of the GPM partition for a predefined GPM partition mode.

[0345] GPM with IBC and IBC prediction using MBVD When merge candidates with block vector difference (MBVD) are used for GPM with IBC and GPM with IBC prediction, the reordered IBC MBVD candidates can be directly reused for GPM with IBC and GPM with IBC prediction, or similar to GPM with MMVD mode, a distance index and a direction index are used to signal the selected MBVD candidate, or different templates can be utilized to reorder the MBVD candidates for different parts of a GPM partition of a predefined GPM partition mode. When the MBVD candidates for different parts of a GPM partition are reordered, an index is signaled to represent the selected MBVD candidate.

[0346] When merge candidates with block vector difference (MBVD) are used for GPM with IBC and GPM with IBC prediction, different methods can be utilized to select the IBC prediction results.

[0347] In the first method, both IBC prediction results are from merge candidates refined with block vector difference (MBVD) or both are from regular merge candidates. In this method, a flag is signaled in the bitstream to represent whether to apply MBVD for GPM with IBC and GPM with IBC prediction. If the flag is true, both IBC prediction results are from regular merge candidates refined with MBVD. If the flag is false, both IBC prediction results are from regular merge candidates.

[0348] In the second method, one IBC prediction result is from merge candidates refined with block vector difference (MBVD) and the other IBC prediction result is from regular merge candidates. In this example, a flag is signaled in the bitstream to represent whether the IBC prediction result for the first GPM partition is from regular merge candidates. If the flag is true, the IBC prediction result for the first GPM partition is from regular merge candidates and the IBC prediction result for the second GPM partition is from regular merge candidates refined with block vector difference. If the flag is false, the IBC prediction result for the first GPM partition is from regular merge candidates refined with block vector difference and the IBC prediction result for the second GPM partition is from regular merge candidates.

[0349] In the third method, one IBC prediction can come from a regular merge candidate or a regular merge candidate refined with a block vector difference, and the other IBC prediction can also come from a regular merge candidate or a regular merge candidate refined with a block vector difference. In this example, for the first GPM partition, a flag is first signaled to indicate whether the IBC prediction for this GPM partition comes from a regular merge candidate. For the second GPM partition, a flag is also signaled to indicate whether the IBC prediction for this GPM partition comes from a regular merge candidate. If both flags are true or both are false, it indicates that both IBC predictions come from regular merge candidates or both come from regular merge candidates refined with a block vector difference. If the first flag is true and the second flag is false or the first flag is false and the second flag is true, it indicates that the IBC prediction for the first GPM partition comes from a regular merge candidate and the IBC prediction for the second GPM partition comes from a regular merge candidate refined with a block vector difference, or the IBC prediction for the first GPM partition comes from a regular merge candidate refined with a block vector difference and the IBC prediction for the second GPM partition comes from a regular merge candidate.

[0350] In the above three methods, when both IBC predictions come from regular merge candidates, the merge indices for the two IBC predictions are different; when one IBC prediction comes from a merge candidate refined with a block vector difference (MBVD) and the other IBC prediction comes from a regular merge candidate, the merge indices for the two IBC predictions can be the same or different, or must be different. When both IBC predictions come from regular merge candidates refined with a block vector difference, when the MV refinements for the two partitions are different, the merge indices for the two IBC predictions are allowed to be the same, and when the two MV refinements are the same, the merge indices for the two IBC predictions are not allowed to be the same.

[0351] When the tool 'GPM with IBC and IBC prediction using MBVD' is combined with the tool 'GPM with IBC and IBC prediction using TM', different methods can be utilized.

[0352] In the first method, the tool 'GPM with IBC and IBC prediction using MBVD' is applied after the tool 'GPM with IBC and IBC prediction using TM'. In this method, the tool 'GPM with IBC and IBC prediction using MBVD' can only be applied when both IBC prediction results are from regular merge candidates. For example, in the first step, both IBC prediction results are allowed to be from regular merge candidates or from TM refined merge candidates. A flag is signaled to indicate whether both IBC prediction results are from regular merge candidates. If the flag is false, which indicates both IBC prediction results are from TM refined merge candidates, the process is terminated. If the flag is true, the second step can be applied. In the second step, one IBC prediction result is allowed to be from regular merge candidate or regular merge candidate refined with block vector difference, and the other IBC prediction result is also allowed to be from regular merge candidate or regular merge candidate refined with block vector difference. For the first GPM partition, a flag is first signaled to indicate whether the IBC prediction result of the GPM partition is from regular merge candidate. For the second GPM partition, a flag is also signaled to indicate whether the IBC prediction result of the GPM partition is from regular merge candidate.

[0353] In the second method, the tool 'GPM with IBC and IBC prediction using MBVD' is applied before the tool 'GPM with IBC and IBC prediction using TM'. In this method, the tool 'GPM with IBC and IBC prediction using TM' can only be applied when both IBC prediction results are from regular merge candidates. For example, in the first step, one IBC prediction result is allowed to be from regular merge candidate or regular merge candidate refined with block vector difference, and the other IBC prediction result is also allowed to be from regular merge candidate or regular merge candidate refined with block vector difference. For the first GPM partition, a flag is first signaled to indicate whether the IBC prediction result of the GPM partition is from regular merge candidate. For the second GPM partition, a flag is also signaled to indicate whether the IBC prediction result of the GPM partition is from regular merge candidate. If at least one flag is false, which indicates at least one IBC prediction result is from regular merge candidate refined with block vector difference, the process is terminated. If both flags are true, the second step can be applied. In the second step, both IBC prediction results are allowed to be from regular merge candidates or from TM refined merge candidates. A flag is signaled to indicate whether both IBC prediction results are from regular merge candidates.

[0354] Split modes for GPM with IBC and IBC prediction When the IBC tool is combined with the GPM tool in the form of IBC and GPM predicted by IBC, different methods can be utilized to encode the GPM partition mode.

[0355] In the first method, similar to GPM in VVC, all allowed partition modes are encoded with equal probability.

[0356] In the second method, all allowed GPM partition modes are divided into several groups, and two indices are encoded to identify the transmitted GPM partition mode, wherein the first index is used to represent which group is utilized, and the second index is used to represent the specific index in the selected group. For example, all allowed GPM partition modes are divided into two groups, wherein the GPM partition modes along the horizontal or vertical direction constitute one group, and the GPM partition modes along the other direction constitute another group. First, the flag of context encoding or bypass encoding is encoded to represent which group the transmitted GPM partition mode belongs to, and then the index encoded with equal probability is encoded to represent which index in the selected group the transmitted GPM partition mode belongs to. In another example, similar to the IBC tool presented in the background art, all allowed GPM partition modes are divided into two groups, the first group is composed of Table 3, and the second group is composed of Table 4. First, the flag of context encoding is encoded to represent which group the transmitted GPM partition mode belongs to, and then the index encoded with equal probability is encoded to represent which index in the selected group the transmitted GPM partition mode belongs to.

[0357] In a third method, GPM partition modes are encoded using TM-based method. In one example, similar to GPM in ECM, all allowed GPM partition modes are reordered using TM-based method, and then an index is signaled using Golomb-Rice code to indicate where the exact GPM partition mode is located in the reordered list. In some examples, the reordering can be based on comparing template matching cost. In another example, after all allowed GPM partition modes are divided into several groups, GPM partition modes in selected groups are reordered using TM-based method, and then an index is signaled using Golomb-Rice code to indicate where the exact GPM partition mode is located in the reordered list. For example, similar to the "IBC with geometric partitioning" tool presented in the background section, all allowed GPM partition modes are divided into two groups, the first group consists of Table 3, and the second group consists of Table 4, a context-coded flag is first encoded to represent which group the transmitted GPM partition mode belongs to, then GPM partition modes in selected group are reordered using TM-based method, and finally an index is signaled using Golomb-Rice code to indicate where the exact GPM partition mode is located in the reordered selected group.

[0358] In a fourth method, since both GPM partition mode and two merge indices of two GPM partition sub-regions need to be transmitted in the bitstream, similar to spatial GPM, all combinations of GPM partition mode and two merge indices of two GPM partition sub-regions are reordered using TM-based method, and then an index is signaled using Golomb-Rice code to indicate where the exact combination of GPM partition mode and two merge indices of two GPM partition sub-regions is located in the reordered list.

[0359] Hybrid method of GPM with IBC and IBC prediction When IBC tool is combined with GPM tool in the form of GPM with IBC and IBC prediction, different methods can be used to mix two GPM sub-regions.

[0360] In a first method, adaptive mixing is used. For a CU coded with GPM with IBC and IBC prediction, different mixing widths (e.g., 5 different mixing widths as shown in Figure 36 ) are compared in the RDO process, and an index representing the selected mixing width is transmitted in the bitstream.

[0361] In the second method, the blending width can be selected via predefined criteria, and then no index needs to be transmitted in the bitstream. In one example, similar to the spatial GPM, the blending width is selected according to the width and height of the current CU. In another example, similar to the GPM in VVC, only one blending width equal to τ (as shown in Figure 36 ) is used for all CU sizes. In a third example, only one blending width equal to 1 / 4 τ (as shown in Figure 36 ) is used for all CU sizes.

[0362] In the third method, different blending methods are utilized for different kinds of content. For example, for screen content, hard blending with blending width equal to 1 / 4 τ is utilized, and no index needs to be transmitted in the bitstream; for natural content, adaptive blending as presented in the first method is utilized, and an index representing the selected blending width is transmitted in the bitstream.

[0363] In the fourth method, different blending methods are utilized for different slice types. For example, for I slice, adaptive blending as presented in the first method is utilized; for P slice or B slice, hard blending with blending width equal to 1 / 4 τ (as shown in Figure 36 ) is utilized. It should be noted that when adaptive blending is utilized only for I slice, only for I slice an index representing the selected blending width is transmitted in the bitstream; for P slice or B slice, no index representing the selected blending width needs to be transmitted in the bitstream.

[0364] Motion covering for GPM with IBC and IBC prediction When combining the IBC tool with the GPM tool in the form of GPM with IBC and IBC prediction, different methods can be used to cover the motion information of the CUs coded with GPM with IBC and IBC prediction. In some examples, covering the motion information of a sub-block refers to filling the motion information of the sub-block according to the center position of the sub-block in the current CU.

[0365] In the first method, as shown in Figure 43 , if the center position of a sub-block (e.g., 4 × 4) in the current CU is located in GPM partition A, the block vector of the IBC merge candidate of GPM partition A is used to fill the sub-block; if the center position of a sub-block (e.g., 4 × 4) in the current CU is located in GPM partition B, the block vector of the IBC merge candidate of GPM partition B is used to fill the sub-block. In some embodiments, the partitions A and B in this method are only divided by the GPM partition line, which means that both partitions A and B can contain a certain blending region.

[0366] In the second method, as shown in Figure 43 As shown, if the center of a sub-block (e.g., 4 × 4) in the current CU is located in a non-mixed region of GPM partition A, the sub-block is filled with the block vector of the IBC merge candidate of GPM partition A. If the center of a sub-block (e.g., 4 × 4) in the current CU is located in a non-mixed region of GPM partition B, the sub-block is filled with the block vector of the IBC merge candidate of GPM partition B. If the center of a sub-block (e.g., 4 × 4) in the current CU is located in a mixed region of a GPM partition, the sub-block is filled with the weighted average of the block vectors of the IBC merge candidates of the two GPM partitions. For example, if the center of a sub-block (e.g., 4 × 4) in the current CU is located in a mixed region of a GPM partition, the sub-block is filled with the equal average block vector of the IBC merge candidates of the two GPM partitions.

[0367] In the third method, such as Figure 43 As shown, regardless of where the GPM partition line is located, the motion information of the current CU is filled entirely with the block vectors of the IBC merge candidates of GPM partition A or GPM partition B.

[0368] Intra-frame prediction mode that utilizes IBC and IBC-predicted GPM When the IBC tool is combined with the GPM tool in the form of IBC and GPM using IBC prediction, different methods can be used to overwrite the intra-prediction mode information of the CU encoded using IBC and GPM using IBC prediction. In some examples, overwriting the intra-prediction mode information of a sub-block means filling the intra-prediction mode information of the sub-block according to the center position of the sub-block in the current CU.

[0369] In the first method, such as Figure 43 As shown, if the center of a sub-block (e.g., 4 × 4) in the current CU is located in GPM segmentation partition A, the sub-block is filled with the intra-prediction mode pointed to by the block vector of the IBC merging candidate of GPM segmentation partition A; if the center of a sub-block (e.g., 4 × 4) in the current CU is located in GPM segmentation partition B, the sub-block is filled with the intra-prediction mode pointed to by the block vector of the IBC merging candidate of GPM segmentation partition B. In some embodiments, partitions A and B in this method are divided only by GPM segmentation lines, which means that both partitions A and B may contain a certain mixed region.

[0370] In the second method, such as Figure 43As shown, if the center position of a sub-block (e.g., 4 x 4) in the current CU is located in the non-mixed region of GPM partition A, the sub-block is filled with the intra prediction mode pointed by the block vector of the IBC merge candidate of GPM partition A; if the center position of a sub-block (e.g., 4 x 4) in the current CU is located in the non-mixed region of GPM partition B, the sub-block is filled with the intra prediction mode pointed by the block vector of the IBC merge candidate of GPM partition B. If the center position of a sub-block (e.g., 4 x 4) in the current CU is located in the mixed region of GPM partition, the sub-block is filled with the intra prediction mode pointed by the weighted average of the block vectors of the IBC merge candidates of the two GPM partitions, or the sub-block is filled with the weighted average of the intra prediction modes pointed by the block vectors of the IBC merge candidates of the two GPM partitions. For example, if the center position of a sub-block (e.g., 4 x 4) in the current CU is located in the mixed region of GPM partition, the sub-block is filled with the intra prediction mode pointed by the equally averaged block vector of the IBC merge candidates of the two GPM partitions. In another example, if the center position of a sub-block (e.g., 4 x 4) in the current CU is located in the mixed region of GPM partition, the sub-block is filled with the equally averaged intra prediction mode pointed by the block vectors of the IBC merge candidates of the two GPM partitions.

[0371] In the third method, as shown, Figure 43 regardless of where the GPM partition line is located, the intra prediction mode of the current CU is only filled with the intra prediction mode pointed by the block vector of the IBC merge candidate of GPM partition A or GPM partition B in total.

[0372] Application channels of GPM with IBC and IBC prediction When the IBC tool is combined with the GPM tool in the form of GPM with IBC and IBC prediction, different methods can be used to apply the GPM with IBC and IBC prediction to different channels.

[0373] In the first method, the GPM with IBC and IBC prediction can be applied to different channels based on the slice type. For example, for an I slice, the GPM with IBC and IBC prediction is only applied to the luminance channel of the current CU; for a P slice or a B slice, the GPM with IBC and IBC prediction is applied to both the luminance channel and the chrominance channel of the current CU. In another example, for an I slice, a P slice or a B slice, the GPM with IBC and IBC prediction is only applied to the luminance channel of the current CU. In a third example, for an I slice, a P slice or a B slice, the GPM with IBC and IBC prediction is applied to both the luminance channel and the chrominance channel of the current CU.

[0374] Mode setting for GPM with IBC and IBC-predicted GPM When the IBC tool is combined with the GPM tool in the form of GPM with IBC and IBC-predicted GPM, different methods can be used to set the mode of the CU coded with GPM with IBC and IBC-predicted GPM.

[0375] In a first method, as shown in FIG. 1 1 A, the merge index of the CU coded with GPM with IBC and IBC-predicted GPM is set to the merge index of the GPM partition A, and the corresponding modes (e.g. IBC-LIC flag, RR-IBC type, etc.) of the CU coded with GPM with IBC and IBC-predicted GPM are set to the corresponding modes of the GPM partition A. Figure 43 In a second method, as shown in FIG. 1 1 B, the merge index of the CU coded with GPM with IBC and IBC-predicted GPM is set to the merge index of the GPM partition B, and the corresponding modes (e.g. IBC-LIC flag, RR-IBC type, etc.) of the CU coded with GPM with IBC and IBC-predicted GPM are set to the corresponding modes of the GPM partition B. Figure 43 In a third method, the mode of the CU coded with GPM with IBC and IBC-predicted GPM can be set to default values. For example, the IBC-LIC flag of the CU coded with GPM with IBC and IBC-predicted GPM is set to false, and the RR-IBC type of the CU coded with GPM with IBC and IBC-predicted GPM is set to 0.

[0376] GPM with IBC and intra-predicted GPM According to one or more embodiments of the present disclosure, the IBC tool is combined with the GPM tool in the form of GPM with IBC and intra-predicted GPM. Different methods can be used to achieve this goal.

[0377] In a first method, the "inter" part of the GPM with inter and intra prediction methods in the ECM is replaced by IBC, where the IBC merge prediction result is weighted averaged with the intra prediction result to obtain the final prediction signal.

[0378] Source of IBC prediction result When the IBC tool is combined with the GPM tool in the form of GPM with IBC and intra-predicted GPM, the IBC prediction result can come from a regular merge candidate, a TM refined merge candidate, or a merge candidate with block vector difference (MBVD).

[0379] When TM refinement merge candidate is used for GPM with IBC and intra prediction, the TM refinement merge candidate can be directly reused to derive IBC predictor of GPM with IBC and intra prediction, or similarly as GPM with TM, different templates can be utilized for IBC prediction GPM partition for pre-defined GPM partition mode. For example, for IBC prediction GPM partition, the template is constructed using left neighboring sample, top neighboring sample, or both left neighboring sample and top neighboring sample according to GPM partition angle as shown in Table 7. When using Table 7 to derive template mode, if the first GPM partition is predicted by IBC, the template mode of the first GPM partition is reused for TM refinement merge candidate; if the second GPM partition is predicted by IBC, the template mode of the second GPM partition is reused for TM refinement merge candidate. Then the block vector is refined by minimizing the difference between the current template and the template in the reference region using the same search mode of TM refinement merge mode with the semi-pixel interpolation filter disabled.

[0380] When merge candidate with block vector difference (MBVD) is used for GPM with IBC and intra prediction, reordered IBC MBVD candidate can be directly reused for GPM with IBC and intra prediction, or similarly as GPM with MMVD mode, the selected MBVD candidate is represented using one distance index and one direction index, or can be reordered for IBC prediction GPM partition of pre-defined GPM partition mode using different templates. When MBVD candidate of IBC prediction GPM partition is reordered, the selected MBVD candidate is represented using a signaled index.

[0381] When tool ‘GPM with IBC and intra prediction using MBVD’ is combined with tool ‘GPM with IBC and intra prediction using TM’, different approaches can be utilized.

[0382] In the first method, the tool 'GPM with IBC and intra prediction using MBVD' is applied after the tool 'GPM with IBC and intra prediction using TM' in the tool. In this method, the tool 'GPM with IBC and intra prediction using MBVD' can only be applied when the IBC prediction result is from a regular merge candidate. For example, in the first step, the IBC prediction result is only allowed to be from a regular merge candidate or from a TM refined merge candidate. A flag is first signaled to indicate whether the IBC prediction result is from a regular merge candidate. If the flag is false, which indicates that the IBC prediction result is from a TM refined merge candidate, the process is terminated. If the flag is true, a second step can be applied. In the second step, the IBC prediction result is allowed to be from a regular merge candidate or a regular merge candidate refined with a block vector difference. And a flag is signaled to indicate whether the IBC prediction result of the GPM partition is from a regular merge candidate.

[0383] In the second method, the tool 'GPM with IBC and intra prediction using MBVD' is applied before the tool 'GPM with IBC and intra prediction using TM' in the tool. In this method, the tool 'GPM with IBC and intra prediction using TM' can only be applied when the IBC prediction result is from a regular merge candidate. For example, in the first step, the IBC prediction result is allowed to be from a regular merge candidate or a regular merge candidate refined with a block vector difference. A flag is signaled to indicate whether the IBC prediction result is from a regular merge candidate. If the flag is false, which indicates that the IBC prediction result is from a regular merge candidate refined with a block vector difference, the process is terminated. If the flag is true, a second step can be applied. In the second step, the IBC prediction result is only allowed to be from a regular merge candidate or a TM refined merge candidate. A flag is signaled to indicate whether the IBC prediction result is from a regular merge candidate.

[0384] Split modes for GPM with IBC and intra prediction When the IBC tool is combined with the GPM tool in the form of GPM with IBC and intra prediction, different methods can be used to code the GPM split modes.

[0385] In the first method, similar to the GPM in VVC, all the allowed split modes are coded with equal probability.

[0386] In the second method, all allowed GPM partition modes are divided into several groups, and two indices are coded to identify the transmitted GPM partition mode, where the first index is used to indicate which group is utilized, and the second index is used to indicate the specific index in the selected group. For example, all allowed GPM partition modes are divided into two groups, where GPM partition modes along the horizontal or vertical direction form one group, and GPM partition modes along the other direction form another group. First, a context-coded or bypass-coded flag is coded to indicate which group the transmitted GPM partition mode belongs to, and then an index coded with equal probability is coded to indicate which index in the selected group the transmitted GPM partition mode belongs to.

[0387] In the third method, the GPM partition mode is coded with a TM-based method. In one example, similar to the GPM in ECM, all allowed GPM partition modes are reordered with a TM-based method, and then an index is signaled to indicate where the exact GPM partition mode in the reordered list using Golomb-Rice codes. In another example, after all allowed GPM partition modes are divided into several groups, the GPM partition modes in the selected group are reordered with a TM-based method, and then an index is signaled to indicate where the exact GPM partition mode in the reordered list using Golomb-Rice codes. For example, similar to the IBC tool with geometric partitioning presented in the background art, all allowed GPM partition modes are divided into two groups, the first group consists of Table 3, and the second group consists of Table 4, first, a context-coded flag is coded to indicate which group the transmitted GPM partition mode belongs to, then the GPM partition modes in the selected group are reordered with a TM-based method, and finally an index is signaled to indicate where the exact GPM partition mode in the reordered selected group using Golomb-Rice codes.

[0388] In the fourth method, since both the GPM partition mode and the IBC merge index and intra prediction mode index of the two GPM partition regions need to be transmitted in the bitstream, similar to the spatial GPM, all combinations of the GPM partition mode and the IBC merge index and intra prediction mode index of the two GPM partition regions are reordered with a TM-based method, and then an index is signaled to indicate where the exact combination of the GPM partition mode and the corresponding IBC merge index and intra prediction mode index of the two GPM partition regions in the reordered list using Golomb-Rice codes.

[0389] Hybrid method of GPM with IBC and intra prediction When combining the IBC tool with the GPM tool in the form of GPM with IBC and intra prediction, different approaches can be used to blend the two GPM partitions.

[0390] In a first approach, adaptive blending is utilized. For a CU coded with GPM with IBC and IBC prediction, different blending widths (e.g., 5 different blending widths as shown in Figure 36 are compared in the RDO process, and an index representing the selected blending width is transmitted in the bitstream.

[0391] In a second approach, the blending width can be selected via a pre-defined criterion, and then no index needs to be transmitted in the bitstream. In one example, similar to the spatial GPM, the blending width is selected according to the width and height of the current CU. In another example, similar to the GPM in VVC, only one blending width equal to τ (as shown in Figure 36 is used for all CU sizes. In a third example, only one blending width equal to 1 / 4 τ (as shown in Figure 36 is used for all CU sizes.

[0392] In a third approach, different blending approaches are utilized for different kinds of content. For example, for screen content, hard blending with blending width equal to 1 / 4 τ is utilized, and no index needs to be transmitted in the bitstream; for natural content, adaptive blending as presented in the first approach is utilized, and an index representing the selected blending width is transmitted in the bitstream.

[0393] In a fourth approach, different blending approaches are utilized for different slice types. For example, for I slices, adaptive blending as presented in the first approach is utilized; for P slices or B slices, hard blending with blending width equal to 1 / 4 τ (as shown in Figure 36 is utilized. It should be noted that when adaptive blending is utilized only for I slices, only for I slices an index representing the selected blending width is transmitted in the bitstream; for P slices or B slices, no index representing the selected blending width needs to be transmitted in the bitstream.

[0394] Motion and intra prediction mode override for GPM with IBC and intra prediction When combining the IBC tool with the GPM tool in the form of GPM with IBC and intra prediction, different approaches can be used to override the motion and intra prediction mode information of a CU coded with GPM with IBC and intra prediction. In some examples, overriding the motion information of a sub-block refers to filling the motion information of the sub-block according to the center position of the sub-block in the current CU. In some examples, overriding the intra prediction mode information of a sub-block refers to filling the intra prediction mode information of the sub-block according to the center position of the sub-block in the current CU.

[0395] In the first method, such as Figure 43 As shown, if the center of a sub-block (e.g., 4 × 4) in the current CU is located in GPM partition A, then if GPM partition A is predicted by IBC, the motion information of the sub-block is filled with the block vector of the IBC merge candidate of GPM partition A, and the intra-prediction mode information of the sub-block is filled with the intra-prediction mode pointed to by the block vector of the IBC merge candidate of GPM partition A; if GPM partition A is predicted by intra-prediction, the motion information of the sub-block is empty, and the intra-prediction mode information of the sub-block is filled with the intra-prediction mode of GPM partition A. If the center of a sub-block (e.g., 4 × 4) in the current CU is located within a GPM partition B, then if GPM partition B is predicted by IBC, the motion information of the sub-block is filled with the block vector of the IBC merge candidate of GPM partition B, and the intra-prediction mode information of the sub-block is filled with the intra-prediction mode pointed to by the block vector of the IBC merge candidate of GPM partition B; if GPM partition B is predicted by intra-prediction, the motion information of the sub-block is empty, and the intra-prediction mode information of the sub-block is filled with the intra-prediction mode of GPM partition B. In some embodiments, if one GPM partition is predicted by IBC, then the other GPM partition must be predicted by intra-prediction.

[0396] In the second method, such as Figure 43 As shown, the motion information of the current CU is filled entirely with the block vectors of the IBC merging candidates of the GPM segmentation partition predicted by IBC, and the intra-prediction mode information of the current CU is filled entirely with the intra-prediction mode pointed to by the block vectors of the IBC merging candidates of the GPM segmentation partition predicted by IBC, or the intra-prediction mode information of the current CU is filled entirely with the intra-prediction mode of the GPM segmentation partition predicted by the intra-frame.

[0397] Unified GPM using intra-frame and intra-frame prediction, GPM using IBC and intra-frame prediction, and GPM using IBC and IBC prediction. According to one or more embodiments of this disclosure, the codec tools utilizing intra-frame and intra-frame prediction GPM, utilizing IBC and intra-frame prediction GPM, and utilizing IBC and IBC prediction GPM are unified into a set of syntax elements.

[0398] Several syntax elements of GPM are coded into the bitstream using entropy coding. In high-level syntax (HLS), the GPM enabled flag and the maximum number of IBC GPM merge candidates and the maximum number of intra GPM mode candidates are coded in the sequence parameter set (SPS). The GPM enabled flag is coded with a fixed length code, while the maximum number of IBC GPM merge candidates is signaled relative to the maximum number of regular IBC merge candidates and coded with a zeroth order exponential-Golomb code.

[0399] At the CU level, the GPM syntax elements are signaled within the merge data syntax. Specifically, a flag indicating whether the CU is coded by GPM is first coded into the bitstream. If the flag is true, the GPM partition mode is transmitted in the bitstream. For one GPM partition, a flag indicating whether the GPM partition is intra coded is first coded into the bitstream. If the flag is true, the intra mode index is transmitted in the bitstream; otherwise, the IBC merge index is transmitted in the bitstream. For the other GPM partition, the same syntax elements are transmitted. In some embodiments, when both GPM partitions are intra coded or both are IBC coded, the intra mode indices or the IBC merge indices of the two GPM partitions are different.

[0400] When only GPM with IBC and intra prediction and GPM with IBC and IBC prediction are enabled, the syntax elements at the CU level can be simplified as follows: a flag indicating whether the CU is coded by GPM is first coded into the bitstream. If the flag is true, the GPM partition mode is transmitted in the bitstream. For the first GPM partition, a flag indicating whether the GPM partition is intra coded is first coded into the bitstream. If the flag is true, the intra mode index is transmitted in the bitstream; otherwise, the IBC merge index is transmitted in the bitstream. For the second GPM partition, if the flag indicating whether the first GPM partition is intra coded is true, the IBC merge index of the second GPM partition is transmitted in the bitstream; if the flag indicating whether the first GPM partition is intra coded is false, another flag indicating whether the second GPM partition is intra coded is coded into the bitstream. If the flag indicating whether the second GPM partition is intra coded is true, the intra mode index of the second GPM partition is transmitted in the bitstream, otherwise the IBC merge index of the second GPM partition is transmitted in the bitstream. In some embodiments, when both GPM partitions are IBC coded, the IBC merge indices of the two GPM partitions are different. To present the syntax elements clearly, the syntax element table is presented as follows:

[0401] To signal the IBC merge index, different methods can be utilized. For example, the IBC merge index is binarized using the 0th order truncated Rice code and encoded using a single context model that is initialized with the same initial value as the regular IBC merge index. In some embodiments, when both GPM split partitions are IBC coded, the used GPM merge index GPM idx0,1 is derived from the signaled syntax elements by GPM idx0 = ibc_merge_gpm_idx0 and GPM idx1 = ibc_merge_gpm_idx1 + (ibc_merge_gpm_idx1 > GPM idx0)? 1 : 0 Since the two IBC merge indices are not allowed to be the same.

[0402] GPM with IBC and inter prediction According to one or more embodiments of the present disclosure, the IBC tool is combined with the GPM tool in the form of GPM with IBC and inter prediction. Different methods can be used to achieve this goal.

[0403] In the first method, one “inter” part of GPM with inter and inter prediction method in VVC is replaced by IBC, where the IBC merge prediction result is weighted averaged with the inter merge prediction result to obtain the final prediction signal.

[0404] In the second method, one “inter” part of GPM with inter and inter prediction method in ECM is replaced by IBC, where some template matching tools can be utilized to further improve the coding performance.

[0405] IBC and intra prediction combination in the form of simplified GPM According to one or more embodiments of the present disclosure, the IBC tool is combined with the GPM tool in the form of simplified GPM with IBC and intra prediction, such as IBC and intra prediction combined in a certain split mode, which can save the bit overhead of split representation. Different methods can be used to achieve this goal.

[0406] In the first method, for one split line, such as 45 degrees, the top-left part of the coding block is encoded using an intra prediction mode, the bottom-right part of the coding block is encoded using an IBC prediction mode, and then they are averaged in the form of GPM to obtain the final prediction signal.

[0407] IBC prediction-intra / inter prediction combination According to one or more embodiments of the present disclosure, the coding block coded with IBC mode is combined with the coding block coded with intra mode or inter mode. Different methods can be used to achieve this goal.

[0408] In the first method, the decoder / encoder can combine the coding block coded with IBC mode with the coding block coded with intra mode. Various methods can be utilized in this combination. In one example, similar to the CIIP technique in VVC, the coding block coded with IBC merge mode is treated as the coding block coded with inter merge mode, and it is combined with the coding block coded with planar intra prediction mode. In another example, similar to the combination technique of CIIP with TIMD and TM merge in ECM, the coding block coded with IBC merge-TM mode is combined with the coding block coded with TIMD derived intra prediction mode.

[0409] When the coding block coded with IBC mode is combined with the coding block coded with intra mode, the weights can be designed similar to the CIIP technique in VVC and the combination technique of CIIP with TIMD and TM merge in ECM, i.e., 1) the weights of both the IBC coding block and the intra coded block are greater than zero and less than one, or the weight of the intra coded block gradually changes from one to zero from one region to another region in the current block (and vice versa for the weight of the IBC coding block); 2) the weights of the IBC coding block and the intra coded block can be decided based on the coding mode of the neighboring blocks and the intra mode of the current block; 3) the weights of the IBC coding block and the intra coded block can be uniform across the current block, or can be different in different locations of the current block.

[0410] For example, the weights of the IBC coding block and the intra coded block can be decided as follows: when the top and left neighboring blocks of the current block are both intra coded and the intra mode of the current block is planar mode, the weights of the IBC coding block and the intra coded block are 1 / 4 and 3 / 4 across the current block. When the top and left neighboring blocks of the current block are both IBC coded and the intra mode of the current block is planar mode, the weights of the IBC coding block and the intra coded block are 3 / 4 and 1 / 4 across the current block. When one of the top or left neighboring blocks is IBC coded and the other neighboring block is intra coded and the intra mode of the current block is planar mode, the weights of the IBC coding block and the intra coded block are 1 / 2 and 1 / 2 across the current block.

[0411] When the intra mode of the current block is close to the horizontal angular mode (2 <= angular mode index < 34), the current block is vertically divided as shown in Figure 16A When the intra mode of the current block is close to the vertical angular mode (34 <= angular mode index <= 66), the current block is horizontally divided as shown in Figure 16BThe blocks are horizontally divided. Table 9 shows the weights of the IBC coded blocks (wIBC) and intra-coded blocks (wIntra) for different sub-blocks. Furthermore, the weights of the IBC and intra-coded blocks can be determined in the CIIP-PDPC version. In this version, the intra-block mode is set to planar mode, and the weight of the intra-coded block gradually decreases as the combination position moves from the upper left to the lower right within the current block, and vice versa for the IBC coded block.

[0412]

[0413] Table 9. Modified weights for angle mode.

[0414] When combining IBC-coded blocks with intra-frame-coded blocks, the weights can be designed as mask versions. That is, the weights of the IBC and intra-frame-coded blocks can be one or zero for different regions of the current block. The specific weights of the IBC and intra-frame-coded blocks can be determined based on the coding modes of neighboring blocks and the intra-frame mode of the current block. For example, the weights of the IBC and intra-frame-coded blocks can be determined as follows: when the intra-frame mode of the current block is close to the horizontal angle mode (2 <= angle mode index < 34), if the upper and left neighboring blocks of the current block are both intra-frame-coded, then the weight of the intra-frame-coded block is one in the left 3 / 4 region of the current block and zero in the right 1 / 4 region of the current block. Figure 21 As shown in (a), and conversely for the weights of IBC coded blocks; if only one neighboring block is intra-coded, the weight of the intra-coded block is one in the left half region of the current block and zero in the right half region of the current block, as shown in (a). Figure 21 As shown in (b), and conversely for the weights of IBC-coded blocks; if neither the upper nor left neighboring blocks of the current block are intra-coded, the weight of the intra-coded block is one in the left 1 / 4 region of the current block and zero in the right 3 / 4 region of the current block, as shown in (b). Figure 21 As shown in (c), and conversely for the weights of IBC coded blocks.

[0415] When the intra-frame mode of the current block is close to the vertical angle mode (34 <= angle mode index <= 66), if the upper and left neighboring blocks of the current block are both intra-coded, then the weight of the intra-coded block is one in the top 3 / 4 region of the current block and zero in the bottom 1 / 4 region of the current block. Figure 21 As shown in (d), and conversely for the weights of IBC coded blocks; if only one neighboring block is intra-coded, the weight of the intra-coded block is one in the top half region of the current block and zero in the bottom half region of the current block, as shown in (d). Figure 21 (e) shown in FIG. 8, and the weights for IBC coded blocks are reversed; if neither the top neighbor nor the left neighbor of the current block is intra coded, the weights of the intra coded block are one in the top 1 / 4 region of the current block, and zero in the bottom 3 / 4 region of the current block, as shown in Figure 21 (f) shown in FIG. 9, and the weights for IBC coded blocks are reversed.

[0416] When the intra mode of the current block is the planar mode, if both the top neighbor and the left neighbor of the current block are intra coded, the weights of the intra coded block are one in the top-left 3 / 4 region of the current block (horizontal index less than 1 / 2 width of the current block or vertical index less than 1 / 2 height of the current block), and zero in the bottom-right 1 / 4 region of the current block (horizontal index equal to or greater than 1 / 2 width of the current block and vertical index equal to or greater than 1 / 2 height of the current block), as shown in Figure 21 (g) shown in FIG. 10, and the weights for IBC coded blocks are reversed; if only the top neighbor is intra coded, the weights of the intra coded block are one in the top 1 / 2 region of the current block, and zero in the bottom 1 / 2 region of the current block, as shown in Figure 21 (e) shown in FIG. 8, and the weights for IBC coded blocks are reversed; if only the left neighbor is intra coded, the weights of the intra coded block are one in the left 1 / 2 region of the current block, and zero in the right 1 / 2 region of the current block, as shown in Figure 21 (b) shown in FIG. 7, and the weights for IBC coded blocks are reversed; if neither the top neighbor nor the left neighbor of the current block is intra coded, the weights of the intra coded block are one in the top-left 1 / 4 region of the current block (horizontal index less than 1 / 2 width of the current block and vertical index less than 1 / 2 height of the current block), and zero in the bottom-right 3 / 4 region of the current block (horizontal index equal to or greater than 1 / 2 width of the current block or vertical index equal to or greater than 1 / 2 height of the current block), as shown in Figure 21 (h) shown in FIG. 11, and the weights for IBC coded blocks are reversed.

[0417] The above two weight design methods can be utilized independently or combined together. For example, when the intra mode of the current block is the planar mode, if neither the top neighbor nor the left neighbor of the current block is intra coded, the weights of the IBC coded block and the intra coded block can be designed similar to the CIIP technique in VVC. In other conditions, the weights of the IBC coded block and the intra coded block can be designed as the mask version.

[0418] When combining the IBC coded block and the intra coded block, the weights of the IBC coded block and the intra coded block can also be designed based on the template matching method, in which the sum of absolute difference (SAD), sum of squared difference (SSD), or sum of absolute transformed difference (SATD) between the predicted samples of the current block template and the reconstructed samples can be used to calculate the weights of the IBC coded block and the intra coded block. SATD is a block matching criterion widely used in scenarios such as fractional motion estimation for video compression. For example, the weights of the IBC coded block and the intra coded block can be determined as follows: for the intra coded block, as shown in Figure 22 the SATD between the predicted samples of the current block template and the reconstructed samples is calculated as where the predicted samples of the current block template are obtained by intra-predicting the template using the reference samples of the current block with the intra mode of the current block. For the IBC coded block, as shown in Figure 23 the SATD between the predicted samples of the current block template and the reconstructed samples is calculated as where the predicted samples of the current block template are predicted using the reference samples pointed by the block vector of the current block. The weights of the IBC coded block and the intra coded block are determined as follows:

[0419]

[0420] When calculating the weights of the IBC coded block and the intra coded block using the template of the current block, if both the left template and the top template are available, both the left template and the top template can be used, or only the left template or the top template is used if only the left template or the top template is available. The use of both the left template and the top template, only the top template, or only the left template can be determined in a rate-distortion optimization (RDO) process or according to a predefined criterion. The RDO technique generally minimizes the amount of distortion (loss of video quality) relative to the amount of data required to encode the video. For the predefined criterion, for example, if the intra mode of the current block is the planar mode or the DC mode, both the left template and the top template are used; if the intra mode of the current block is close to the horizontal angle mode (2<=angle mode index<34), only the left template is used; if the intra mode of the current block is close to the vertical angle mode (34<=angle mode index<=66), only the top template is used.

[0421] When combining the coding block coded with IBC mode and the coding block coded with intra mode, the weight derived based on the template matching method can be compared with the weight derived by CIIP in the reference ECM (or the weight designed as a mask version) in the RDO process, which means that a flag at the coding block level needs to be transmitted in the bitstream to indicate which method is utilized; or the weight derived based on the template matching method can be used to replace all or part of the weight derived by CIIP in the reference ECM (or the weight designed as a mask version). For example, if the intra mode of the current block is the planar mode or the DC mode, the weight of the intra coded block and the IBC coded block is decided based on the template matching method; otherwise, the weight is decided by CIIP (or in a mask version) in the reference ECM.

[0422] In the second method, the decoder / encoder can combine the coding block coded with IBC mode and the coding block coded with inter mode. Various methods can be utilized in the combination. In one example, similar to the CIIP technique in VVC, the coding block coded with IBC merge mode is treated as the coding block coded with the planar intra mode, and it is combined with the coding block coded with inter merge mode. In another example, the coding block coded with IBC merge mode is treated as the coding block coded with the inter merge mode, and it is combined with the coding block coded with inter merge mode by equal average.

[0423] In the third method, the decoder / encoder can combine the coding block coded with IBC mode with the coding block coded with intra mode and the coding block coded with inter mode. Various methods can be utilized in the combination. In one example, the coding block coded with IBC mode, the coding block coded with intra mode, and the coding block coded with inter mode are directly combined by equal average. In another example, the coding block coded with IBC mode is first combined separately with the coding block coded with intra mode and the coding block coded with inter mode, as presented in the first method and the second method. Then, the separate combination results are combined by equal average.

[0424] CIIP improvement When designing the weight of the intra coded block and the inter coded block in CIIP, the weight can be designed based on the template matching method, in which the sum of absolute difference (SAD), sum of squared difference (SSD), or sum of absolute transformed difference (SATD) between the predicted samples and the reconstructed samples of the current block template can be used to calculate the weight of the intra coded block and the inter coded block. For example, the weight of the intra coded block and the inter coded block can be decided as follows: for the intra coded block, as shown in Equation (1), the SATD between the predicted samples and the reconstructed samples of the current block template is calculated as Figure 22 ​In this context, the predicted samples of the current block template are obtained by intra-frame prediction using the reference samples of the template in the intra-frame mode of the current block. For inter-coded blocks, such as... Figure 23 As shown (replace "IBC merge candidate BV" with "Inter-frame merge candidate MV" in the figure), the SATD between the predicted samples and reconstructed samples of the current block template is calculated as follows: In this context, the predicted samples of the current block template are predicted using the motion vectors of the current block and the reference samples of the template. (Inter-coded block) and intra-frame coded blocks The weights are determined as follows:

[0425]

[0426] When using the template of the current block to calculate the weights of inter-coded blocks and intra-coded blocks, if both the left and top templates are available, both can be used; otherwise, if only the left or top template is available, only that template can be used. The decision to use both the left and top templates, only the top template, or only the left template can be made during the RDO process or based on predefined criteria. For example, if the intra-mode of the current block is planar or DC mode, both the left and top templates are used; if the intra-mode of the current block is close to a horizontal angle mode (2 <= angle mode index < 34), only the left template is used; if the intra-mode of the current block is close to a vertical angle mode (34 <= angle mode index <= 66), only the top template is used.

[0427] When using template matching-based methods to derive the weights of intra- and inter-coded blocks in CIIP, the weights derived using template matching can be compared with those derived using the original method during the RDO process. This means that block-level flags need to be transmitted in the bitstream to indicate which method was used; alternatively, weights derived using template matching can replace all or part of the weights derived using the original method. For example, if the intra-mode of the current block is planar or DC, the weights of intra- and inter-coded blocks in CIIP are determined using template matching; otherwise, the weights are determined using the original method.

[0428] Multiple Hypothesis IBC Prediction According to one or more embodiments of this disclosure, the number of block vectors (BVs) in the IBC tool is increased to two or more, and two or more hypotheses are combined to obtain the final prediction result. Different methods can be used to achieve this goal.

[0429] In the first approach, the decoder / encoder can combine 2 hypotheses corresponding to 2 BVs to obtain the final prediction result. Various methods can be utilized to achieve this goal. In one example, the 2 BVs corresponding to the smallest and the second smallest rate-distortion metrics in the IBC AMVP mode are equally averaged to obtain the final prediction result. In another example, the prediction result corresponding to the IBC AMVP mode and the prediction result corresponding to the IBC merge mode are equally averaged to obtain the final prediction result.

[0430] In the second approach, the decoder / encoder can combine more hypotheses corresponding to more BVs to obtain the final prediction result. Various methods can be utilized to achieve this goal. In one example, the iterative accumulation method proposed in the multiple hypothesis prediction (MHP) technique is utilized to obtain the final prediction result. In another example, all BVs corresponding to the smallest, the second smallest, the third smallest,... rate-distortion metrics in the IBC AMVP mode are equally averaged to obtain the final prediction result.

[0431] Prediction block candidate derivation In some embodiments, the prediction block candidates are searched and selected according to the criterion of minimizing the template matching cost, i.e., the top N candidates that result in the smallest BV matching cost are selected. The BV matching cost can not be limited to SAD (sum of absolute difference) and SSE (sum of squared error).

[0432] In some embodiments, the prediction block candidates can be selected according to a predefined mode (i.e., planar mode).

[0433] In some embodiments, the prediction block candidates can be selected according to a neighboring predefined mode (i.e., top predefined mode, left predefined mode).

[0434] Fixed multiple hypothesis IBC In this embodiment, the weighting factors used to generate the final prediction block are predefined and fixed at both the encoder side and the decoder side. As an example, equal weighting factors can be used, i.e., the weighting factor for all candidate blocks is 1 / N.

[0435] Adaptive multiple hypothesis IBC To accommodate different characteristics of the video content, an adaptive multiple hypothesis IBC approach is also provided.

[0436] In some embodiments, the weighting factors can be derived based on the BV matching cost. The BV matching cost of the N candidates is denoted as , , The weighting factors are calculated as follows.

[0437] (4) In some embodiments, the BV matching cost can be measured with (but not limited to) SAD and SSE.

[0438] In yet another embodiment, the weighting factors can be derived / switched based on the block size or syntax elements signaled at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / Region / CTU / CU / subblock / samples level.

[0439] In yet another embodiment, the weighting factors can be derived at the encoder side and then signaled in the bitstream to the decoder. Let the N prediction block candidates be represented as , ,…, and the current block be represented as then the weighting factors can be solved by the following equation: (5) Equation (5) can be solved using the Wiener-Hopf equation as ALF. The derived filter coefficients are then quantized to integer type and signaled at block level.

[0440] In yet another embodiment, the weighting factors can be derived at the encoder side and then signaled in the bitstream to the decoder. Let the N prediction block candidates be represented as , ,…, and the current block be represented as then the weighting factors can be solved by the following equation: (6) Equation (6) can be solved using LDL decomposition or Gaussian elimination.

[0441] In yet another embodiment, the weighting factors are derived based on the template and the derived weighting factors are applied to the prediction block candidates to generate the final prediction block. Let the template of the prediction candidates be represented as , ,…, and the current block be represented as then the weighting factors can be derived using the following equation: (7) Equation (7) can be solved using the Wiener-Hopf equation. Then, the final prediction block can be calculated as where, represents the i-th prediction block candidate.

[0442] The IBC model utilizes nonlocal correlation to improve prediction accuracy, where similar blocks are searched and used to generate the final prediction block. In this embodiment, a combination of nonlocal mean filtering and multiple hypothesis IBC is provided, as described below. In the first step, N prediction block candidates are searched and identified, as performed in IBC. In the second step, weighting factors are calculated as follows.

[0443] (8) in, Used to measure the distance between the template of the i-th prediction block candidate and the template of the current block. Used as a weighting degree, and It is a normalization constant: (9) To calculate the weighting factors in equation (8), the weighting strengths must first be determined. Several methods are provided in this disclosure for determining the weighting strengths.

[0444] In the first approach, a candidate list of weighted strength values, including some typical weighted strength values, is defined and fixed on both the encoder and decoder sides. On the encoder side, rate-distortion optimization is used to examine the weighted strength values, and the optimal weighted strength value is identified and transmitted as a signal to the decoder side in the bitstream.

[0445] In the second method, the templates of the predicted block candidates and the template of the current block are used to estimate the weighted intensity value. The template of the predicted candidate is represented as... , ,…, And represent the current block as Then, the weighted intensity value can be solved using the following equation: (10) In the third method, the weighted intensity value can be estimated using the QP value and the variance of the template of the current block. That is, the relationship between the weighted intensity value, the QP value and the template variance can be fitted offline.

[0446] To better utilize the nonlocal correlations in IBC, this embodiment employs Singular Value Decomposition (SVD) to generate the final prediction block from the prediction block candidates. The width and height of the current block are represented by W and H, and the area of ​​the current block is represented by... .

[0447] Step 1. Search and identify K candidate prediction blocks Such as in FIBC.

[0448] Step 2. Current block K predicted block candidate building block groups and arranged as a matrix: (11) where, is a matrix of size by arranging each candidate in group as a column vector.

[0449] Step 3. Perform SVD decomposition on matrix .

[0450] (12) Step 4. Apply soft thresholding operation on singular value matrix .

[0451] (13) where, is a function that shrinks the diagonal elements of using a threshold . For the kth diagonal element in , it is shrunk at level by a non-linear function : (14) is a matrix composed of the shrunk singular values located on the diagonal.

[0452] Step 5. Perform inverse SVD to obtain the filtered patch group.

[0453] (15) One of the key steps is to determine the threshold for each diagonal element in Step 4. In this disclosure, the threshold is calculated as follows. For each group of image patches, the threshold is estimated using the following equation: (16) where, is the standard deviation of the noise, and is the standard deviation of the original block in the kth dimension of the SVD space of group . The deviation of the original block in the SVD space is estimated as follows.

[0454] (17) where, is the kth singular value of . When is zero, the soft thresholding operation is skipped. In addition, the threshold is used with and Parametric power function, with the bias of the prediction block to estimate the bias of the noise.

[0455] (18) where is calculated as follows, (19) Here, denotes the i-th pixel of the prediction block candidate vector .

[0456] Multiple hypothesis IBC signal transmission In the present disclosure, the proposed multiple hypothesis IBC can be used as a replacement of the current IBC mode, or the encoder can adaptively select the IBC mode or the multiple hypothesis IBC mode.

[0457] In some embodiments, the multiple hypothesis IBC can be used as a replacement of the current IBC mode, i.e., always use multiple hypotheses for prediction.

[0458] In yet another embodiment, one of the multiple hypothesis IBC methods in the above sections is used jointly with the current IBC mode. A flag is signaled in the bitstream to indicate whether the multiple hypothesis IBC mode is applied to the CU.

[0459] In yet another embodiment, more than one of the multiple hypothesis IBC methods in the above sections is used jointly with the current IBC mode. First, a flag is signaled in the bitstream to indicate whether the multiple hypothesis IBC mode is applied. Then, an index is signaled to indicate which one of the multiple hypothesis IBC methods is applied to the CU.

[0460] In yet another embodiment, the multiple hypothesis IBC methods in the above sections are used jointly with the current IBC mode. The multiple hypothesis IBC can be used as a replacement of the current IBC mode according to certain coding information of the current block, such as SAD (sum of absolute difference), SSE (sum of squared error), quantization parameter (QP) associated with the TB / CB and / or slice, neighboring prediction mode (e.g., IBC mode or intra or inter) of the CU, and / or slice type (e.g., I slice, P slice or B slice).

[0461] Combined intra TMP-intra / inter prediction According to one or more embodiments of the present disclosure, a coding block coded with the intra TMP mode is combined with a coding block coded with the intra mode or the inter mode. Different methods can be used to achieve this goal.

[0462] In the first method, the decoder / encoder can combine the coding block coded with the intra-TMP mode with the coding block coded with the intra mode. Various methods can be utilized in this combination. In one example, similar to the CIIP technique in VVC, the coding block coded with the intra-TMP mode is treated as a coding block coded with the inter merge mode, and is combined with the coding block coded with the planar intra prediction mode. In another example, similar to the combination technique of CIIP with TIMD and TM in ECM, the coding block coded with the intra-TMP mode is combined with the coding block coded with the intra prediction mode derived from TIMD.

[0463] When combining the coding block coded with the intra-TMP mode with the coding block coded with the intra mode, the weights can be decided by referring to the weight design of the CIIP technique in VVC or ECM, or, the weights can also be designed based on the template matching method, in which the sum of absolute difference (SAD), sum of squared difference (SSD), or sum of absolute transformed difference (SATD) between the predicted samples of the current block template and the reconstructed samples can be used to calculate the weights of the intra-TMP coded block and the intra coded block. For example, the weights of the intra-TMP coded block and the intra coded block can be decided as follows: for the intra coded block, as shown in Figure 22 , the SATD between the predicted samples of the current block template and the reconstructed samples is calculated as , where the predicted samples of the current block template are obtained by intra-predicting the template using the reference samples with the intra mode of the current block. For the intra-TMP coded block, the SATD between the predicted samples of the current block template and the reconstructed samples is calculated as , where the predicted samples of the current block template are predicted using the reference samples pointed by the block vector of the first block. The weights of the intra-TMP coded block and the intra coded block are decided as follows:

[0464]

[0465] When computing the weights of the intra-TMP coded block and the intra-coded block using the template of the current block, if both the left template and the top template are available, both the left template and the top template can be utilized, or only the left template or the top template is utilized if only the left template or the top template is available. The utilization of both the left template and the top template, only the top template, or only the left template can be decided in the RDO process or according to a predefined criterion. For example, if the intra mode of the current block is the planar mode or the DC mode, both the left template and the top template are utilized; if the intra mode of the current block is close to the horizontal angular mode (2 <= angular mode index < 34), only the left template is utilized; if the intra mode of the current block is close to the vertical angular mode (34 <= angular mode index <= 66), only the top template is utilized.

[0466] When combining the coded block coded with the intra-TMP mode and the coded block coded with the intra mode, the weights derived based on the template matching method can be compared with the weights derived by the CIIP in the reference ECM in the RDO process, which means that a flag at the coded block level needs to be transmitted in the bitstream to indicate which method is utilized; or the weights derived based on the template matching method can be used to replace all or part of the weights derived by the CIIP in the reference ECM. For example, if the intra mode of the current block is the planar mode or the DC mode, the weights of the intra-coded block and the IBC coded block are decided based on the template matching method; otherwise, the CIIP in the reference ECM is referred to decide the weights.

[0467] In the second method, the decoder / encoder can combine the coded block coded with the intra-TMP mode and the coded block coded with the inter mode. Various methods can be utilized in the combination. In one example, similar to the CIIP technique in VVC, the coded block coded with the intra-TMP mode is treated as the coded block coded with the planar intra mode, and is combined with the coded block coded with the inter merge mode. In another example, the coded block coded with the intra-TMP mode is treated as the coded block coded with the inter merge mode, and is combined with the coded block coded with the inter merge mode by equal average.

[0468] In the third method, the decoder / encoder can combine the coding block coded with the intra-TMP mode with the coding block coded with the intra mode and the coding block coded with the inter mode. Various methods can be utilized in this combination. In one example, the coding block coded with the intra-TMP mode, the coding block coded with the intra mode, and the coding block coded with the inter mode are directly combined by equal averaging. In another example, the coding block coded with the intra-TMP mode is first combined separately with the coding block coded with the intra mode and the coding block coded with the inter mode as presented in the first method and the second method. Then, the separate combination results are combined by equal averaging.

[0469] GPM with intra-TMP and intra-TMP prediction According to one or more embodiments of the present disclosure, the intra-TMP tool is combined with the GPM tool in the form of GPM with intra-TMP and intra-TMP prediction. Different methods can be used to achieve this goal.

[0470] In the first method, both of the two “inter” parts of the GPM with inter and inter prediction method in VVC are replaced by intra-TMP. This means that the two intra-TMP prediction results are weighted average with each other according to the splitting line in the coding block. The weights can be obtained by referring to the GPM with inter and inter prediction method in VVC.

[0471] In the second method, both of the two “inter” parts of the GPM with inter and inter prediction method in ECM are replaced by intra-TMP, where some template matching tools can be utilized to further improve the coding performance.

[0472] GPM with intra-TMP and intra prediction According to one or more embodiments of the present disclosure, the intra-TMP tool is combined with the GPM tool in the form of GPM with intra-TMP and intra prediction. Different methods can be used to achieve this goal.

[0473] In the first method, the “inter” part of the GPM with inter and intra prediction method in ECM is replaced by intra-TMP, where the intra-TMP prediction result is weighted average with the intra prediction result to obtain the final prediction signal.

[0474] GPM with intra-TMP and inter prediction According to one or more embodiments of the present disclosure, the intra-TMP tool is combined with the GPM tool in the form of GPM with intra-TMP and inter prediction. Different methods can be used to achieve this goal.

[0475] In the first method, one “inter” part of GPM in VVC which utilizes both inter and inter prediction methods is replaced by intra-TMP, where the intra-TMP prediction result is weighted averaged with the inter merge prediction result to obtain the final prediction signal.

[0476] In the second method, one “inter” part of GPM in ECM which utilizes both inter and inter prediction methods is replaced by intra-TMP, where some template matching tools can be utilized to further improve the coding performance.

[0477] Intra-TMP and intra-prediction combination in simplified GPM form According to one or more embodiments of the present disclosure, the intra-TMP tool is combined with the GPM tool in the form of simplified GPM with intra-TMP and intra-prediction, such as intra-TMP and intra-prediction are combined in a certain partition mode, which can save the bit overhead of partition representation. Different methods can be used to achieve this goal.

[0478] In the first method, for one partition line, such as 45 degrees, the top-left part of the coding block is encoded in intra-prediction mode, the bottom-right part of the coding block is encoded in intra-TMP prediction mode, and then they are averaged in GPM form to obtain the final prediction signal.

[0479] Combination of IBC and TIMD mode According to one or more embodiments of the present disclosure, the IBC tool is combined with the TIMD tool. Different methods can be used to achieve this goal.

[0480] In the first method, the IBC mode is regarded as one intra-prediction mode added in the MPM list, then the IBC mode is compared with other intra-prediction modes in the MPM list using template matching cost, and finally the two modes with the minimum cost and the second minimum cost are fused using the TIMD method to obtain the final prediction result.

[0481] In the second method, first, the conventional TIMD prediction result is obtained, then the template matching cost of the IBC mode and the conventional TIMD prediction result are calculated, and finally the IBC mode and the conventional TIMD prediction result are fused using the TIMD method to obtain the final prediction result.

[0482] Combination of intra-TMP mode and TIMD mode According to one or more embodiments of the present disclosure, the intra-TMP tool is combined with the TIMD tool. Different methods can be used to achieve this goal.

[0483] In the first method, the intra-TMP mode is treated as one of the intra prediction modes added in the MPM list, then the intra-TMP mode is compared with other intra prediction modes in the MPM list using template matching cost, and finally the two modes with the minimum cost and the second minimum cost are fused using the TIMD method to obtain the final prediction result.

[0484] In the second method, first the regular TIMD prediction result is obtained, then the template matching cost of the intra-TMP mode and the regular TIMD prediction result are calculated, and finally the intra-TMP mode and the regular TIMD prediction result are fused using the TIMD method to obtain the final prediction result.

[0485] Combining intra-TMP with LIC According to one or more embodiments of the present disclosure, the intra-TMP tool is combined with the LIC tool. Different methods can be used to achieve this goal.

[0486] In the first method, the intra-TMP mode is treated as an inter mode, and LIC is used to model the local illumination change between the current block and its intra-TMP predicted block as a function of the local illumination change between the current block template and the reference block template. The function is a linear equation as used in the regular LIC method.

[0487] Combining IBC with OBMC According to one or more embodiments of the present disclosure, the IBC tool is combined with the OBMC tool. Different methods can be used to achieve this goal.

[0488] In the first method, the IBC mode is treated as an inter mode, and the regular OBMC method is applied to refine the top and left boundary pixels of the IBC coded CU using weighted prediction with the block vector information of the neighboring blocks.

[0489] In the second method, the IBC mode is treated as an inter mode, and the template matching based OBMC method is applied to refine the top and left boundary pixels of the IBC coded CU using a template matching based method.

[0490] In some embodiments, when IBC is combined with OBMC, for a CU coded with IBC mode, when the top and left boundary pixels of the current CU are refined using the shift information of the neighboring blocks using the regular OBMC method or the template matching based OBMC method, the neighboring blocks can be coded with the IBC mode or the intra-TMP mode.

[0491] Combining intra-TMP with OBMC According to one or more embodiments of the present disclosure, the intra-TMP tool is combined with the OBMC tool. Different methods can be used to achieve this goal.

[0492] In the first method, the intra-TMP mode is treated as an inter mode, and the regular OBMC method is applied to refine the top and left boundary pixels of the intra-TMP coded CU using the block vector information of the neighboring blocks with weighted prediction.

[0493] In the second method, the intra-TMP mode is treated as an inter mode, and the template matching based OBMC method is applied to refine the top and left boundary pixels of the intra-TMP coded CU using the template matching based method.

[0494] In some embodiments, when combining the intra-TMP with the OBMC, for a CU coded with the intra-TMP mode, when using the regular OBMC method or the template matching based OBMC method to refine the top and left boundary pixels of the current CU using the shift information of the neighboring blocks, the neighboring blocks can be coded with the intra-TMP mode or the IBC mode.

[0495] Non-adjacent candidate derivation for IBC AMVP mode or IBC merge mode According to one or more embodiments of the present disclosure, the candidate derivation process of the IBC merge mode or the IBC AMVP mode is extended by using not only the adjacent neighboring blocks but also the non-adjacent neighboring blocks. The relevant content is summarized in the sections of “candidate scanning and candidate pruning”, “candidate reordering”, “motion information storage”, “application range” and “application block size”, and is presented as follows: Candidate scanning and candidate pruning For candidate scanning, the non-adjacent neighboring blocks are scanned and selected by the following method: Scanning area and distance: In one or more embodiments, the non-adjacent neighboring blocks can be scanned from the left area and the upper area of the current coding block. The scanning distance can be defined as the number of coding blocks from the scanning position to the left or top side of the current coding block.

[0496] As shown in FIG. 1 1, in the left or upper side of the current coding block, multiple rows of non-adjacent neighboring blocks can be scanned. The distance as shown in FIG. 12 represents the number of coding blocks from each candidate position to the left or top side of the current block. For example, the area with “distance 2” on the left side of the current block indicates that the candidate neighboring block located in this area is 2 blocks away from the current block. Similar indications can be applied to other scanning areas with different distances. Figure 24 Figure 24 As shown in FIG. 1 1, in the left or upper side of the current coding block, multiple rows of non-adjacent neighboring blocks can be scanned. The distance as shown in FIG. 12 represents the number of coding blocks from each candidate position to the left or top side of the current block. For example, the area with “distance 2” on the left side of the current block indicates that the candidate neighboring block located in this area is 2 blocks away from the current block. Similar indications can be applied to other scanning areas with different distances.

[0497] ​In one or more embodiments, the non-adjacent neighboring blocks at each distance can have the same block size as the current coding block, as shown in Figure 25A Note that when the non-adjacent neighboring blocks at each distance have the same block size as the current coding block, the value of the block size can be adaptively changed according to the partition granularity at each different region in the picture.

[0498] In some embodiments, the non-adjacent neighboring blocks at each distance can have different block size than the current coding block, as shown in Figure 25B Note that when the non-adjacent neighboring blocks at each distance have different block size than the current coding block, the value of the block size can be predefined as a constant value, such as 4 x 4, 8 x 8, or 16 x 16.

[0499] Based on the defined scanning distance, the total size of the scanning region to the left or above the current coding block can be determined by the configurable distance value. In one or more embodiments, the maximum scanning distance for the left side and the top side can use the same value or different values. For example, the maximum distance for the left side and the top side share the same value 2. The maximum scanning distance value(s) can be determined by the encoder side and signaled in the bitstream. Alternatively, the maximum scanning distance value(s) can be predefined as fixed value(s), such as value 2 or 4. When the maximum scanning distance is predefined as value 4, it indicates that the scanning process terminates when the candidate list is full or all the non-adjacent neighboring blocks with maximum distance 4 have been scanned (whichever comes first).

[0500] In one or more embodiments, within each scanning region at a particular distance, the starting neighboring block and the ending neighboring block can be position dependent.

[0501] In one or more embodiments, for the left side scanning region, the starting neighboring block can be the left-bottom neighboring block of the starting neighboring block in the neighboring scanning region with smaller distance. For example, as shown in Figure 24 the starting neighboring block of the “distance 2” scanning region to the left of the current block is the left-bottom neighboring block of the starting neighboring block of the “distance 1” scanning region. The ending neighboring block can be the left-adjacent block to the ending neighboring block in the above scanning region with smaller distance. For example, as shown in Figure 24 the ending neighboring block of the “distance 2” scanning region to the left of the current block is the left-adjacent block of the ending neighboring block of the “distance 1” scanning region above the current block.

[0502] Similarly, for the top side scanning region, the starting neighboring block can be the right-top neighboring block of the starting neighboring block in the neighboring scanning region with smaller distance. The ending neighboring block can be the left-top neighboring block of the ending neighboring block in the neighboring scanning region with smaller distance.

[0503] In one or more embodiments, the sampling interval between the starting neighboring block and the ending neighboring block within each scan region of a particular distance can be position dependent. In one or more embodiments, the sampling interval between the starting neighboring block and the ending neighboring block is smaller in scan regions with smaller distance. For example, as shown in FIG. 16, each neighboring block between the starting neighboring block and the ending neighboring block is scanned in a scan region with "distance 1", and every 2 neighboring blocks between the starting neighboring block and the ending neighboring block is scanned in a scan region with "distance 2". Figure 24 In one or more embodiments, the sampling interval between the starting neighboring block and the ending neighboring block can be the same or different for different side scan regions with a particular distance. For example, the sampling interval between the starting neighboring block and the ending neighboring block is the same for a left side scan region and an upper side scan region with a particular distance.

[0504] Scan order: When scanning neighboring blocks in non-adjacent regions, a certain order or / and rule can be followed to determine the selection of the scanned neighboring blocks.

[0505] In one or more embodiments, the left side region can be scanned first, and then the upper region. As shown in FIG. 17, the first three rows of non-adjacent regions (e.g., from distance 1 to distance 3) on the left side can be scanned first, and then the first three rows of non-adjacent regions above the current block are scanned. Figure 24

[0506] In some embodiments, the left side region and the upper region can be scanned alternately. For example, as shown in FIG. 18, a left side scan region with "distance 1" is scanned first, and then an upper region with "distance 1" is scanned. Figure 24

[0507] For scan regions on the same side (e.g., left side region or upper region), the scan order is from regions with smaller distance to regions with larger distance. This order can be flexibly combined with other embodiments of the scan order. For example, the left side region and the upper region can be scanned alternately, and the order of regions on the same side is arranged from small distance to large distance.

[0508] A scan order within each scan region of a particular distance can be defined. In one or more embodiments, for a left side scan region, the scan can start from the bottom neighboring block to the top neighboring block. For an upper scan region, the scan can start from the right side block to the left side block.

[0509] In one or more embodiments, non-adjacent regions in one direction can be scanned first, and then non-adjacent regions in other directions are scanned. Within one direction, a scan order can be defined. In one or more embodiments, within each direction, the scan can start from smaller distance to larger distance. In one or more embodiments, the sampling interval between the starting neighboring block and the ending neighboring block within each scan region of a particular distance can be position dependent. In one or more embodiments, the sampling interval between the starting neighboring block and the ending neighboring block is smaller in scan regions with smaller distance. For example, as shown in FIG. 16, each neighboring block between the starting neighboring block and the ending neighboring block is scanned in a scan region with "distance 1", and every 2 neighboring blocks between the starting neighboring block and the ending neighboring block is scanned in a scan region with "distance 2". Figure 24 In one or more embodiments, the sampling interval between the starting neighboring block and the ending neighboring block can be the same or different for different side scan regions with a particular distance. For example, the sampling interval between the starting neighboring block and the ending neighboring block is the same for a left side scan region and an upper side scan region with a particular distance.

[0504] Scan order: When scanning neighboring blocks in non-adjacent regions, a certain order or / and rule can be followed to determine the selection of the scanned neighboring blocks.

[0505] In one or more embodiments, the left side region can be scanned first, and then the upper region. As shown in FIG. 17, the first three rows of non-adjacent regions (e.g., from distance 1 to distance 3) on the left side can be scanned first, and then the first three rows of non-adjacent regions above the current block are scanned. Figure 24

[0506] In some embodiments, the left side region and the upper region can be scanned alternately. For example, as shown in FIG. 18, a left side scan region with "distance 1" is scanned first, and then an upper region with "distance 1" is scanned. Figure 24

[0507] For scan regions on the same side (e.g., left side region or upper region), the scan order is from regions with smaller distance to regions with larger distance. This order can be flexibly combined with other embodiments of the scan order. For example, the left side region and the upper region can be scanned alternately, and the order of regions on the same side is arranged from small distance to large distance.

[0508] A scan order within each scan region of a particular distance can be defined. In one or more embodiments, for a left side scan region, the scan can start from the bottom neighboring block to the top neighboring block. For an upper scan region, the scan can start from the right side block to the left side block.

[0509] In one or more embodiments, non-adjacent regions in one direction can be scanned first, and then non-adjacent regions in other directions are scanned. Within one direction, a scan order can be defined. In one or more embodiments, within each direction, the scan can start from smaller distance to larger distance.

[0510] In some embodiments, non-adjacent regions in different directions can be scanned alternately. For example, as shown in FIG. 3A, the non-adjacent regions with small distances in the direction of 225 degrees of angle degree value are scanned first, and then the non-adjacent regions with small distances in the directions of 45, 90, 180, and 135 degrees of angle degree value are scanned in turn. Thereafter, the non-adjacent regions with large distances in the direction of 225 degrees of angle degree value are scanned, and then the non-adjacent regions with large distances in the directions of 45, 90, 180, and 135 degrees of angle degree value are scanned. Figures 26 to 27

[0511] In some examples, a total of 18 blocks are scanned, as shown in FIG. 3B, where the scanned blocks are indicated by the integer n with a box, where n is in the range from 1 to 18 (inclusive), and where n represents the scanning order. Figure 26

[0512] In some examples, a total of 48 blocks are scanned, as shown in FIG. 3C, where the scanned blocks are indicated by the integer n with a box, where n is in the range from 1 to 48 (inclusive), and where n represents the scanning order; in these examples, the degree values 270, 0, 247.5, 22.5, 202.5, 67.5, 157.5, and 112.5 can additionally be used to determine the scanning order. Figure 27

[0513] Scan termination: For non-adjacent candidates, the neighboring blocks coded with IBC mode or Intra TMP mode are defined as eligible candidates.

[0514] In one or more embodiments, the scanning process can be performed interactively. For example, the scanning performed in a particular region of a particular distance can be stopped at the moment when the first X eligible candidates are identified, where X is a predefined positive value. For example, as shown in FIG. 3D, the scanning in the left scanning region of distance 1 can be stopped when the first one or more eligible candidates are identified. Then, the next iteration of the scanning process is started by targeting another scanning region, which is regulated by the predefined scanning order / rule. Figure 24

[0515] In one or more embodiments, X can be defined for each distance. For example, at each distance, X is set to 1, which means that for each distance, if the first eligible candidate is found, the scanning is terminated, and the scanning process is restarted from a different distance of the same region or from the same or different distances of different regions. Note that for different distances, the value of X can be set to the same value or different values. If the maximum number of eligible candidates is found from all allowed distances (e.g., specified by the maximum distance) of a region, the scanning process of that region is completely terminated. ​​​​

[0516] In another embodiment, X can be defined for regions. For example, X is set to 3, which means for the whole region (e.g., the region above or left of the current block), if the top 3 eligible candidates are found, the scanning is terminated and the scanning process is restarted from the same or different distance of another region. Note that the value of X can be set to the same or different values for different regions. If the maximum number of eligible candidates is found from all regions, the whole scanning process is terminated completely.

[0517] The value of X can be defined for both distance and region. For example, X is set to 3 for each region (e.g., the region above or left of the current block), and X is set to 1 for each distance. The value of X can be set to the same or different values for different regions and distances.

[0518] In some embodiments, the scanning process can be performed continuously. For example, the scanning performed in a particular region at a particular distance can stop at the moment when all covered neighboring blocks are scanned and no more eligible candidates are identified or the maximum allowable number of candidates is reached. The maximum allowable number of candidates can be set according to different methods. In one example, the maximum allowable number of candidates is set to a predefined value, which can be set to the maximum allowable size of the IBC merge candidate list (equal to 28 in ECM), or the maximum allowable size of the IBC merge candidate list minus 1, or other values. In another example, the maximum allowable number of candidates can be set to different values depending on the coding condition, and the value is transmitted in the bitstream.

[0519] In the candidate scanning process, each candidate non-adjacent neighboring block is determined and scanned by following the scanning method proposed above. To make it easier to implement, each candidate non-adjacent neighboring block can be indicated or located by a specific scanning position. For example, the right-bottom position is used for both the above and left non-adjacent neighboring blocks.

[0520] After an eligible candidate is identified after the above process, the candidate can be checked for similarity against all existing candidates already in the candidate list. The details of the similarity check can refer to the similarity check rules already existing in the current IBC candidate derivation. If the new eligible candidate is found to be similar to any existing candidate in the candidate list, the new eligible candidate is removed / pruned.

[0521] In some embodiments, the above candidate scanning and candidate pruning processes can be the same or different for IBC AMVP candidate derivation and IBC merge candidate derivation. For example, Figure 26 The candidate scanning and pruning processes presented in the above can be used for both IBC AMVP candidate derivation and IBC merge candidate derivation. In another example, Figure 26The candidate scanning and pruning processes presented in this disclosure can be used for IBC AMVP candidate derivation, Figure 27 The candidate scanning and pruning processes presented in this disclosure can be used for IBC merge candidate derivation. In a third example, Figure 26 The candidate scanning and pruning processes presented in this disclosure can be used for IBC AMVP candidate derivation, Figure 41 The candidate scanning and pruning processes presented in this disclosure can be used for IBC merge candidate derivation. In a fourth example, Figure 44 The candidate scanning and pruning processes presented in this disclosure can be used for IBC AMVP candidate derivation, Figure 41 The candidate scanning and pruning processes presented in this disclosure can be used for IBC merge candidate derivation. In a fifth example, Figure 26 The candidate scanning and pruning processes presented in this disclosure can be used for IBC AMVP candidate derivation, Figure 45 The candidate scanning and pruning processes presented in this disclosure can be used for IBC merge candidate derivation. In a sixth example, Figure 46 The candidate scanning and pruning processes presented in this disclosure can be used for IBC AMVP candidate derivation, Figure 41 The candidate scanning and pruning processes presented in this disclosure can be used for IBC merge candidate derivation. In a seventh example, Figure 47 The candidate scanning and pruning processes presented in this disclosure can be used for IBC AMVP candidate derivation, Figure 41 The candidate scanning and pruning processes presented in this disclosure can be used for IBC merge candidate derivation. In an eighth example, Figure 26 The candidate scanning and pruning processes presented in this disclosure can be used for IBC AMVP candidate derivation, Figure 48 The candidate scanning and pruning processes presented in this disclosure can be used for IBC merge candidate derivation. In a ninth example, Figure 26 The candidate scanning and pruning processes presented in this disclosure can be used for IBC AMVP candidate derivation, Figure 49 The candidate scanning and pruning processes presented in this disclosure can be used for IBC merge candidate derivation. In a tenth example, Figure 41 The candidate scanning and pruning processes presented in this disclosure can be used for both IBC AMVP candidate derivation and IBC merge candidate derivation. In an eleventh example, Figure 41 The candidate scanning and pruning processes presented in this disclosure can be used for IBC merge candidate derivation, and IBC AMVP candidate derivation is unchanged. In the above examples, Figure 26 , Figure 27 , Figure 41 , Figure 44 , Figure 45 , Figure 46 , Figure 47 , Figure 48 and Figure 49Examples of scan area, scan distance, and scan order information are presented. Regarding scan termination, in one example, a scan performed in a specific area at a specific distance stops when all covered neighboring blocks have been scanned and no more qualified candidates have been identified, or the maximum allowed number of IBC merge candidates has been reached. In another example, a scan performed in a specific area at a specific distance stops when all covered neighboring blocks have been scanned and no more qualified candidates have been identified, or the maximum allowed number of IBC merge candidates has been reached minus one. To illustrate this more clearly... Figure 26 , Figure 41 and Figure 44 Each non-adjacent neighboring block at a distance has the same characteristics as... Figure 26 , Figure 41 and Figure 44 The current encoded block has the same block size, in Figure 26 and Figure 44 The maximum scan distance on the left and top sides is set to 4 (excluding adjacent blocks), and... Figure 41 The maximum scan distance on the left and top sides is set to 7 (excluding adjacent blocks). Figure 26 , Figure 41 and Figure 44 In the middle, the bottom right scan position is used for non-adjacent neighboring blocks along the bottom left, top right, and top left directions. Figure 26 , Figure 41 and Figure 44 The middle and lower middle (horizontal index equal to half the block width, vertical index equal to the vertical index of the non-adjacent neighboring blocks along the upper right direction within the same scan distance) scan position is used for non-adjacent neighboring blocks along the upper direction. Figure 26 , Figure 41 and Figure 44 The middle and middle-right (vertical index equal to half the block height, horizontal index equal to the horizontal index of the non-adjacent neighboring blocks along the lower left direction within the same scan distance) scan positions are used for non-adjacent neighboring blocks along the left direction.

[0522] Candidate Reordering When inserting spatially non-adjacent candidates into the IBC candidate list, all spatially non-adjacent candidates can be grouped and inserted as a whole into different positions in the IBC candidate list, or spatially non-adjacent candidates can be divided into several subgroups and each subgroup can be inserted into a different position in the IBC candidate list.

[0523] In one or more embodiments, spatially non-adjacent candidates may be inserted into the IBC candidate list in the following order: 1. Spatial BVP from spatially adjacent neighboring blocks 2. Spatial BVP from non-adjacent neighboring blocks 3. Historical BVP from FIFO table 4. Pairwise average BVP 5. BVP candidate located in IBC reference region as shown in Figure 13 6. Zero BVP In another embodiment, the spatial non-adjacent candidates can be inserted into the IBC candidate list in the following order: 1. Spatial BVP from spatial-adjacent neighboring blocks 2. History-based BVP from FIFO table 3. Spatial BVP from spatial non-adjacent neighboring blocks 4. Pairwise average BVP 5. BVP candidate located in IBC reference region as shown in Figure 13 6. Zero BVP In another embodiment, the spatial non-adjacent candidates can be inserted into the IBC candidate list in the following order: 1. Spatial BVP from spatial-adjacent neighboring blocks 2. First X spatial BVP from spatial non-adjacent neighboring blocks 3. History-based BVP from FIFO table 4. Other Y spatial BVP from spatial non-adjacent neighboring blocks 5. Pairwise average BVP 6. BVP candidate located in IBC reference region as shown in Figure 13 7. Zero BVP where the values of X and Y can be pre-defined fixed values (e.g. value 2), signaled values received by the decoder (signaled parameters at sequence / slice / block / CTU level), or values configurable at the encoder / decoder, or values determined dynamically according to the number of available neighboring blocks on the left and above of each individual coded block (e.g. X<= 3, Y<=3), or any combination of methods to determine the values of X and Y. In one example, the value of X can be the same as the value of Y. In another example, the value of X can be different from the value of Y.

[0524] Since the candidates placed in the later positions of the IBC candidate list can incur higher signaling overhead when selected and signaled by the encoder, the order of the above different categories of candidates can be designed in the following different ways: In one or more embodiments, the order of these candidates remains the same as the above insertion order. Adaptive reordering methods can then be applied to reorder these candidates; the adaptive reordering methods can be template matching based methods (ARMC).

[0525] ​​​In one or more embodiments, the derived spatial non-adjacent candidates can first be applied with an adaptive reordering method (which can be an ARMC based method) before being inserted into the IBC candidate list, and then the top X candidates can be inserted into the IBC candidate list based on the above insertion method.

[0526] The value of X can be a predefined fixed value (such as value 2), a signaled value received by the decoder (a signaled parameter at sequence / slice / block / CTU level), or a value that can be configured at the encoder / decoder, or a value that is dynamically decided according to the number of available neighboring blocks on the left and above of each individual coded block (e.g., X <= 3), or any combination of methods to determine the value of X.

[0527] The above reordering methods can be selected and applied according to different factors: in one or more embodiments, the reordering method can be selected based on the type of video frame / slice. For example, for low-delay pictures or slices, all spatial non-adjacent candidates can be placed after all spatial-adjacent candidates. And for non-low-delay pictures or slices, the top X spatial non-adjacent candidates can be placed after the spatial-adjacent candidates, and the remaining spatial non-adjacent candidates can be placed after the history-based BVP candidates.

[0528] In some embodiments, the above candidate reordering process can be the same or different for IBC AMVP candidate list derivation and IBC merge candidate list derivation. For example, for both IBC AMVP candidate list derivation and IBC merge candidate list derivation, all spatial non-adjacent candidates are placed after all spatial-adjacent candidates. In another example, for IBC AMVP candidate list derivation, all spatial non-adjacent candidates are placed after all spatial-adjacent candidates; for IBC merge candidate list derivation, the top X spatial non-adjacent candidates can be placed after the spatial-adjacent candidates, and the remaining spatial non-adjacent candidates can be placed after the history-based BVP candidates. In a third example, for IBC merge candidate list derivation, all spatial non-adjacent candidates are placed after all spatial-adjacent candidates; for IBC AMVP candidate list derivation, all spatial non-adjacent candidates are placed after the history-based BVP candidates.

[0529] Motion information storage When scanning the spatial non-adjacent neighboring blocks based on the candidate derivation methods provided above, the selected spatial non-adjacent neighboring blocks can be IBC coded blocks or intra-TMP coded blocks. In the case of both IBC coded blocks and intra-TMP coded blocks, the motion information can include a translational BV.

[0530] The motion information of these blocks, whether IBC or intra-TMP coded blocks, can need to be saved in memory after the blocks are coded. To save memory usage, the spatial non-adjacent neighboring blocks can be restricted within a certain area.

[0531] As shown in Figure 28 , the allowed non-adjacent area for scanning the spatial non-adjacent neighboring blocks can be restricted to a limited area size.

[0532] In one or more embodiments, the restricted area can apply to IBC or intra-TMP spatial neighboring blocks.

[0533] The size of the allowed non-adjacent area can be defined in terms of the size of the current coding tree unit (CTU), e.g., an integer (e.g., 1 or 2 or other integer) or a fraction (e.g., 0.5 or 0.25 or other fraction) of the current CTU size.

[0534] The size of the allowed non-adjacent area can be defined in terms of a fixed number of pixels or samples, e.g., 128 samples above the current CTU or / and to the left of the current CTU.

[0535] The size (e.g., in terms of CTU size or number of samples) can be a pre-specified value or a signaled value determined at the encoder and carried in the bitstream.

[0536] The size of the restricted area can be defined separately for the top and left non-adjacent neighboring blocks. In some examples, the non-adjacent neighboring blocks can be spatial non-adjacent neighboring blocks, the top non-adjacent neighboring blocks can be top spatial non-adjacent neighboring blocks, and the left non-adjacent neighboring blocks can be left spatial non-adjacent neighboring blocks. In one example, the above non-adjacent neighboring blocks can be restricted within the current CTU or outside the current CTU but within a fixed number of samples / pixels from the top boundary of the current CTU, such that no extra row buffer is needed to save the motion information of the above non-adjacent neighboring blocks. For example, if an existing row buffer already covers the neighboring area 8 samples row from the top of the current CTU, the fixed number can be defined as 8. In another example, the left non-adjacent neighboring blocks can be restricted within the current CTU or outside the current CTU but within a predefined or signaled number of samples / pixels from the left boundary of the current CTU.

[0537] As shown in Figures 29A to 29BAs shown, if the allowed non-adjacent region exceeds the current CTU, the allowed non-adjacent region above the current CU (non-adjacent IBC neighbor or intra-TMP neighbor) can have a large memory cost. In this case, the actual memory cost increases proportionally with the maximum allowed scan distance in the picture width and vertical direction. To reduce the line buffer (i.e., above non-adjacent region beyond the current CTU) cost, the height of the above non-adjacent region beyond the current CTU can be limited to a value h (as shown in Figure 29A Note that this value of h can be configurable and signaled to the decoder. In the case that IBC motion and intra-TMP motion are stored in separate buffers, as shown in the example of Figure 29B Different methods can exist to save the motion in the line buffer: in one method, the line buffer used to store IBC motion can indicate that the buffer region where CU B resides is set to be invalid since CU B is not an IBC CU. In another method, the line buffer used to store IBC motion can indicate that the buffer region where CU B resides is set to be valid and IBC motion is copied from CU A since CU A is the adjacent IBC neighbor of CU B.

[0538] In the example of Figures 29A to 29B The height value h and width value w can be set to multiples of 4 for easier implementation. In one embodiment, the values h and w can be set to the minimum value of an IBC CU (e.g., 4).

[0539] When the allowed non-adjacent region for scanning spatial non-adjacent neighboring blocks is limited to a finite region size, as shown in the examples of Figure 28 and FIG. 29, the scanned non-adjacent neighboring position can be outside the allowed non-adjacent region. In this case, different methods can be used to resolve this issue: In one method, the scanning process can indicate that the scanned position has no valid neighboring information.

[0540] In another method, the scanning process can project or clip this out-of-range position to another position within the allowed non-adjacent region. As shown in the example of Figure 30As shown in the example of FIG. 3, there are two positions (i.e., two points 3001) outside the range of allowable non-adjacent regions. These two positions are projected / clipped to two other positions, which are at the same vertical / horizontal coordinates but within the allowable non-adjacent regions. The projected / clipped new positions can be located on the boundary of the allowable non-adjacent region closest to the original positions. In the case where the allowable non-adjacent region beyond the current CTU is set to have values w and h set to the minimum value (e.g., 4) of IBC CU, the projected / clipped new positions can be interchangeably set on one boundary or the other, as the buffer is so small that it can store motion information from only one CU, and in this case, there is no difference to be clipped to one boundary side or the other.

[0541] In another approach, the allowable spatial non-adjacent region can include three regions. As shown in FIG. 4, the three regions are spatial regions of the top-left, left-side, and top regions that are located outside the current CTU and adjacent to the current CTU. The height of the top-left and top regions in the allowable spatial non-adjacent region is defined as h, while the width can depend on the picture width. The width of the left-side allowable spatial non-adjacent region is defined as w, while the height is equal to the height of the current CTU. As shown in FIG. 4, if a scanned non-adjacent position (point 3001 in FIG. 3) is outside the allowable spatial region, then the projected / clipped new position (one of points 3002 in FIG. 3) can be defined differently. Figure 31 Figure 31 As shown in FIG. 4, if a scanned non-adjacent position (one of points 3001 in FIG. 3) is outside the allowable spatial region, then the projected / clipped new position (one of points 3002 in FIG. 3) can be defined differently. Figure 31 Figure 31 For the top-left non-adjacent region, the projected / clipped new position is always the pixel position adjacent to the top-left position of the current CTU. For example, if the top-left position of the current CTU is (ctu_x, ctu_y), then the projected / clipped new position is (ctu_x - 1, ctu_y - 1). For the top non-adjacent region, the projected / clipped new position has the same horizontal coordinate, but the vertical coordinate becomes (ctu_y - 1). For the left-side non-adjacent region, the projected / clipped new position has the same vertical coordinate, but the horizontal coordinate becomes (ctu_x - 1).

[0542] When the motion information of an IBC coded block is saved in memory, the motion information can be saved at the granularity of the minimum IBC block size (e.g., 4 x 4 block). In the case where the current IBC coded block is a coding unit of a size larger than the minimum IBC block, then a different approach can be used to save the motion information. In one or more embodiments, the motion information saved at each minimum IBC block (e.g., 4 x 4 block) within the current block is simply a repeated copy of the motion information of the current block.

[0543] ​​Alternatively or additionally, the motion information of an IBC coding block can be saved at different granularities a x b (e.g., 8 x 8 or 8 x 16 or 16 x 8 or 16 x 16 granularities, etc.) instead of the minimum IBC block size (e.g., 4 x 4 granularity), where the granularity values of a and b can be configurable or decided by the encoder and then signaled to the decoder. Without loss of generality, an illustrative example is taken with 8 x 8 granularity (e.g., a = b = 8). If further assume the minimum IBC block size is 4 x 4, it means that each 8 x 8 block can only save one set of IBC motion information, which represents one single IBC model, even though the four 4 x 4 sub-blocks within this 8 x 8 block can come from more than one IBC block, as shown in Figure 32 Figure 32 In this case, the four 4 x 4 sub-blocks A, B, C, D form one 8 x 8 block / region, only one IBC model information is saved. However, the four 4 x 4 sub-blocks come from four different IBC blocks, which represent four IBC models and include four sets of IBC motion information. In this case, there can be different ways to derive and save one set of IBC information: In one or more embodiments, one set of the available IBC motion information can be selected and saved. In one example, the IBC motion information at a fixed or configurable location (e.g., the top-left minimum IBC block) is selected for motion storage. In another example, the average IBC motion information of the multiple models can be calculated for motion storage.

[0544] The IBC motion information at the selected neighboring IBC blocks can be simplified / compressed before storage. In one embodiment, each saved BV can be compressed before storage to further reduce the memory size. One example is to use general techniques for data compression. For example, such techniques are provided to save a complex value consisting of one exponent and a tail array to approximately represent each saved BV.

[0545] The above-provided methods for motion information storage can be applied in any combination. For example, the restricted region defined for non-adjacent neighboring blocks can be combined with the use of compressed IBC motion information.

[0546] Scope of application ​The methods provided above in the "candidate scanning and candidate pruning", "candidate reordering", and "motion information storage" sections are primarily for deriving spatial non-adjacent candidates for IBC AMVP mode or IBC merge mode. When temporal candidates are also used for IBC AMVP mode or IBC merge mode, the temporal non-adjacent candidates can also utilize the non-adjacent neighboring blocks in the collocated picture. Unlike the spatial non-adjacent candidates that are primarily extracted from the left region and the above region of the current block, the temporal non-adjacent candidates can be extracted from the left region, the above region, the right region, the below region, and the collocated region of the current block. In addition, the methods provided above in the "candidate scanning and candidate pruning", "candidate reordering", and "motion information storage" sections can also be applied to the temporal non-adjacent candidates for IBC AMVP mode or IBC merge mode in a similar manner.

[0547] Applying block sizes The methods provided above in the "candidate scanning and candidate pruning", "candidate reordering", "motion information storage", and "applying scope" sections for deriving spatial / temporal non-adjacent candidates for IBC AMVP mode or IBC merge mode can be applied to different combinations of block sizes. The applying block size for spatial / temporal adjacent candidates and the applying block size for spatial / temporal non-adjacent candidates can be the same or different. In a first example, both spatial adjacent candidates and spatial non-adjacent candidates are applied to coding blocks with an area greater than 16. In a second example, both spatial adjacent candidates and spatial non-adjacent candidates are applied to all coding block sizes. In a third example, spatial adjacent candidates are applied to coding blocks with an area greater than 16, and spatial non-adjacent candidates are applied to all coding block sizes. In a fourth example, spatial adjacent candidates are applied to all coding block sizes, and spatial non-adjacent candidates are applied to coding blocks with an area greater than 16. In the above four applying block size examples, for the scanning region, scanning distance, and scanning order, in one example, the following applies: Figure 26 The candidate scanning and pruning processes presented in the "candidate scanning and candidate pruning" section are applied to derive spatial non-adjacent candidates for IBC AMVP mode and IBC merge mode. In another example, the candidate scanning and pruning processes presented in the "candidate scanning and candidate pruning" section are applied to derive spatial non-adjacent candidates for IBC AMVP mode, and the candidate scanning and pruning processes presented in the "candidate scanning and candidate pruning" section are applied to derive spatial non-adjacent candidates for IBC merge mode. Figure 26 The candidate scanning and pruning processes presented in the "candidate scanning and candidate pruning" section are applied to derive spatial non-adjacent candidates for IBC AMVP mode and IBC merge mode. In another example, the candidate scanning and pruning processes presented in the "candidate scanning and candidate pruning" section are applied to derive spatial non-adjacent candidates for IBC AMVP mode, and the candidate scanning and pruning processes presented in the "candidate scanning and candidate pruning" section are applied to derive spatial non-adjacent candidates for IBC merge mode. Figure 41The candidate scanning and pruning process presented in the above can be applied to derive spatial non-adjacent candidates for IBC merge mode. In the above four application block size examples, for scan termination, in one example, the scan performed in a certain region at a certain distance stops at the moment when all covered neighboring blocks are scanned and no more qualified candidate is identified or the maximum allowable IBC merge candidate number is reached. In another example, the scan performed in a certain region at a certain distance stops at the moment when all covered neighboring blocks are scanned and no more qualified candidate is identified or the maximum allowable IBC merge candidate number minus one is reached. The above application block size examples, scan region, scan distance and scan order examples, and scan termination examples can be freely combined, which can constitute different examples of the overall scheme of IBC with non-adjacent candidates.

[0548] Subblock-based IBC mode According to one or more embodiments of the present disclosure, IBC mode is extended to subblock level, in which a subblock in the current block can have its own BV for motion compensation. Different example methods can be used to achieve this goal.

[0549] In a first example method, as shown in FIG. 1 1, the BVs of the subblocks in the current block are obtained by reusing the BVs of the subblocks of the collocated blocks in the collocated picture. If the BV of one of the subblocks in the collocated block cannot be obtained, e.g., the subblock is intra coded, the BV of the subblock can be set to the BV of the collocated block. In addition to the collocated blocks in the collocated picture, blocks at other positions can also be utilized to obtain the subblock level BVs, in which the other positions can be referred to temporal non-adjacent candidates. In addition, a template matching method can be used to refine the BVs of the collocated blocks in the collocated picture or the subblocks at other positions. Figure 33 In a second example method, as shown in FIG. 12, the BVs of the left or above subblocks in the current block are obtained by refining the BVs of the current block with a template matching method. The BVs of the current block can be obtained by regular IBC mode, TM IBC mode or other modes. For the left subblock in the current block, only the left template can be utilized to refine the BV of the subblock. For the above subblock in the current block, only the above template can be utilized to refine the BV of the subblock. For the top-left subblock in the current block, both the left template and the above template can be utilized to refine the BV of the subblock.

[0550] Figure 34 Multiple template modes for TM IBC mode and TM regular mode According to one or more embodiments of the present disclosure, the template modes for TM IBC mode and TM regular mode are extended by introducing more kinds of templates. Different methods can be used to achieve this goal.

[0551] Multiple template modes for TM IBC mode and TM regular mode

[0552] ​In the first method, in addition to the template mode using both the above template and the left template currently used, other template modes (e.g., using only the left template, using only the above template, etc.) can also be used for TM IBC mode or TM regular inter mode. When more than one template mode is used for TM IBC mode or TM regular inter mode, a pre-defined criterion or a flag transmitted in the bitstream can be used to decide which template mode is finally used for the current block.

[0553] IBC and Intra TMP for Deblocking Filter Boundary Strength Derivation According to one or more embodiments of the disclosure, when obtaining the boundary strength, the deblocking filter treats the blocks coded with IBC mode and the blocks coded with Intra TMP mode equally. Different methods can be used to achieve this goal.

[0554] In the first method, when obtaining the boundary strength of the deblocking filter, both the blocks coded with IBC mode and the blocks coded with Intra TMP mode are treated as blocks coded with Intra mode. Then, the boundary strength decision criteria for blocks coded with Intra mode can be applied to the blocks coded with IBC mode and the blocks coded with Intra TMP mode. For example, if both of the two neighboring blocks are coded with IBC mode or Intra TMP mode, or one of the two neighboring blocks is coded with IBC mode or Intra TMP mode, the boundary strength is set to a predetermined positive integer, e.g., 2.

[0555] In the second method, when obtaining the boundary strength of the deblocking filter, both the blocks coded with IBC mode and the blocks coded with Intra TMP mode are treated as blocks coded with Inter mode. Then, the boundary strength decision criteria for blocks coded with Inter mode can be applied to the blocks coded with IBC mode and the blocks coded with Intra TMP mode. For example, if both of the two neighboring blocks are coded with IBC mode, or both of the two neighboring blocks are coded with Intra TMP mode, or one of the two neighboring blocks is coded with IBC mode and the other is coded with Intra TMP mode, the boundary strength is set to a predetermined positive integer, e.g., 1, if the two block vectors of the two neighboring blocks are different (or the absolute difference of the horizontal component or the vertical component of the two block vectors is larger than a threshold (e.g., half-pixel)). In another example, if the difference of the two block vectors of the two neighboring blocks is large, the boundary strength is set to a larger value.

[0556] In a third method, when a block coded with IBC mode or a block coded with Intra-TMP mode is combined with Intra tools (e.g. IBC is combined with Intra prediction, i.e. GPM with IBC and Intra prediction, Intra-TMP is combined with Intra prediction, i.e. GPM with Intra-TMP and Intra prediction, etc.), the block is considered as a block coded with Intra mode when obtaining the boundary strength of the deblocking filter.

[0557] In a fourth method, when a block coded with IBC mode or a block coded with Intra-TMP mode is combined with Inter tools (e.g. IBC is combined with Inter prediction, i.e. GPM with IBC and Inter prediction, Intra-TMP is combined with Inter prediction, i.e. GPM with Intra-TMP and Inter prediction, etc.), the block is considered as a block coded with Inter mode when obtaining the boundary strength of the deblocking filter.

[0558] Combination of IBC and Intra-TMP According to one or more embodiments of the present disclosure, the IBC tool is combined with the Intra-TMP tool. Different methods can be used to achieve this goal.

[0559] In a first method, the IBC tool is combined with the Intra-TMP tool in the form of GPM with IBC and Intra-TMP prediction. In one example, the two “inter” parts of GPM with Inter and Intra prediction method in VVC are replaced by IBC and Intra-TMP prediction respectively. This means that the IBC prediction result and the Intra-TMP prediction result are weighted average with each other according to the partition line in the coded block. The weights can be obtained by referring to GPM with Inter and Intra prediction method in VVC. In another example, the two “inter” parts of GPM with Inter and Intra prediction method in ECM are replaced by IBC and Intra-TMP prediction, where some template matching tools can be used to further improve the coding performance.

[0560] In a second method, the IBC tool is combined with the Intra-TMP tool in the form of CIIP with IBC and Intra-TMP prediction. In one example, the IBC prediction result and the Intra-TMP prediction result are weighted with fixed weights, where the specific weight values depend on the coding mode of the neighboring blocks. In another example, the fixed weight values are obtained based on a template matching method.

[0561] TM-based reordering of MMVD candidates of GPM with MMVD mode According to one or more embodiments of the present disclosure, a template matching (TM)-based method is used to reorder MMVD candidates of GPM with MMVD mode. Different methods can be used to achieve this goal.

[0562] In the first method, similar to GPM with template matching, different templates are utilized to reorder MMVD candidates for different parts of a GPM partition of a predefined split mode. After reordering, an index is signaled to represent the selected MMVD candidate for a part of the GPM partition.

[0563] In the second method, one template is utilized to reorder MMVD candidates for different parts of a GPM partition of different split modes. After reordering, an index is signaled to represent the selected MMVD candidate for a part of the GPM partition.

[0564] IBC / regular inter / affine inter HMVP candidate utilization According to one or more embodiments of the present disclosure, IBC / regular inter / affine inter HMVP candidates are utilized based on their relative positions to the current block. Different methods can be used to achieve this goal.

[0565] In the first method, when encoding the current block, first all IBC / regular inter / affine inter HMVP candidates are divided into several groups according to the distance between the center position of the HMVP candidate and the center position of the current block. For example, using some predefined thresholds , , , the HMVP candidates with distance smaller than are divided into group 1, the HMVP candidates with distance equal to or larger than but smaller than are divided into group 2, …, the HMVP candidates with distance equal to or larger than but smaller than are divided into group N. Different methods can be used to set the specific values of the predefined thresholds. For example, is set to the maximum of the height and width of the current block. is set to 2 times of , is set to N times of . Then, in each divided group, the first valid candidate is selected in a predefined scan order. Different methods can be used to define the predefined scan order, for example, the HMVP candidates in each divided group are scanned in the following order: left direction, up direction, left-down direction, right-up direction, left-up direction. The value of may be set to be the same in all divided groups, or different in different divided groups. Finally, the selected candidates in each divided group are added to the IBC / regular inter / affine inter candidate list in the order from smaller distance to larger distance.

[0566] In the second method, after dividing all IBC / regular inter / affine inter HMVP candidates into several groups according to the distance between the center position of the HMVP candidate and the center position of the current block, the first effective candidate in each divided group is selected based on the template matching method. Specifically, the HMVP candidates in each divided group are reordered based on the template matching method, and then the first effective candidate with the smallest template matching cost is selected. Finally, the selected candidate in each divided group is added to the IBC / regular inter / affine inter candidate list in the order from smaller distance to larger distance.

[0567] When utilizing IBC / regular inter / affine inter HMVP candidates based on the encoding order of the HMVP candidates (original scheme in VVC or ECM) or the relative position of the HMVP candidate to the current block (proposed scheme), the number of HMVP candidates can be further enlarged. Different methods can be used to achieve this goal. In one example, there is only one HMVP table to save the HMVP candidates as in the original design in VVC or ECM, but the size of the HMVP table is further enlarged. In another example, there are several HMVP tables to save the HMVP candidates, different HMVP tables can be assigned for different CTUs, or different HMVP tables can be assigned for different regions of the coded picture. For example, there are 5 HMVP tables, one of which is assigned to the current CTU, one of which is assigned to the left neighboring CTU, one of which is assigned to the top neighboring CTU, one of which is assigned to the top-right neighboring CTU, and one of which is assigned to the top-left neighboring CTU. For different HMVP tables, the HMVP table size can be set to the same number, or different numbers for different HMVP tables.

[0568] When managing IBC HMVP candidates, the coded blocks coded in both IBC mode or intra TMP mode can be added to the IBC HMVP table, or only the coded blocks coded in IBC mode can be added to the IBC HMVP table. When adding a coded block to the IBC HMVP table, coded blocks with all block sizes can be added to the IBC HMVP table, or only coded blocks with an area greater than 16 can be added to the IBC HMVP table.

[0569] Intra MPM list construction related to IBC or intra TMP coded blocks According to one or more embodiments of the present disclosure, during the construction of the Intra MPM list of a current block, when one neighboring block is IBC coded or Intra TMP coded, the Intra prediction mode pointed by the block vector of IBC or Intra TMP is used to construct the Intra MPM list of the current block. Different methods can be used to achieve this goal.

[0570] In a first method, when the current block is coded with regular Intra mode, multi-reference line Intra mode, GPM with Inter and Intra mode, spatial GPM mode, or GPM with IBC and Intra mode to construct the Intra MPM list of the current block, when one neighboring block is IBC coded or Intra TMP coded, the Intra prediction mode pointed by the block vector of IBC or Intra TMP is used, instead of setting the Intra prediction mode of the IBC or Intra TMP coded block to the planar mode.

[0571] Figure 35 A computing environment (or computing device) 1610 is shown coupled with a user interface 1650. The computing environment 1610 can be part of a data processing server. In some embodiments, the computing device 1610 can perform any of the various methods or processes (such as encoding / decoding methods or processes) as described above according to various examples of the present disclosure. The computing environment 1610 includes a processor 1620, a memory 1630, and an input / output (I / O) interface 1640.

[0572] The processor 1620 generally controls the overall operation of the computing environment 1610, such as operations associated with displaying, data acquisition, data communication, and image processing. The processor 1620 can include one or more processors to execute instructions to perform all or some of the steps in the above methods. Additionally, the processor 1620 can include one or more modules that facilitate interaction between the processor 1620 and other components. The processor can be a central processing unit (CPU), a microprocessor, a microcontroller, a graphics processing unit (GPU), etc.

[0573] The memory 1630 is configured to store various types of data to support operations of the computing environment 1610. The memory 1630 can include predetermined software 1632. Examples of such data include instructions for any application or method operating on the computing environment 1610, video data sets, image data, etc. The memory 1630 can be implemented by using any type of volatile or non-volatile storage devices, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic or optical disks.

[0574] The I / O interface 1640 provides an interface between the processor 1620 and peripheral interface modules (e.g., a keyboard, a click wheel, a button, etc.). The buttons can include, but are not limited to, a home button, a start scanning button, and a stop scanning button. The I / O interface 1640 can be coupled with an encoder and a decoder.

[0575] Figure 50 is a flowchart illustrating a video decoding method according to an example of the present disclosure.

[0576] In step 5010, the processor 1620 can obtain, on a decoder side, a current block encoded with a geometric partition mode (GPM).

[0577] In step 5020, the processor 1620 can obtain, on a decoder side, a first intra block copy (IBC) prediction and a second IBC prediction based on partitions of the current block using the GPM.

[0578] In step 5030, the processor 1620 can receive, on a decoder side, an encoding mode of the current block. In some examples, the encoding mode is related to or represented by a syntax element.

[0579] In step 5040, the processor 1620 can obtain, on a decoder side, a final prediction of the current CU using the first IBC prediction, the second IBC prediction, and the encoding mode of the current block.

[0580] In some examples, the current block includes a first GPM split partition and a second GPM split partition, and in step 5020, the processor 1620 can obtain, on a decoder side, a first IBC prediction corresponding to the first GPM split partition, and obtain a second IBC prediction corresponding to the second GPM split partition.

[0581] In one or more examples, the encoding mode of the current block includes an IBC local illumination compensation (IBC-LIC) flag of the current block or a reconstruction reordering IBC (RR-IBC) type of the current block; and in step 5030, the processor 1620 can receive, on a decoder side, the encoding mode of the current block as one of: a first encoding mode of the first GPM split partition, or a second encoding mode of the second GPM split partition, or a default value.

[0582] In one or more examples, the merge index of the current block is set to a first merge index of a first GPM partition, and in step 5030, the processor 1620 can receive, on the decoder side, the coding mode of the current block as a first coding mode of the first GPM partition. In one or more examples, the merge index of the current block is set to a second merge index of a second GPM partition, and in step 5030, the processor 1620 can receive, on the decoder side, the coding mode of the current block as a second coding mode of the second GPM partition.

[0583] In one or more examples, in step 5030, the processor 1620 can receive, on the decoder side, an IBC-LIC flag of the current block as a first constant, or receive an RR-IBC type of the current block as a second constant. In one or more examples, the first constant can be false; in one or more examples, the second constant can be 0.

[0584] Figure 51 is a flowchart illustrating a video encoding method corresponding to the video decoding method as shown in Figure 50

[0585] In step 5110, the processor 1620 can obtain, on the encoder side, a current block encoded using a geometric partition mode (GPM).

[0586] In step 5120, the processor 1620 can obtain, on the encoder side, a first intra block copy (IBC) prediction and a second IBC prediction based on partitions of the current block using the GPM.

[0587] In step 5130, the processor 1620 can set, on the encoder side, a coding mode of the current block. In some examples, the coding mode is related to or represented by a syntax element.

[0588] In step 5140, the processor 1620 can obtain, on the encoder side, a final prediction of the current CU using the first IBC prediction, the second IBC prediction, and the coding mode of the current block.

[0589] In step 5150, the processor 1620 can generate, on the encoder side, a bitstream based on the final prediction.

[0590] In some examples, the current block includes a first GPM partition and a second GPM partition, and in step 5120, the processor 1620 can obtain, on the encoder side, a first IBC prediction corresponding to the first GPM partition, and obtain a second IBC prediction corresponding to the second GPM partition.

[0591] ​In one or more examples, the coding mode of the current block includes an IBC local illumination compensation (IBC-LIC) flag of the current block or a reconstruction reordering IBC (RR-IBC) type of the current block; and in step 5130, the processor 1620 can set, at the encoder side, the coding mode of the current block to one of: the first coding mode of the first GPM partition, or the second coding mode of the second GPM partition, or a default value.

[0592] In one or more examples, the merge index of the current block is set to the first merge index of the first GPM partition, and in step 5130, the processor 1620 can set, at the encoder side, the coding mode of the current block to the first coding mode of the first GPM partition. In one or more examples, the merge index of the current block is set to the second merge index of the second GPM partition, and in step 5130, the processor 1620 can set, at the encoder side, the coding mode of the current block to the second coding mode of the second GPM partition.

[0593] In one or more examples, in step 5130, the processor 1620 can set, at the encoder side, the IBC-LIC flag of the current block to a first constant, or set the RR-IBC type of the current block to a second constant. In one or more examples, the first constant can be false; in one or more examples, the second constant can be 0.

[0594] Figure 52 is a flowchart illustrating a video decoding method according to an example of the present disclosure.

[0595] In step 5210, the processor 1620 can obtain, at the decoder side and for applying an intra block copy (IBC) merge mode or an IBC adaptive motion vector prediction (AMVP) mode to the current block, IBC candidates from non-adjacent neighboring blocks according to a scanning rule corresponding to the IBC merge mode or the IBC AMVP mode, wherein the non-adjacent neighboring blocks are at a distance of at least one block from the current block and are located in multiple directions relative to the current block, and the scanning rule is obtained based on the distance and the directions.

[0596] In step 5220, the processor 1620 can obtain, at the decoder side, a prediction of the current block by applying the IBC merge mode or the IBC AMVP mode based on the IBC candidates.

[0597] In some examples, in response to determining that the IBC merge mode is applied to the current block, the IBC candidate is an IBC merge candidate; in response to determining that the IBC AMVP mode is applied to the current block, the IBC candidate is an IBC AMVP candidate; and the scan rule comprises: determining a plurality of scan regions according to a scan distance range indicating a range of block numbers from one side of the current block, and determining a plurality of scan positions in each of the plurality of scan regions according to a plurality of directions relative to the current block, the plurality of directions being measured by a plurality of angles relative to a horizontal right direction of the current block.

[0598] In some examples, the scan distance range is from 1 to 7, or from 1 to 4, or from 1 to 3, or from 1 to 2; and the plurality of angles relative to the horizontal right direction of the current block comprises 5 angles each being a multiple of 45 degrees, or 9 angles each being a multiple of 22.5 degrees, or 13 angles each being a multiple of 22.5 degrees.

[0599] In one or more examples, in response to determining that the IBC merge mode is applied to the current block, the processor 1620 can determine a plurality of scan regions according to a scan distance range and determine a plurality of scan positions in each of the plurality of scan regions according to a plurality of directions relative to the current block, determine the scan distance range as from 1 to 7, and determine the plurality of angles as comprising 5 angles each being a multiple of 45 degrees.

[0600] In at least one example, in response to determining that the IBC AMVP mode is applied to the current block, the processor 1620 can determine a plurality of scan regions according to a scan distance range and determine a plurality of scan positions in each of the plurality of scan regions according to a plurality of directions relative to the current block, determine the scan distance range as from 1 to 4, and determine the plurality of angles as comprising 5 angles each being a multiple of 45 degrees.

[0601] In at least one example, in response to determining that the IBC AMVP mode is applied to the current block, the processor 1620 can determine a plurality of scan regions according to a scan distance range and determine a plurality of scan positions in each of the plurality of scan regions according to a plurality of directions relative to the current block, determine the scan distance range as from 1 to 3, and determine the plurality of angles as comprising 5 angles each being a multiple of 45 degrees.

[0602] In at least one example, in response to determining that the IBC AMVP mode is applied to the current block, the processor 1620 can determine a plurality of scan regions according to a scan distance range and a plurality of directions relative to the current block, determine the scan distance range to be from 1 to 2, and determine the plurality of angles to include 5 angles, each of which is a multiple of 45 degrees.

[0603] In at least one example, in response to determining that the IBC AMVP mode is applied to the current block, the processor 1620 can determine a plurality of scan regions according to a scan distance range and a plurality of directions relative to the current block, determine the scan distance range to be from 1 to 7, and determine the plurality of angles to include 5 angles, each of which is a multiple of 45 degrees.

[0604] In one or more examples, in response to determining that the IBC AMVP mode is applied to the current block, the processor 1620 can determine a plurality of scan regions according to a scan distance range and a plurality of directions relative to the current block, determine the scan distance range to be from 1 to 4, and determine the plurality of angles to include 5 angles, each of which is a multiple of 45 degrees; wherein, in response to determining that the IBC AMVP mode is applied to the current block and determining that an angle of a scan region in a horizontal right direction relative to the current block is 90 degrees or 180 degrees, the scan rule further comprises: determining that a minimum scan distance of the scan region is 2.

[0605] In at least one example, in response to determining that the IBC merge mode is applied to the current block, the processor 1620 can determine a plurality of scan regions according to a scan distance range and a plurality of directions relative to the current block, determine the scan distance range to be from 1 to 7, and determine the plurality of angles to include 5 angles, each of which is a multiple of 45 degrees; wherein, in response to determining that the IBC merge mode is applied to the current block and determining that an angle of a scan region in a horizontal right direction relative to the current block is 90 degrees or 180 degrees, the scan rule further comprises: determining that a minimum scan distance of the scan region is 2.

[0606] In at least one example, in response to determining that the IBC merge mode is applied to the current block, the processor 1620 can determine a plurality of scan regions according to a scan distance range and a plurality of directions relative to the current block, determine the scan distance range to be from 1 to 7, and determine the plurality of angles to include 9 angles, each of which is a multiple of 22.5 degrees.

[0607] In at least one example, in response to determining to apply the IBC merge mode to the current block, the processor 1620 can determine a plurality of scanning regions according to a scanning distance range and a plurality of scanning positions in each of the plurality of scanning regions according to a plurality of directions relative to the current block, determine the scanning distance range as from 1 to 7, and determine the plurality of angles to include 13 angles each being a multiple of 22.5 degrees.

[0608] Figure 53 is a flowchart illustrating a video encoding method corresponding to the video decoding method as shown in Figure 52

[0609] In step 5310, the processor 1620 can obtain, at an encoder side and for applying an intra block copy (IBC) merge mode or an IBC adaptive motion vector prediction (AMVP) mode to a current block, IBC candidates from non-adjacent neighboring blocks according to a scanning rule corresponding to the IBC merge mode or the IBC AMVP mode, wherein the non-adjacent neighboring blocks are at a distance of at least one block from the current block and are located in a plurality of directions relative to the current block, and the scanning rule is obtained based on the distance and the directions.

[0610] In step 5320, the processor 1620 can obtain, at the encoder side, a prediction of the current block by applying the IBC merge mode or the IBC AMVP mode based on the IBC candidates.

[0611] In step 5330, the processor 1620 can generate, at the encoder side, a bitstream based on the prediction.

[0612] In some examples, in response to determining to apply the IBC merge mode to the current block, the IBC candidates are IBC merge candidates; in response to determining to apply the IBC AMVP mode to the current block, the IBC candidates are IBC AMVP candidates; and the scanning rule includes determining a plurality of scanning regions according to a scanning distance range indicating a range of block numbers from a side of the current block, and determining a plurality of scanning positions in each of the plurality of scanning regions according to a plurality of directions relative to the current block, the plurality of directions being measured by a plurality of angles relative to a horizontal right direction of the current block.

[0613] In some examples, the scanning distance range is from 1 to 7, or from 1 to 4, or from 1 to 3, or from 1 to 2; and the plurality of angles relative to the horizontal right direction of the current block includes 5 angles each being a multiple of 45 degrees, or 9 angles each being a multiple of 22.5 degrees, or 13 angles each being a multiple of 22.5 degrees.

[0614] ​In one or more examples, in response to determining that the IBC merge mode is applied to the current block, the processor 1620 can determine a plurality of scan regions according to a scan distance range and a plurality of directions relative to the current block, determine a plurality of scan positions in each of the plurality of scan regions according to the plurality of directions, determine the scan distance range to be from 1 to 7, and determine the plurality of angles to include 5 angles each of which is a multiple of 45 degrees.

[0615] In at least one example, in response to determining th...

Claims

1. A video decoding method, comprising: The decoder obtains the current block encoded using Geometric Partitioning Mode (GPM); The decoder obtains a first intra-block copy (IBC) prediction and a second IBC prediction based on the partitioning of the current block using GPM. The decoder receives the encoding mode of the current block; as well as The decoder uses the first IBC prediction, the second IBC prediction, and the encoding mode of the current block to obtain the final prediction of the current block.

2. The method as described in claim 1, wherein, The current block includes a first GPM partition and a second GPM partition; and The process by which the decoder obtains the first IBC prediction and the second IBC prediction based on the partitioning of the current block using the GPM includes: Obtain the first IBC prediction corresponding to the first GPM segmentation partition, and obtain the second IBC prediction corresponding to the second GPM segmentation partition.

3. The method as described in claim 2, wherein, The encoding mode of the current block includes the IBC Local Illumination Compensation (IBC-LIC) flag of the current block or the Reconstruction Reordering IBC (RR-IBC) type of the current block; and The encoding mode received by the decoder for the current block includes: The encoding mode of the current block is received as one of the following: the first encoding mode of the first GPM partition, or the second encoding mode of the second GPM partition, or the default value.

4. The method of claim 3, wherein, The merge index of the current block is set to the first merge index of the first GPM partition; and The encoding mode received by the decoder for the current block includes: The encoding mode of the current block is received as the first encoding mode of the first GPM segmentation partition.

5. The method of claim 3, wherein, The merge index of the current block is set to the second merge index of the second GPM partition; and The encoding mode received by the decoder for the current block includes: The encoding mode of the current block is received as the second encoding mode of the second GPM segmentation partition.

6. The method of claim 3, wherein, The encoding mode received by the decoder for the current block includes: Receive the IBC-LIC flag of the current block as a first constant, or receive the RR-IBC type of the current block as a second constant.

7. A video encoding method, comprising: The encoder obtains the current block encoded using Geometric Partitioning Mode (GPM); The encoder obtains a first intra-block copy (IBC) prediction and a second IBC prediction based on the partitioning of the current block using GPM. The encoder sets the encoding mode for the current block; The encoder obtains the final prediction of the current block using the first IBC prediction, the second IBC prediction, and the encoding mode of the current block; as well as The encoder generates a bitstream based on the final prediction.

8. The method of claim 7, wherein, The current block includes a first GPM partition and a second GPM partition; and The process by which the encoder obtains the first IBC prediction and the second IBC prediction based on the partitioning of the current block using the GPM includes: Obtain the first IBC prediction corresponding to the first GPM segmentation partition, and obtain the second IBC prediction corresponding to the second GPM segmentation partition.

9. The method of claim 8, wherein, The encoding mode of the current block includes either the IBC Local Illumination Compensation (IBC-LIC) flag of the current block or the Reconstruction Reordering IBC (RR-IBC) type of the current block; and The encoding mode of the current block set by the encoder includes: Set the encoding mode of the current block to one of the following: the first encoding mode of the first GPM partition, or the second encoding mode of the second GPM partition, or the default value.

10. The method of claim 9, wherein, The merge index of the current block is set to the first merge index of the first GPM partition; and The encoding mode of the current block set by the encoder includes: Set the encoding mode of the current block to the first encoding mode of the first GPM partition.

11. The method of claim 9, wherein, The merge index of the current block is set to the second merge index of the second GPM partition; and The encoding mode of the current block set by the encoder includes: Set the encoding mode of the current block to the second encoding mode of the second GPM partition.

12. The method of claim 9, wherein, Setting the encoding mode of the current block by the encoder includes: Set the IBC-LIC flag of the current block to a first constant, or set the RR-IBC type of the current block to a second constant.

13. A video decoding method, comprising: To apply Intra-Block Copy (IBC) merging mode or IBC Adaptive Motion Vector Prediction (AMVP) mode to the current block, the decoder obtains IBC candidates from non-adjacent neighbor blocks according to a scanning rule corresponding to the IBC merging mode or the IBC AMVP mode. These non-adjacent neighbor blocks are at least one block's distance from the current block and are located in multiple directions relative to the current block, and the scanning rule is obtained based on the distance and the direction. The decoder obtains the prediction of the current block by applying the IBC merging mode or the IBC AMVP mode based on the IBC candidate.

14. The method of claim 13, wherein, In response to determining that the IBC merge mode will be applied to the current block, the IBC candidate is an IBC merge candidate; Wherein, in response to determining that the IBC AMVP mode should be applied to the current block, the IBC candidate is an IBC AMVP candidate; and The scanning rules include: Multiple scanning regions are determined based on a scanning distance range indicating a range of blocks from one side of the current block; and Multiple scan positions in each of the multiple scan regions are determined based on the multiple directions relative to the current block, the multiple directions being measured by multiple angles relative to the horizontal right direction of the current block.

15. The method of claim 14, wherein, The scanning distance range is from 1 to 7, or from 1 to 4, or from 1 to 3, or from 1 to 2; and The plurality of angles relative to the current block in the horizontal right direction include 5 angles, each of which is a multiple of 45 degrees, or 9 angles, each of which is a multiple of 22.5 degrees, or 13 angles, each of which is a multiple of 22.5 degrees.

16. The method of claim 15, wherein, In response to determining that the IBC merging mode should be applied to the current block, determining the plurality of scan regions based on the scan distance range and determining the plurality of scan positions in each of the plurality of scan regions based on the plurality of directions relative to the current block includes: The scanning distance range is defined as 1 to 7, and the plurality of angles are defined as including 5 angles, each of which is a multiple of 45 degrees.

17. The method of claim 16, wherein, In response to determining that the IBC AMVP mode should be applied to the current block, determining the plurality of scan regions based on the scan distance range and determining the plurality of scan positions in each of the plurality of scan regions based on the plurality of directions relative to the current block includes: The scanning distance range is defined as 1 to 4, and the plurality of angles are defined as including 5 angles, each of which is a multiple of 45 degrees.

18. The method of claim 15, wherein, In response to determining that the IBC AMVP mode should be applied to the current block, determining the plurality of scan regions based on the scan distance range and determining the plurality of scan positions in each of the plurality of scan regions based on the plurality of directions relative to the current block includes: The scanning distance range is defined as 1 to 4, and the plurality of angles is defined as including 5 angles, each angle being a multiple of 45 degrees; and Wherein, in response to determining that the IBC AMVP mode is applied to the current block and determining that the angle of the horizontal right direction relative to the current block in the scan region is 90 degrees or 180 degrees, the scan rule further includes: The minimum scanning distance for the scanning area is determined to be 2.

19. The method of claim 18, wherein, In response to determining that the IBC merging mode should be applied to the current block, determining the plurality of scan regions based on the scan distance range and determining the plurality of scan positions in each of the plurality of scan regions based on the plurality of directions relative to the current block includes: The scanning distance range is defined as 1 to 7, and the plurality of angles are defined as including 5 angles, each angle being a multiple of 45 degrees; and Wherein, in response to determining that the IBC merging mode is applied to the current block and determining that the angle in the horizontal right direction relative to the current block in the scan region is 90 degrees or 180 degrees, the scan rule further includes: The minimum scanning distance for the scanning area is determined to be 2.

20. The method of claim 16, wherein, In response to determining that the IBC AMVP mode should be applied to the current block, determining the plurality of scan regions based on the scan distance range and determining the plurality of scan positions in each of the plurality of scan regions based on the plurality of directions relative to the current block includes: The scanning distance range is defined as 1 to 3, and the plurality of angles are defined as including 5 angles, each of which is a multiple of 45 degrees.

21. The method of claim 16, wherein, In response to determining that the IBC AMVP mode should be applied to the current block, determining the plurality of scan regions based on the scan distance range and determining the plurality of scan positions in each of the plurality of scan regions based on the plurality of directions relative to the current block includes: The scanning distance range is defined as 1 to 2, and the plurality of angles are defined as including 5 angles, each of which is a multiple of 45 degrees.

22. The method of claim 18, wherein, In response to determining that the IBC merging mode should be applied to the current block, determining the plurality of scan regions based on the scan distance range and determining the plurality of scan positions in each of the plurality of scan regions based on the plurality of directions relative to the current block includes: The scanning distance range is defined as 1 to 7, and the plurality of angles is defined as including 9 angles, each angle being a multiple of 22.5 degrees.

23. The method of claim 18, wherein, In response to determining that the IBC merging mode should be applied to the current block, determining the plurality of scan regions based on the scan distance range and determining the plurality of scan positions in each of the plurality of scan regions based on the plurality of directions relative to the current block includes: The scanning distance range is defined as 1 to 7, and the plurality of angles are defined as including 13 angles, each of which is a multiple of 22.5 degrees.

24. The method of claim 16, wherein, In response to determining that the IBC AMVP mode should be applied to the current block, determining the plurality of scan regions based on the scan distance range and determining the plurality of scan positions in each of the plurality of scan regions based on the plurality of directions relative to the current block includes: The scanning distance range is defined as 1 to 7, and the plurality of angles are defined as including 5 angles, each of which is a multiple of 45 degrees.

25. A video coding method, comprising: In order to apply the Intra-Block Copy (IBC) merging mode or the IBC Adaptive Motion Vector Prediction (AMVP) mode to the current block, the encoder obtains IBC candidates from non-adjacent neighboring blocks according to the scanning rules corresponding to the IBC merging mode or the IBC AMVP mode, wherein the non-adjacent neighboring blocks are at least one block away from the current block and are located in multiple directions relative to the current block, and the scanning rules are obtained based on the distance and the direction; The encoder obtains the prediction of the current block by applying the IBC merging mode or the IBC AMVP mode based on the IBC candidates; and The encoder generates a bitstream based on the prediction.

26. The method of claim 25, wherein, In response to determining that the IBC merge mode will be applied to the current block, the IBC candidate is an IBC merge candidate; Wherein, in response to determining that the IBC AMVP mode should be applied to the current block, the IBC candidate is an IBC AMVP candidate; and The scanning rules include: Multiple scanning regions are determined based on a scanning distance range indicating a range of blocks from one side of the current block; and Multiple scan positions in each of the multiple scan regions are determined based on the multiple directions relative to the current block, the multiple directions being measured by multiple angles relative to the horizontal right direction of the current block.

27. The method of claim 26, wherein, The scanning distance range is from 1 to 7, or from 1 to 4, or from 1 to 3, or from 1 to 2; and The plurality of angles relative to the current block in the horizontal right direction include 5 angles, each of which is a multiple of 45 degrees, or 9 angles, each of which is a multiple of 22.5 degrees, or 13 angles, each of which is a multiple of 22.5 degrees.

28. The method of claim 27, wherein, In response to determining that the IBC merging mode should be applied to the current block, determining the plurality of scan regions based on the scan distance range and determining the plurality of scan positions in each of the plurality of scan regions based on the plurality of directions relative to the current block includes: The scanning distance range is defined as 1 to 7, and the plurality of angles are defined as including 5 angles, each of which is a multiple of 45 degrees.

29. The method of claim 28, wherein, In response to determining that the IBC AMVP mode should be applied to the current block, determining the plurality of scan regions based on the scan distance range and determining the plurality of scan positions in each of the plurality of scan regions based on the plurality of directions relative to the current block includes: The scanning distance range is defined as 1 to 4, and the plurality of angles are defined as including 5 angles, each of which is a multiple of 45 degrees.

30. The method of claim 27, wherein, In response to determining that the IBC AMVP mode should be applied to the current block, determining the plurality of scan regions based on the scan distance range and determining the plurality of scan positions in each of the plurality of scan regions based on the plurality of directions relative to the current block includes: The scanning distance range is defined as 1 to 4, and the plurality of angles is defined as including 5 angles, each angle being a multiple of 45 degrees; and Wherein, in response to determining that the IBC AMVP mode is applied to the current block and determining that the angle of the horizontal right direction relative to the current block in the scan region is 90 degrees or 180 degrees, the scan rule further includes: The minimum scanning distance for the scanning area is determined to be 2.

31. The method of claim 30, wherein, In response to determining that the IBC merging mode should be applied to the current block, determining the plurality of scan regions based on the scan distance range and determining the plurality of scan positions in each of the plurality of scan regions based on the plurality of directions relative to the current block includes: The scanning distance range is defined as 1 to 7, and the plurality of angles are defined as including 5 angles, each angle being a multiple of 45 degrees; and Wherein, in response to determining that the IBC merging mode is applied to the current block and determining that the angle in the horizontal right direction relative to the current block in the scan region is 90 degrees or 180 degrees, the scan rule further includes: The minimum scanning distance for the scanning area is determined to be 2.

32. The method of claim 28, wherein, In response to determining that the IBC AMVP mode should be applied to the current block, determining the plurality of scan regions based on the scan distance range and determining the plurality of scan positions in each of the plurality of scan regions based on the plurality of directions relative to the current block includes: The scanning distance range is defined as 1 to 3, and the plurality of angles are defined as including 5 angles, each of which is a multiple of 45 degrees.

33. The method of claim 28, wherein, In response to determining that the IBC AMVP mode should be applied to the current block, determining the plurality of scan regions based on the scan distance range and determining the plurality of scan positions in each of the plurality of scan regions based on the plurality of directions relative to the current block includes: The scanning distance range is defined as 1 to 2, and the plurality of angles are defined as including 5 angles, each of which is a multiple of 45 degrees.

34. The method of claim 30, wherein, In response to determining that the IBC merging mode should be applied to the current block, determining the plurality of scan regions based on the scan distance range and determining the plurality of scan positions in each of the plurality of scan regions based on the plurality of directions relative to the current block includes: The scanning distance range is defined as 1 to 7, and the plurality of angles is defined as including 9 angles, each angle being a multiple of 22.5 degrees.

35. The method of claim 30, wherein, In response to determining that the IBC merging mode should be applied to the current block, determining the plurality of scan regions based on the scan distance range and determining the plurality of scan positions in each of the plurality of scan regions based on the plurality of directions relative to the current block includes: The scanning distance range is defined as 1 to 7, and the plurality of angles are defined as including 13 angles, each of which is a multiple of 22.5 degrees.

36. The method of claim 28, wherein, In response to determining that the IBC AMVP mode should be applied to the current block, determining the plurality of scan regions based on the scan distance range and determining the plurality of scan positions in each of the plurality of scan regions based on the plurality of directions relative to the current block includes: The scanning distance range is defined as 1 to 7, and the plurality of angles are defined as including 5 angles, each of which is a multiple of 45 degrees.

37. A video decoding method, comprising: In order to apply the Intra-Block Copy (IBC) merging mode or the IBC Adaptive Motion Vector Prediction (AMVP) mode to the current block, the decoder obtains an ordered list of IBC candidates. The decoder inserts spatially non-adjacent candidates into an ordered list of IBC candidates after spatially adjacent candidates or history-based block vector prediction (BVP) candidates; as well as The decoder obtains the prediction of the current block by applying the IBC merging mode or the IBC AMVP mode based on the ordered list of IBC candidates.

38. The method of claim 37, wherein, In response to determining that the IBC merge mode should be applied to the current block, the spatially non-adjacent candidates include spatially non-adjacent IBC merge candidates; and The process by which the decoder inserts the spatially non-adjacent candidates into the ordered list of IBC candidates after the spatially adjacent candidates or the history-based BVP candidates includes: The spatially non-adjacent IBC merging candidates are inserted into the ordered list of IBC candidates after the spatially adjacent IBC merging candidates.

39. The method of claim 37, wherein, In response to determining that the IBC AMVP mode should be applied to the current block, the spatially non-adjacent candidates include spatially non-adjacent IBC AMVP candidates; and The process by which the decoder inserts the spatially non-adjacent candidates into the ordered list of IBC candidates after the spatially adjacent candidates or the history-based BVP candidates includes: The spatially non-adjacent IBC AMVP candidates are inserted into the ordered list of IBC candidates after the history-based BVP candidates.

40. The method of claim 37, wherein, The spatially non-adjacent candidates include one or more spatial BVPs from spatially non-adjacent neighboring blocks, and the spatially adjacent candidates include one or more spatial BVPs from spatially adjacent neighboring blocks.

41. The method of claim 40, wherein, The decoder inserts the spatially non-adjacent candidates into the ordered list of IBC candidates after the spatially adjacent candidates or the history-based BVP candidates, including performing the following actions in sequence: Insert one or more spatial BVPs from the spatially adjacent neighbor blocks into the ordered list of IBC candidates; The history-based BVP candidates from the first-in-first-out (FIFO) table are inserted into the ordered list of IBC candidates; as well as Insert one or more spatial BVPs from the spatial non-adjacent neighboring blocks into the ordered list of IBC candidates.

42. The method of claim 41, wherein, After inserting the one or more spatial BVPs from the spatially non-adjacent neighbor blocks into the ordered list of IBC candidates, the method further includes performing the following actions in sequence: Insert one or more pairwise average BVPs into the ordered list of IBC candidates; Insert the BVP candidate located in the IBC reference region into the ordered list of IBC candidates; and Insert zero BVP into the ordered list of IBC candidates.

43. A video encoding method, comprising: In order to apply the Intra-Block Copy (IBC) merging mode or the IBC Adaptive Motion Vector Prediction (AMVP) mode to the current block, the encoder obtains an ordered list of IBC candidates. The encoder inserts spatially non-adjacent candidates into the ordered list of IBC candidates after spatially adjacent candidates or history-based block vector prediction (BVP) candidates; The encoder obtains the prediction of the current block by applying the IBC merging mode or the IBCAMVP mode based on the ordered list of IBC candidates; as well as The encoder generates a bitstream based on the prediction.

44. The method of claim 43, wherein, In response to determining that the IBC merge mode should be applied to the current block, the spatially non-adjacent candidates include spatially non-adjacent IBC merge candidates; and The process by which the encoder inserts the spatially non-adjacent candidates into the ordered list of IBC candidates after the spatially adjacent candidates or the history-based BVP candidates includes: The spatially non-adjacent IBC merging candidates are inserted into the ordered list of IBC candidates after the spatially adjacent IBC merging candidates.

45. The method of claim 43, wherein, In response to determining that the IBC AMVP mode should be applied to the current block, the spatially non-adjacent candidates include spatially non-adjacent IBC AMVP candidates; and The process by which the encoder inserts the spatially non-adjacent candidates into the ordered list of IBC candidates after the spatially adjacent candidates or the history-based BVP candidates includes: The spatially non-adjacent IBC AMVP candidates are inserted into the ordered list of IBC candidates after the history-based BVP candidates.

46. ​​The method of claim 43, wherein, The spatially non-adjacent candidates include one or more spatial BVPs from spatially non-adjacent neighboring blocks, and the spatially adjacent candidates include one or more spatial BVPs from spatially adjacent neighboring blocks.

47. The method of claim 46, wherein, The encoder inserts the spatially non-adjacent candidates into the ordered list of IBC candidates after the spatially adjacent candidates or the history-based BVP candidates, including performing the following actions in sequence: Insert one or more spatial BVPs from the spatially adjacent neighbor blocks into the ordered list of IBC candidates; The history-based BVP candidates from the first-in-first-out (FIFO) table are inserted into the ordered list of IBC candidates; as well as Insert one or more spatial BVPs from the spatial non-adjacent neighboring blocks into the ordered list of IBC candidates.

48. The method of claim 47, wherein, After inserting the one or more spatial BVPs from the spatially non-adjacent neighbor blocks into the ordered list of IBC candidates, the method further includes performing the following actions in sequence: Insert one or more pairwise average BVPs into the ordered list of IBC candidates; Insert the BVP candidate located in the IBC reference region into the ordered list of IBC candidates; and Insert zero BVP into the ordered list of IBC candidates.

49. An apparatus for video decoding, the apparatus comprising: One or more processors; as well as A memory coupled to the one or more processors and configured to store instructions executable by the one or more processors. The one or more processors are configured to perform the method as described in any one of claims 1 to 6, 13 to 24, or 37 to 42 when executing the instructions.

50. An apparatus for video encoding, the apparatus comprising: One or more processors; as well as A memory coupled to the one or more processors and configured to store instructions executable by the one or more processors. The one or more processors are configured to perform the method as described in any one of claims 7 to 12, 25 to 36, or 43 to 48 when executing the instructions.

51. A non-transitory computer-readable storage medium for storing computer-executable instructions, which, when executed by one or more computer processors, cause the one or more computer processors to perform the method as described in any one of claims 1 to 6, 13 to 24, or 37 to 42.

52. A non-transitory computer-readable storage medium for storing computer-executable instructions, which, when executed by one or more computer processors, cause the one or more computer processors to perform the method as described in any one of claims 7 to 12, 25 to 36, or 43 to 48.

53. A non-transitory computer-readable storage medium for storing a bit stream to be decoded by any one of claims 1 to 6, 13 to 24 or 37 to 42.

54. A non-transitory computer-readable storage medium for storing a bit stream generated by the method of any one of claims 7 to 12, 25 to 36 or 43 to 48.

55. A method for storing a bitstream, the method comprising storing the bitstream on a non-transitory computer-readable storage medium, wherein, The bitstream includes encoded video information generated by the method described in any one of claims 7 to 12, 25 to 36, or 43 to 48.

56. A method for storing a bitstream, the method comprising storing the bitstream on a non-transitory computer-readable storage medium, wherein, The bitstream includes encoded video information to be decoded by any one of claims 1 to 6, 13 to 24, or 37 to 42.

57. A method for transmitting a bit stream, the method comprising transmitting a bit stream generated by any one of claims 7 to 12, 25 to 36 or 43 to 48.

58. A method for receiving a bit stream, the method comprising receiving a bit stream to be decoded by any one of claims 1 to 6, 13 to 24 or 37 to 42.

Images