Electronic device and non-transitory machine readable medium for encoding or decoding video data

Abstract

An electronic device for decoding video data is provided. When executed by at least one processor of the electronic device, the one or more computer executable instructions cause the electronic device to: receive video data; determining a block unit and a vector reference block of the block unit from a current frame in the video data; deriving a filter model of the block unit; determining a first template matching cost; determining a first arrangement of the filter model based on the first template matching cost; and reconstructing the block unit based on the first arrangement. Each of the vector reference blocks is indicated by a corresponding one of the block vector candidates of the block unit. Each of the filter models is derived based on a corresponding one of the vector reference blocks. Each of the first template matching costs is calculated using a corresponding one of the filter models of the block unit.

Description

Electronic devices and non-transitory machine-readable media used for encoding or decoding video data Technical Field

[0001] This disclosure relates generally to video coding, and more specifically, to techniques for predicting and / or reconstructing block cells by arranging multiple filter models based on multiple reference blocks derived from multiple block vector candidate indications of block cells.

[0002] This disclosure claims the benefit and priority of U.S. Provisional Patent Application Serial No. 63 / 587,828, filed October 4, 2023, entitled “Reconstruction-reordered IntraBlock Copy with Local Illumination Compensation,” and U.S. Provisional Patent Application Serial No. 63 / 619,593, filed January 10, 2024, entitled “Intra Block Copy Merge Mode with Filtered Intra Block Copy,” the entire contents of which are incorporated herein by reference for all purposes. Background Technology

[0003] Filtered Intra-Block Copy (FIBC) mode is an encoding tool used for video coding, in which the encoder and / or decoder can predict each block sample among multiple block samples in the current block by using a filter model. Additionally, the encoder and / or decoder can derive the filter model for the current block based on the block template of the current block and the reference template of a reference block indicated by the intra-block vector of the current block. However, in FIBC mode, the encoder needs to provide the decoder with some prediction syntax to indicate the prediction information for the current block.

[0004] Therefore, encoders and / or decoders may require efficient operating mechanisms for FIBC mode to reduce the number of bits in the bitstream. Summary of the Invention

[0005] This disclosure relates to an apparatus and method for predicting and / or reconstructing block cells by arranging multiple filter models derived from multiple reference blocks indicated by multiple block vector candidates of the block cells.

[0006] In a first aspect of this disclosure, an electronic device for decoding video data is provided. The electronic device includes: at least one processor; and one or more non-transitory computer-readable media coupled to the at least one processor and storing one or more computer-executable instructions, which, when executed by the at least one processor, cause the electronic device to: receive the video data; determine a block unit from a current frame included in the video data; determine a plurality of vector reference blocks of the block unit from the current frame, each of the plurality of vector reference blocks being indicated by a corresponding block vector candidate from a plurality of block vector candidates of the block unit; derive a plurality of filter models of the block unit, each of the plurality of filter models being derived based on a corresponding vector reference block from the plurality of vector reference blocks; determine a plurality of first template matching costs, each of the plurality of first template matching costs being computed using a corresponding filter model from the plurality of filter models of the block unit; determine a first arrangement of the plurality of filter models based on the plurality of first template matching costs; and reconstruct the block unit based on the first arrangement of the plurality of filter models.

[0007] In a specific embodiment of the first aspect of this disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a plurality of adjacent blocks adjacent to the block unit; determine a plurality of adjacent block vectors, each of the plurality of adjacent block vectors indicating a corresponding adjacent reference block among a plurality of adjacent reference blocks in the current frame, for reconstructing the corresponding adjacent block among the plurality of adjacent blocks; and determine the plurality of block vector candidates based on the plurality of adjacent block vectors.

[0008] In another specific embodiment of the first aspect of this disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a first block template region and a plurality of first reference template regions adjacent to the block unit, each of the plurality of first reference template regions being adjacent to a corresponding vector reference block among the plurality of vector reference blocks; and determine a second block template region and a plurality of second reference template regions adjacent to the block unit, each of the plurality of second reference template regions being adjacent to a corresponding vector reference block among the plurality of vector reference blocks, wherein: each of the plurality of filter models is further derived based on the first block template region and a corresponding first reference template region among the plurality of first reference template regions, and each of the plurality of first template matching costs is further calculated using the corresponding filter model among the plurality of filter models based on the second template region and a corresponding second reference template region among the plurality of second reference template regions.

[0009] In another specific embodiment of the first aspect of this disclosure, the first template area is the same as the second template, and each of the plurality of first reference template areas is the same as a corresponding second template area among the plurality of second template areas.

[0010] In another specific embodiment of the first aspect of this disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a block-vector-based candidate list comprising a plurality of block-vector-based prediction candidates, wherein: each of the block-vector-based prediction candidates corresponds to a corresponding block-vector candidate among the plurality of block-vector candidates and a corresponding filter model among the plurality of filter models, the plurality of block-vector-based prediction candidates in the block-vector-based candidate list are ordered based on the first arrangement of the plurality of filter models, and the block cell is reconstructed based on the plurality of block-vector-based prediction candidates ordered in the block-vector-based candidate list.

[0011] In another specific embodiment of the first aspect of this disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a plurality of second template matching costs, each of the plurality of second template matching costs being computed directly based on a corresponding block vector candidate among the plurality of block vector candidates, without using the plurality of filter models of the block unit; and determine a second arrangement of a plurality of block vector-based prediction candidates based on the plurality of first template matching costs and the plurality of second template matching costs, wherein reconstructing the block unit is also based on the second arrangement of the plurality of block vector-based prediction candidates.

[0012] In another specific embodiment of the first aspect of this disclosure, the plurality of block vector-based prediction candidates includes at least one of a plurality of first block vector-based candidates or a plurality of second block vector-based candidates. Each of the plurality of first block vector-based candidates corresponds to a corresponding block vector candidate among the plurality of block vector candidates and a corresponding filter model among the plurality of filter models. Furthermore, each of the plurality of second block vector-based candidates corresponds only to a corresponding block vector candidate among the plurality of block vector candidates, without using the plurality of filter models.

[0013] In another specific embodiment of the first aspect of this disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine, based on the plurality of first template matching costs, a filter flag indicating whether to derive the plurality of filter models for determining the first arrangement of the plurality of filter models.

[0014] In a second aspect of this disclosure, a non-transitory machine-readable medium for an electronic device is provided, the non-transitory machine-readable medium storing one or more computer-executable instructions for decoding video data. When executed by at least one processor of the electronic device, the one or more computer-executable instructions cause the electronic device to: receive the video data; determine a block unit from a current frame included in the video data; determine a plurality of vector reference blocks of the block unit from the current frame, each of the plurality of vector reference blocks being indicated by a corresponding block vector candidate from a plurality of block vector candidates of the block unit; derive a plurality of filter models of the block unit, each of the plurality of filter models being derived based on a corresponding vector reference block from the plurality of vector reference blocks; determine a plurality of first template matching costs, each of the plurality of first template matching costs being computed using a corresponding filter model from the plurality of filter models of the block unit; determine a first arrangement of the plurality of filter models based on the plurality of first template matching costs; and reconstruct the block unit based on the first arrangement of the plurality of filter models.

[0015] In a specific embodiment of the second aspect of this disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a plurality of adjacent blocks adjacent to the block unit; determine a plurality of adjacent block vectors, each of the plurality of adjacent block vectors indicating a corresponding adjacent reference block among a plurality of adjacent reference blocks in the current frame, for reconstructing the corresponding adjacent block among the plurality of adjacent blocks; and determine the plurality of block vector candidates based on the plurality of adjacent block vectors.

[0016] In another specific embodiment of the second aspect of this disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a first block template region and a plurality of first reference template regions adjacent to the block unit, each of the plurality of first reference template regions being adjacent to a corresponding vector reference block among the plurality of vector reference blocks; and determine a second template region and a plurality of second reference template regions adjacent to the block unit, each of the plurality of second reference template regions being adjacent to a corresponding vector reference block among the plurality of vector reference blocks, wherein: each of the plurality of filter models is further derived based on the first block template region and a corresponding first reference template region among the plurality of first reference template regions, and each of the plurality of first template matching costs is further calculated using the corresponding filter model among the plurality of filter models based on the second template region and a corresponding second reference template region among the plurality of second reference template regions.

[0017] In another specific embodiment of the second aspect of this disclosure, the first template area is the same as the second template, and each of the plurality of first reference template areas is the same as a corresponding second template area among the plurality of second template areas.

[0018] In another specific embodiment of the second aspect of this disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a block-vector-based candidate list comprising a plurality of block-vector-based prediction candidates, wherein: each of the block-vector-based prediction candidates corresponds to a corresponding block-vector candidate among the plurality of block-vector candidates and a corresponding filter model among the plurality of filter models, the plurality of block-vector-based prediction candidates in the block-vector-based candidate list are ordered based on the first arrangement of the plurality of filter models, and the block cell is reconstructed based on the plurality of block-vector-based prediction candidates ordered in the block-vector-based candidate list.

[0019] In another specific embodiment of the second aspect of this disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a plurality of second template matching costs, each of the plurality of second template matching costs being computed directly based on a corresponding block vector candidate among the plurality of block vector candidates, without using the plurality of filter models of the block unit; and determine a second arrangement of a plurality of block vector-based prediction candidates based on the plurality of first template matching costs and the plurality of second template matching costs, wherein reconstructing the block unit is also based on the second arrangement of the plurality of block vector-based prediction candidates.

[0020] In another specific embodiment of the second aspect of this disclosure, the plurality of block vector-based prediction candidates includes at least one of a plurality of first block vector-based candidates or a plurality of second block vector-based candidates. Each of the plurality of first block vector-based candidates corresponds to a corresponding block vector candidate among the plurality of block vector candidates and a corresponding filter model among the plurality of filter models. Furthermore, each of the plurality of second block vector-based candidates corresponds only to a corresponding block vector candidate among the plurality of block vector candidates, without using the plurality of filter models.

[0021] In another specific embodiment of the second aspect of this disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine, based on the plurality of first template matching costs, a filter flag indicating whether to derive the plurality of filter models for determining the first arrangement of the plurality of filter models.

[0022] In a third aspect of this disclosure, an electronic device for encoding video data is provided. The electronic device includes: at least one processor; and one or more non-transitory computer-readable media coupled to the at least one processor and storing one or more computer-executable instructions, which, when executed by the at least one processor, cause the electronic device to: receive the video data; determine a block unit from a current frame included in the video data; determine a plurality of vector reference blocks of the block unit from the current frame, each of the plurality of vector reference blocks being indicated by a corresponding block vector candidate from a plurality of block vector candidates of the block unit; derive a plurality of filter models of the block unit, each of the plurality of filter models being derived based on a corresponding vector reference block from the plurality of vector reference blocks; determine a plurality of first template matching costs, each of the plurality of first template matching costs being computed using a corresponding filter model from the plurality of filter models of the block unit; determine a first arrangement of the plurality of filter models based on the plurality of first template matching costs; and reconstruct the block unit based on the first arrangement of the plurality of filter models.

[0023] In a specific embodiment of the third aspect of this disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a plurality of adjacent blocks adjacent to the block cell; determine a plurality of adjacent block vectors, each of the plurality of adjacent block vectors indicating a corresponding adjacent reference block among a plurality of adjacent reference blocks in the current frame, for reconstructing the corresponding adjacent block among the plurality of adjacent blocks; and determine the plurality of block vector candidates based on the plurality of adjacent block vectors.

[0024] In another specific embodiment of the third aspect of this disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a first block template region and a plurality of first reference template regions adjacent to the block cell, each of the plurality of first reference template regions being adjacent to a corresponding vector reference block among the plurality of vector reference blocks; and determine a second block template region and a plurality of second reference template regions adjacent to the block cell, each of the plurality of second reference template regions being adjacent to a corresponding vector reference block among the plurality of vector reference blocks, wherein: each of the plurality of filter models is further derived based on the first block template region and a corresponding first reference template region among the plurality of first reference template regions, and each of the plurality of first template matching costs is further calculated using the corresponding filter model among the plurality of filter models based on the second template region and a corresponding second reference template region among the plurality of second reference template regions.

[0025] In another specific embodiment of the third aspect of this disclosure, the first template area is the same as the second template, and each of the plurality of first reference template areas is the same as a corresponding second template area among the plurality of second template areas.

[0026] In another specific embodiment of the third aspect of this disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a block-vector-based candidate list comprising a plurality of block-vector-based prediction candidates, wherein: each of the block-vector-based prediction candidates corresponds to a corresponding block-vector candidate among the plurality of block-vector candidates and a corresponding filter model among the plurality of filter models, the plurality of block-vector-based prediction candidates in the block-vector-based candidate list are ordered based on the first arrangement of the plurality of filter models, and the block cell is reconstructed based on the plurality of block-vector-based prediction candidates ordered in the block-vector-based candidate list.

[0027] In another specific embodiment of the third aspect of this disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a plurality of second template matching costs, each of the plurality of second template matching costs being computed directly based on a corresponding block vector candidate among the plurality of block vector candidates, without using the plurality of filter models of the block unit; and determine a second arrangement of a plurality of block vector-based prediction candidates based on the plurality of first template matching costs and the plurality of second template matching costs, wherein reconstructing the block unit is also based on the second arrangement of the plurality of block vector-based prediction candidates.

[0028] In another specific embodiment of the third aspect of this disclosure, the plurality of block vector-based prediction candidates includes at least one of a plurality of first block vector-based candidates or a plurality of second block vector-based candidates, each of the plurality of first block vector-based candidates corresponding to a corresponding block vector candidate among the plurality of block vector candidates and a corresponding filter model among the plurality of filter models, and each of the plurality of second block vector-based candidates corresponds only to a corresponding block vector candidate among the plurality of block vector candidates, without using the plurality of filter models.

[0029] In another specific embodiment of the third aspect of this disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine, based on the plurality of first template matching costs, a filter flag indicating whether to derive the plurality of filter models for determining the first arrangement of the plurality of filter models. Attached Figure Description

[0030] Various aspects of this disclosure can be best understood from the following detailed disclosure and corresponding drawings. The features are not drawn to scale, and for clarity of discussion, the dimensions of the features may be arbitrarily increased or decreased.

[0031] Figure 1 is a block diagram illustrating a system having a first electronic device and a second electronic device for encoding and decoding video data, specifically implemented according to one or more examples of the present disclosure.

[0032] Figure 2 is a block diagram illustrating a decoder module of the second electronic device illustrated in Figure 1, specifically implemented according to one or more examples of the present disclosure.

[0033] Figure 3 is a flowchart illustrating a method / process for decoding and / or encoding video data by an electronic device, specifically implemented according to one or more examples of this disclosure.

[0034] Figures 4A and 4B are schematic diagrams of reference blocks of block units indicated by block vectors, specifically implemented according to one or more examples of this disclosure.

[0035] Figures 5A and 5B are schematic diagrams illustrating the correspondence between a block template region and a reference template region in one or more embodiments of this disclosure.

[0036] Figures 6A to 6C are schematic diagrams illustrating the horizontal flipping relationship between a block template region and a reference template region, specifically implemented according to one or more examples of this disclosure.

[0037] Figure 7 is a flowchart illustrating a method / process for decoding and / or encoding video data by an electronic device, specifically implemented according to one or more examples of this disclosure.

[0038] Figures 8A and 8B are schematic diagrams illustrating the relative positions between a block cell and a first block template region, and between a reference block and a first reference template region, according to one or more examples of this disclosure.

[0039] Figure 9 is a block diagram illustrating an encoder module of the first electronic device illustrated in Figure 1, specifically implemented according to one or more examples of the present disclosure. Detailed Implementation

[0040] The following disclosure contains specific information relating to specific embodiments of this disclosure. The accompanying drawings and corresponding detailed disclosure relate to exemplary embodiments. However, this disclosure is not limited to these exemplary embodiments. Other variations and embodiments of this disclosure will occur to those skilled in the art.

[0041] Unless otherwise stated, similar or corresponding elements in the accompanying drawings are indicated by similar or corresponding reference numerals. The drawings and illustrations in this disclosure are generally not drawn to scale and are not intended to correspond to actual relative dimensions.

[0042] For the sake of consistency and ease of understanding, features are identified by reference indicators in the exemplary figures (although not illustrated in some examples). However, features in different specific embodiments may differ in other respects and should not be narrowly limited to what is illustrated in the figures.

[0043] This disclosure uses the phrases “in one embodiment” or “in some embodiments”, which may refer to one or more of the same or different embodiments. The term “coupled” is defined as a connection, whether direct or indirect through an intermediate component, and is not necessarily limited to a physical connection. The term “comprising” means “including but not necessarily limited to”, and specifically indicates an open inclusion or membership relationship in said combination, group, series, and equivalent.

[0044] For purposes of explanation rather than limitation, specific details such as functional entities, technologies, protocols, and standards are set forth to provide an understanding of the disclosed technologies. Detailed disclosures of well-known methods, technologies, systems, architectures, etc., are omitted to avoid obscuring this disclosure with unnecessary detail.

[0045] Those skilled in the art will recognize that any disclosed coding function or algorithm described herein can be implemented by hardware, software, or a combination of both. The disclosed functions may correspond to modules, which may be software, hardware, firmware, or any combination thereof.

[0046] Software implementations may include programs having one or more computer-executable instructions stored on a computer-readable medium, such as memory or other types of storage devices. For example, one or more microprocessors or general-purpose computers with communication processing capabilities may be programmed with computer-executable instructions to perform the disclosed functions or algorithms.

[0047] Microprocessors or general-purpose computers may be formed from application-specific integrated circuits (ASICs), programmable logic arrays, and / or one or more digital signal processors (DSPs). Although some of the disclosed embodiments are geared toward software installed and executed on computer hardware, alternative embodiments implemented as firmware, or hardware, or a combination of hardware and software are also within the scope of this disclosure. Computer-readable media include, but are not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, optical disc read-only memory (CD-ROM), magnetic tape cassettes, magnetic tape, disk storage, or any other equivalent medium capable of storing computer-executable instructions. Computer-readable media may be non-transitory computer-readable media.

[0048] Figure 1 is a block diagram illustrating a system 100 having a first electronic device and a second electronic device for encoding and decoding video data, implemented according to one or more examples of the present disclosure.

[0049] System 100 includes a first electronic device 110, a second electronic device 120, and a communication medium 130.

[0050] The first electronic device 110 can be a source device including any device configured to encode video data and transmit the encoded video data to the communication medium 130. The second electronic device 120 can be a destination device including any device configured to receive and decode the encoded video data via the communication medium 130.

[0051] The first electronic device 110 can communicate with the second electronic device 120 via a communication medium 130, either wired or wirelessly. The first electronic device 110 may include a source module 112, an encoder module 114, a first interface 116, and other components. The second electronic device 120 may include a display module 122, a decoder module 124, a second interface 126, and other components. The first electronic device 110 may be a video encoder, and the second electronic device 120 may be a video decoder.

[0052] The first electronic device 110 and / or the second electronic device 120 may be a mobile phone, tablet computer, desktop computer, laptop computer, or other electronic device. Figure 1 illustrates an example of the first electronic device 110 and the second electronic device 120. The first electronic device 110 and the second electronic device 120 may include more or fewer components than those illustrated, or have different configurations of various illustrated components.

[0053] Source module 112 may include a video capture device for capturing new video, a video archive for storing previously captured video, and / or a video feed interface for receiving video from a video content provider. Source module 112 may generate computer graphics-based data as source video, or may generate a combination of real-time video, archived video, and computer-generated video as source video. The video capture device may include a charge-coupled device (CCD) image sensor, a complementary metal-oxide-semiconductor (CMOS) image sensor, or a camera.

[0054] Encoder module 114 and decoder module 124 may each be implemented as any of a variety of suitable encoder / decoder circuits, such as one or more microprocessors, central processing units (CPUs), graphics processing units (GPUs), system-on-a-chip (SoCs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When implemented in part in software, the device may store a program having computer-executable instructions for software in a suitable non-transitory computer-readable medium and use one or more processors to execute the stored computer-executable instructions to perform the disclosed methods. Each of encoder module 114 and decoder module 124 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (CODEC) in the device.

[0055] The first interface 116 and the second interface 126 may utilize custom protocols or comply with existing or de facto standards, including but not limited to Ethernet, IEEE 802.11 or IEEE 802.15 series, wireless USB, or telecommunications standards, including (but not limited to) Global System for Mobile Communications (GSM), Code Division Multiple Access 2000 (CDMA2000), Time Division Synchronous Code Division Multiple Access (TD-SCDMA), Global System for Microwave Access (WiMAX), 3GPP Long Term Evolution (3GPP-LTE), or Time Division LTE (TD-LTE). The first interface 116 and the second interface 126 may each include any device configured to transmit and receive compatible video bitstreams via the communication medium 130.

[0056] The first interface 116 and the second interface 126 may include a computer system interface that enables compatible video bitstreams to be stored on or received from a storage device. For example, the first interface 116 and the second interface 126 may include a chipset that supports Peripheral Component Interconnect (PCI) and Peripheral Component Interconnect Fast (PCIe) bus protocols, proprietary bus protocols, Universal Serial Bus (USB) protocols, Internal Integrated Circuit (I2C) protocols, or any other logical and physical architecture that can be used to interconnect peer devices.

[0057] Display module 122 may include a display using liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED), or light-emitting polymer (LPD) technology, and other display technologies may be used in some other embodiments. Display module 122 may include a high-definition display or an ultra-high-definition display.

[0058] Figure 2 is a block diagram illustrating a decoder module 124 of the second electronic device 120 illustrated in Figure 1, specifically implemented according to one or more examples of the present disclosure. The decoder module 124 may include an entropy decoder (e.g., entropy decoding unit 2241), a prediction processor (e.g., prediction processing unit 2242), an inverse quantization / inverse transform processor (e.g., inverse quantization / inverse transform unit 2243), an adder (e.g., adder 2244), a filter (e.g., filter unit 2245), and a decoded image buffer (e.g., decoded image buffer 2246). The prediction processing unit 2242 may also include an intra-frame prediction processor (e.g., intra-frame prediction unit 22421) and an inter-frame prediction processor (e.g., inter-frame prediction unit 22422). The decoder module 124 receives a bitstream, decodes the bitstream, and outputs decoded video.

[0059] The entropy decoding unit 2241 can receive a bitstream including multiple syntax elements from the second interface 126, as shown in FIG1, and perform a parsing operation on the bitstream to extract syntax elements from the bitstream. As part of the parsing operation, the entropy decoding unit 2241 can perform entropy decoding on the bitstream to generate quantized transform coefficients, quantization parameters, transform data, motion vectors, intra-frame modes, segmentation information and / or other syntax information.

[0060] The entropy decoding unit 2241 can perform context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probabilistic interval segmented entropy (PIPE) coding, or another entropy coding technique to generate quantized transform coefficients. The entropy decoding unit 2241 can provide the quantized transform coefficients, quantization parameters, and transform data to the inverse quantization / inverse transform unit 2243, and provide motion vectors, intra-frame modes, segmentation information, and other syntactic information to the prediction processing unit 2242.

[0061] The prediction processing unit 2242 may receive syntax elements, such as motion vectors, intra-frame modes, segmentation information, and other syntax information, from the entropy decoding unit 2241. The prediction processing unit 2242 may receive syntax elements including segmentation information and segment image frames according to the segmentation information.

[0062] Each of these image frames can be divided into at least one image block based on the segmentation information. This at least one image block may include a luminance block for reconstructing multiple luminance samples and at least one chrominance block for reconstructing multiple chrominance samples. The luminance block and the at least one chrominance block may be further subdivided to generate macroblocks, coding tree units (CTUs), coding blocks (CBs), their sub-segments, and / or other equivalent coding units.

[0063] During the decoding process, the prediction processing unit 2242 may receive prediction data including intra-frame mode or motion vectors of the current image block of a specific image frame. The current image block may be a luminance block or a chrominance block in the specific image frame.

[0064] Intra-prediction unit 22421 can perform intra-prediction coding on the current block unit relative to one or more adjacent blocks in the same frame as the current block unit, based on syntax elements associated with the intra-frame mode, to generate a prediction block. The intra-frame mode can specify the position of a reference sample selected from adjacent blocks in the current frame. When the chroma components are reconstructed by prediction processing unit 2242, intra-prediction unit 22421 can reconstruct multiple chroma components of the current block unit based on multiple luma components of the current block unit.

[0065] When the prediction processing unit 2242 reconstructs multiple luminance components of the current block unit, the intra-prediction unit 22421 can reconstruct multiple chrominance components of the current block unit based on the multiple luminance components in the current block unit.

[0066] Inter-frame prediction unit 22422 can perform inter-frame prediction coding of the current block unit relative to one or more blocks in one or more reference image blocks, based on syntax elements associated with motion vectors, to generate a prediction block. The motion vectors can indicate the displacement of the current block unit within the current image block relative to reference block units within the reference image block. Reference block units can be blocks determined to closely match the current block unit. Inter-frame prediction unit 22422 can receive reference image blocks stored in the decoded image buffer 2246 and reconstruct the current block unit based on the received reference image blocks.

[0067] The inverse quantization / inverse transform unit 2243 can apply inverse quantization and inverse transform to reconstruct the residual block in the pixel domain. The inverse quantization / inverse transform unit 2243 can apply inverse quantization to the quantized transform coefficients of the residual to generate residual transform coefficients, and then apply inverse transform to the residual transform coefficients to generate the residual block in the pixel domain.

[0068] Inverse transforms can be applied in reverse order of the transform process, such as Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Adaptive Multiple Transform (AMT), Mode-dependent Inseparable Quadratic Transform (MDNSST), Hypercube Givens Transform (HyGT), Signal-dependent Transform, Karhunen-Loeve Transform (KLT), Wavelet Transform, Integer Transform, Subband Transform, or conceptually similar transforms. Inverse transforms can convert residual information from the transform domain (such as the frequency domain) back to the pixel domain. The degree of inverse quantization can be modified by adjusting the quantization parameters.

[0069] Adder 2244 can add the reconstructed residual block to the prediction block provided by prediction processing unit 2242 to produce a reconstructed block.

[0070] Filtering unit 2245 may include a deblocking filter, a sample adaptive offset (SAO) filter, a bilateral filter, and / or an adaptive loop filter (ALF) to remove block artifacts from the reconstructed blocks. In addition to the deblocking filter, SAO filter, bilateral filter, and ALF, additional filters (in-loop or post-loop) may be used. Such filters (not explicitly illustrated for simplicity) can filter the output of adder 2244. After filtering unit 2245 performs the filtering process on the reconstructed blocks of a specific image frame, filtering unit 2245 can output the decoded video to display module 122 or other video receiving unit.

[0071] The decoded image buffer 2246 may be a reference image memory that stores a reference block used by the prediction processing unit 2242 when decoding the bitstream (e.g., in inter-frame coding mode). The decoded image buffer 2246 may be formed of any of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM)) or other types of memory devices. The decoded image buffer 2246 may be on-chip along with other components of the decoder module 124, or it may be off-chip relative to these components.

[0072] Figure 3 is a flowchart illustrating a method / process 300 for decoding and / or encoding video data by an electronic device, specifically implemented according to one or more examples of this disclosure. Method / process 300 is an exemplary implementation, as various mechanisms for decoding video data may exist.

[0073] Method / process 300 may be performed by an electronic device using the configuration illustrated in Figures 1 and / or 2, wherein various elements of the method / process 300 may be described with reference to these figures. Each block illustrated in Figure 3 may represent one or more processes, methods, or subroutines performed by the electronic device.

[0074] The order in which the boxes appear in Figure 3 is for illustrative purposes only and should not be construed as limiting the scope of this disclosure; therefore, the order may differ from the illustrative order. Additional boxes may be added or fewer boxes may be used without departing from the scope of this disclosure.

[0075] At box 310, method / process 300 may begin by receiving video data (e.g., via decoder module 124, as shown in FIG2). The video data received by decoder module 124 may include a bitstream.

[0076] Referring to Figures 1 and 2, the second electronic device 120 can receive bitstreams from an encoder such as the first electronic device 110 (or other video provider) via a second interface 126.

[0077] At box 320, decoder module 124 can determine block units from the current frame included in the video data.

[0078] Referring to Figures 1 and 2, when the video data received by the decoder module 124 includes a bitstream, the decoder module 124 can determine the image frames included in the bitstream. The current frame can be one image frame determined according to the bitstream. The decoder module 124 can further divide the current frame according to the segmentation indication in the bitstream to determine block units. In some embodiments, the decoder module 124 can divide the current frame to generate multiple CTUs, and can further divide the current CTUs included in the CTUs to generate multiple partition blocks, and determine block units from the partition blocks according to the segmentation indication (e.g., based on any video coding standard). In some other embodiments, the decoder module 124 can divide the current frame to generate multiple slices, and further divide the current slices included in the slices to generate multiple CTUs. In addition, the decoder module 124 can further divide the current CTUs included in the CTUs to generate multiple partition blocks, and determine block units from the partition blocks according to the segmentation indication. The size of the block unit can be Wb×Hb. In some specific implementations, each of Wb and Hb can be a positive integer that can be equal to or different from each other (e.g., 4, 8, etc.).

[0079] At box 330, decoder module 124 can determine the reference block indicated by the block vector from the current frame.

[0080] Referring to Figures 1 and 2, the decoder module 124 can derive block vectors in Reconstructed Reordering (RR) Intra-Frame Block Copying (IBC) (RR-IBC) mode. Therefore, the RR-IBC mode can be used to determine the reference block indicated by the block vector.

[0081] Decoder module 124 determines a vector candidate list for predicting block units. The vector candidate list can be derived based on multiple adjacent blocks adjacent to the block unit. Adjacent blocks can be reconstructed before reconstructing the block unit. Adjacent blocks may include at least one of the following: a first adjacent block located above the upper right corner of the block unit; a second adjacent block located to the left of the lower left corner of the block unit; a third adjacent block located to the upper right side of the block unit; a fourth adjacent block located to the lower left side of the block unit; and a fifth adjacent block located to the upper left side of the block unit. When reconstructing a specific adjacent block based on its adjacent block vectors, the adjacent block vector of that specific adjacent block can be determined as an adjacency candidate vector for the block unit. The adjacent block vectors can be generated using either IBC mode or IntraTMP mode for predicting or reconstructing a specific adjacent block.

[0082] In some implementations, decoder module 124 may determine the association type of a block unit from video data. This association type may be a flip type, specifying which of several flip directions is applied. In some implementations, the flip direction may include horizontal and vertical flips. When the flip direction includes only horizontal and vertical flips, the video data may include an association flag specifying which of the horizontal and vertical flips is applied. In some other implementations, the flip direction may also include other directional flips. When the number of flip directions is greater than two, the video data may include an association index specifying which of the flip directions is applied. Since both the association flag and the association index are determined for the entire block unit, the association type corresponding to each different adjacent candidate vector of the block unit may be the same for each other. In RR-IBC mode, the association flag may be a flip flag.

[0083] In some other implementations, the association type of the block cell corresponding to the adjacent block vector of a specific adjacent block can be inherited from the association type of that specific adjacent block. Therefore, when the association type of a specific adjacent block specifies horizontal flipping, the association type of the block cell corresponding to the adjacent block vector of that specific adjacent block can also be horizontal flipping. Additionally, when the association type of a specific adjacent block specifies vertical flipping, the association type of the block cell corresponding to the adjacent block vector of that specific adjacent block can also be vertical flipping. Since different adjacent blocks can have different association types, the association types corresponding to different adjacent block vectors of each adjacent block can also be different from each other.

[0084] The decoder module 124 can also compare adjacent candidate vectors with corresponding association types, which can be the association type of the block unit indicated by video data or inherited from the association type of a specific adjacent block. In some embodiments, when the vector direction of an adjacent candidate vector is the same as the flip direction specified by the corresponding association type, the adjacent candidate vector can be determined as a reference candidate vector in the vector candidate list for deriving block vectors. For example, when the adjacent candidate vector is a horizontal vector and the corresponding association type specifies horizontal flip, the adjacent candidate vector can be determined as a reference candidate vector in the vector candidate list. Additionally, when the adjacent candidate vector is a vertical vector and the corresponding association type specifies vertical flip, the adjacent candidate vector can also be determined as a reference candidate vector in the vector candidate list. In some other embodiments, when the vector direction of an adjacent candidate vector is different from the flip direction specified by the corresponding association type, the adjacent candidate vector of the block unit can be excluded from the vector candidate list. In still other embodiments, when the flip direction includes only horizontal and vertical flips, adjacent candidate vectors with vector directions different from the horizontal and vertical directions can be excluded from the vector candidate list.

[0085] Figures 4A and 4B are schematic diagrams of reference blocks for block units indicated by block vectors, specifically implemented according to one or more examples of this disclosure. Figure 4A illustrates a reference block 410 of block unit 400, indicated by a block vector 4012 derived from the adjacent block vector 4011 of adjacent block 401. The adjacent block vector 4011 of adjacent block 401 can be used to indicate an adjacent reference 411 for predicting and / or reconstructing adjacent block 401. The adjacent block vector 4011 of adjacent block 401 can be a horizontal vector. Therefore, when the association type corresponding to adjacent block vector 4011 is horizontal flip, adjacent block vector 4011 can be determined as a candidate reference vector for deriving block vector 4012. As shown in Figure 4A, since the association type corresponding to adjacent block vector 4011 is horizontal flip, block unit 400 and reference block 410 are symmetrical about each other along a vertical line of symmetry 4013. Therefore, the association type can also indicate the association between block unit 400 and reference block 410. As shown in Figure 4A, the relationship between block unit 400 and reference block 410 can be left-right symmetrical.

[0086] The coordinates of the center point within block cell 400 can be (Xc, Yc). When the horizontal coordinate of the center point within adjacent block 401 is Xn and the adjacent block vector 4011 of reference block 410 is (BVn, 0), the block vector 4012 of block cell 400 can be (2Xn - 2Xc + BVn, 0). Therefore, the coordinates of the center point within reference block 410 can be (2Xn - Xc + BVn, Yc).

[0087] Figure 4B illustrates a reference block 420 of block unit 400, indicated by a block vector 4022 derived from the adjacent block vector 4021 of adjacent block 401. The adjacent block vector 4021 of adjacent block 401 can be used to indicate an adjacent reference 421 for predicting and / or reconstructing adjacent block 401. The adjacent block vector 4021 of adjacent block 401 can be a vertical vector. Therefore, when the association type corresponding to adjacent block vector 4021 is vertical flip, adjacent block vector 4021 can be determined as a candidate reference vector for deriving block vector 4022. As shown in Figure 4B, since the association type corresponding to adjacent block vector 4021 is vertical flip, block unit 400 and reference block 420 are symmetrical about each other along a horizontal line of symmetry 4023. Therefore, the association type can also indicate the association between block unit 400 and reference block 420. As shown in Figure 4B, the association between block unit 400 and reference block 420 can be a top-bottom symmetrical relationship.

[0088] The coordinates of the center point within block cell 400 can be (Xc, Yc). When the vertical coordinate of the center point within adjacent block 401 is Yn and the adjacent block vector 4011 of reference block 410 is (0, BVn), the block vector 4012 of block cell 400 can be (0, 2Yn - 2Yc + BVn). Therefore, the coordinates of the center point within reference block 410 can be (Xc, 2Yn -Yc + BVn).

[0089] In some implementations, when the number of reference candidate vectors is equal to 1, the reference candidate vector can be directly determined as the reference vector for deriving the reference block. In other implementations, when the number of reference candidate vectors is greater than 1, one of the reference candidate vectors can be selected as the reference vector by using a candidate index determined from the video data.

[0090] Referring again to Figure 3, at box 340, the decoder module 124 can determine the correspondence between the block template region adjacent to the block unit and the reference template region adjacent to the reference block based on the association type.

[0091] Referring to Figures 1 and 2, the decoder module 124 can determine the block template region adjacent to the block unit in the current frame. The block template region may include at least one of a first adjacent region located to the left of the block unit, a second adjacent region located above the block unit, and a third adjacent region located to the upper left of the block unit. Since the first to third adjacent regions can be reconstructed before reconstructing the block unit, the block template region can be reconstructed before reconstructing the block unit. Therefore, when predicting and / or reconstructing the block unit, the decoder module 124 is allowed to directly determine the reconstruction result of the block template region.

[0092] In some implementations, the size of the block template region can be determined based on the size of the block cells. In other implementations, the size of the block template region can be predefined. For example, the height of the second adjacent region and the width of the first adjacent region can be equal to 1, 2, 3, 4, or other positive integers.

[0093] The decoder module 124 can determine the reference template region adjacent to the reference block in the current frame based on the association type. The reference template region may include at least one of a first reference adjacent region located to the left of the reference block, a second reference adjacent region located above the reference block, a third reference adjacent region located to the right of the reference block, a fourth reference adjacent region located below the reference block, a fifth adjacent region located to the upper right of the block unit, and a sixth adjacent region located to the lower left of the block unit. In some embodiments, when the association type specifies horizontal flipping, the reference template region may include at least one of the second reference adjacent region located above the reference block, the third reference adjacent region located to the right of the reference block, and the fifth adjacent region located to the upper right of the block unit. In some other embodiments, when the association type specifies vertical flipping, the reference template region may include at least one of the first reference adjacent region located to the left of the reference block, the fourth reference adjacent region located below the reference block, and the sixth adjacent region located to the lower left of the block unit.

[0094] Since the first to sixth reference adjacent regions can be reconstructed before the reconstructed block unit, the reference template region can also be reconstructed before the reconstructed block unit. Therefore, when predicting and / or reconstructing a block unit, the decoder module 124 is allowed to directly determine the reconstruction result of the reference template region. The size of the reference template region can be the same as the size of the block template region.

[0095] The correspondence between the block template region and the reference template region can be determined based on the association type. In some implementations, when the association type specifies horizontal flipping, the first adjacent region on the left side of the block unit can correspond to the third reference adjacent region on the right side of the reference block. The second adjacent region above the block unit can correspond to the second reference adjacent region above the reference block. Additionally, when the block template region includes a third adjacent region, the third adjacent region on the upper left side of the block unit can correspond to the fifth adjacent region on the upper right side of the reference block. In some other implementations, when the association type specifies vertical flipping, the first adjacent region on the left side of the block unit can correspond to the first reference adjacent region on the left side of the reference block. The second adjacent region above the block unit can correspond to the fourth reference adjacent region below the reference block. Additionally, when the block template region includes a third adjacent region, the third adjacent region on the upper left side of the block unit can correspond to the sixth adjacent region on the lower left side of the reference block.

[0096] Figures 5A and 5B are schematic diagrams illustrating the correspondence between block template regions and reference template regions according to one or more embodiments of the present disclosure. Figure 5A illustrates the correspondence between a first adjacent region 403 and a second adjacent region 404 in the block template region and a third reference adjacent region 413 and a second reference adjacent region 414 in the reference template region. The first adjacent region 403 located on the left side of block unit 400 may correspond to the third reference adjacent region 413 located on the right side of reference block 410. The second adjacent region 404 located above block unit 400 may correspond to the second reference adjacent region 414 located above reference block 410.

[0097] Because the association type specifies horizontal flipping, as shown in FIG5A, the block template region and block unit 400 can be symmetrical with the reference template region and reference block 410 along the vertical symmetry line 4013. In some embodiments, the first block template sample 4031 in the first block adjacent region 403 can correspond to the first reference template sample 4131 in the third reference adjacent region 413. The second block template sample 4032 in the first block adjacent region 403 can correspond to the second reference template sample 4132 in the third reference adjacent region 413. In addition, other block template samples in the first block adjacent region 403 can correspond to other reference template samples in the third reference adjacent region 413 along the vertical symmetry line 4013. In some other embodiments, the first block template sample 4041 in the second block adjacent region 404 can correspond to the first reference template sample 4141 in the second reference adjacent region 414. The second template sample 4042 in the second block adjacent region 404 can correspond to the second reference template sample 4142 in the second reference adjacent region 414. In addition, other block template samples in the second adjacent region 404 can correspond to other reference template samples in the second reference adjacent region 414 along the vertical symmetry line 4013.

[0098] The widths of the first adjacent region 403 and the third reference adjacent region 413 can be Wt, and the heights of the second adjacent region 404 and the second reference adjacent region 414 can be Ht. In some specific implementations, each of Wt and Ht can be a positive integer that can be equal to or different from each other (e.g., 2, 3, 4, etc.).

[0099] The coordinates of the first template sample 4031 located on the upper left side of the first adjacent region 403 can be set to T1(0, 0). The coordinates of the second reference template sample 4132 located on the upper left side of the third reference adjacent region 413 can be set to T1'(0, 0). Therefore, the coordinates of the first reference template sample 4131 in the third reference adjacent region 413, corresponding to the first template sample 4031 in the first adjacent region 403, can be set to T1'(Wt-1, 0). Additionally, the coordinates of the second template sample 4032 in the first adjacent region 403, corresponding to the second reference template sample 4132 in the third reference adjacent region 413, can be set to T1(Wt-1, 0). Furthermore, the coordinates of the block template sample at the bottom of the first adjacent region 403 can be set to T1(0, Hb-1), ..., T1(Wt-1, Hb-1), and the coordinates of the reference template sample at the bottom of the third reference adjacent region 413 can be set to T1'(0, Hb-1), ..., T1'(Wt-1, Hb-1). The block template sample located at T1(0, Hb-1) can correspond to the reference template sample located at T1'(Wt-1, Hb-1), and the block template sample located at T1(Wt-1, Hb-1) can correspond to the reference template sample located at T1'(0, Hb-1).

[0100] The coordinates of the first template sample 4041 located on the upper left side of the second adjacent region 404 can be set to T2(0, 0). The coordinates of the reference template sample located on the upper left side of the second reference adjacent region 414 can be set to T2'(0, 0). Therefore, the coordinates of the first reference template sample 4141 in the second reference adjacent region 414, corresponding to the first template sample 4041 in the second adjacent region 404, can be set to T2'(Wb-1, 0). Furthermore, the coordinates of the block template samples located on the right side of the second adjacent region 404 can be set to T2(Wb-1, 0), ..., T2(Wb-1, Ht-1). The coordinates of the reference template samples located on the left side of the second reference adjacent region 414 can be set to T2'(0, 0), ..., T2'(0, Ht-1). The block template sample located at T2(Wb-1, 0) can correspond to the reference template sample located at T2'(0, 0). The block template sample located at T2(Wb-1,Ht-1) can correspond to the reference template sample located at T2'(0,Ht-1).

[0101] Figure 5B illustrates the correspondence between the first adjacent region 403 and the second adjacent region 404 in the block template region and the first reference adjacent region 423 and the fourth reference adjacent region 424 in the reference template region. The first adjacent region 403 located on the left side of the block unit 400 can correspond to the first reference adjacent region 423 located on the left side of the reference block 420. The second adjacent region 404 located above the block unit 400 can correspond to the fourth reference adjacent region 424 located below the reference block 420.

[0102] Because the association type specifies vertical flipping, as shown in FIG5B, the block template region and block unit 400 can be symmetrical with the reference template region and reference block 420 along the horizontal symmetry line 4023. In some embodiments, the first block template sample 4031 in the first block adjacent region 403 can correspond to the first reference template sample 4231 in the first reference adjacent region 423. The second block template sample 4032 in the first block adjacent region 403 can correspond to the second reference template sample 4232 in the first reference adjacent region 423. In addition, other block template samples in the first block adjacent region 403 can correspond to other reference template samples in the first reference adjacent region 423 along the horizontal symmetry line 4023. In some other embodiments, the first block template sample 4041 in the second block adjacent region 404 can correspond to the first reference template sample 4241 in the fourth reference adjacent region 424. The second template sample 4042 in the second block adjacent region 404 can correspond to the second reference template sample 4242 in the fourth reference adjacent region 424. In addition, other block template samples in the second adjacent region 404 can correspond to other reference template samples in the fourth reference adjacent region 424 along the horizontal symmetry line 4023.

[0103] The coordinates of the reference template sample located on the upper left side of the first reference adjacency region 423 can be set to T1'' (0,0). Therefore, the coordinates of the first reference template sample 4231 in the first reference adjacency region 423, corresponding to the first template sample 4031 in the first adjacency region 403, can be set to T1'' (0, Hb-1). Additionally, the coordinates of the second reference template sample 4232 in the first reference adjacency region 423, corresponding to the second template sample 4032 in the first adjacency region 403, can be set to T1'' (Wt-1, Hb-1).

[0104] The coordinates of the reference template sample located on the upper left side of the fourth reference adjacency region 424 can be set to T2'' (0,0). Therefore, the coordinates of the first reference template sample 4241 in the fourth reference adjacency region 424, which corresponds to the first template sample 4041 in the second adjacency region 404, can be set to T2'' (0, Ht-1). Additionally, the coordinates of the second reference template sample 4232 in the fourth reference adjacency region 424, which corresponds to the second template sample 4042 in the second adjacency region 404, can be set to T2'' (1, Ht-1).

[0105] Referring again to Figure 3, at box 350, the decoder module 124 can derive the prediction model of the block cell based on the correspondence between the block template region and the reference template region.

[0106] The prediction model can be selected from multiple candidate models, including at least one of multiple linear candidate models and multiple nonlinear candidate models. Linear candidate models may include Nm-tap linear models. The number Nm is a positive integer and may be predefined. For example, the number Nm may be equal to 1, 2, 3, 4, 5, 7, or 9. Nonlinear candidate models may include quadratic, cubic, and quartic candidate models. In some embodiments, when the order of a candidate model is equal to 1, the candidate model may be a linear candidate model among linear candidate models. In some other embodiments, when the order of a candidate model is greater than 1, the candidate model may be a nonlinear candidate model among nonlinear candidate models. For example, since quartic candidate models include squared terms, they may be included in the nonlinear candidate models. It should be noted that other candidate models may be added without departing from the scope of this disclosure.

[0107] In some specific implementations, the nonlinear candidate model may include a quadratic candidate model. A quadratic candidate model comprises five spatial sample terms, a nonlinear term P, and a bias term B, and can be similar to a convolutional cross-component model (CCCM). The five spatial sample terms may include a center sample C, a north sample N, a south sample S, a west sample W, and an east sample E.

[0108] The central sample C can be located at the sample position T'(i, j) or T''(i, j) of the top-left reference template sample in the corresponding reference adjacency region that includes the central sample C. The north sample N, located above the central sample C, can be located at the sample position T'(i, j-1) or T''(i, j-1) of the top-left reference template sample in the corresponding reference adjacency region. The south sample S, located below the central sample C, can be located at the sample position T'(i, j+1) or T''(i, j+1) of the top-left reference template sample in the corresponding reference adjacency region. The west sample W, located to the left of the central sample C, can be located at the sample position T'(i-1, j) or T''(i-1, j) of the top-left reference template sample in the corresponding reference adjacency region. The eastern sample E, located to the right of the central sample C, can be located at sample position T'(i+1, j) or T''(i+1, j) relative to the upper-left reference template sample in the corresponding reference adjacency region. Therefore, the quadratic model can be represented by the following function: predLumaVal = C0×C + C1×N + C2×S + C3×E + C4×W + C5×P + C6×BP =(C × C + midVal)>>bitDepthB = 2 bitDepth-1 The coefficients C0, C1, C2, C3, C4, C5, and C6 are seven prediction coefficients used for the spatial sample terms C, N, S, W, and E, the nonlinear term P, and the bias term B. The parameters midVal and bias term B can be set to intermediate brightness values, and the parameter bitDepth can be the bit depth of the sample in the bitstream. For example, for 10 bits of content, the bias term B can be set to 512.

[0109] When deriving the prediction model, the predLumaVal value can be the predicted brightness value of a specific block template sample determined based on a correspondence. A specific block template sample can be included in a block adjacency region corresponding to a reference adjacency region including the center sample C, which is currently applied to the derived prediction model. Therefore, a block template sample with a predLumaVal value can be located at the sample position T(m, n) relative to the upper-left block template sample of the corresponding block adjacency region. For example, referring to Figure 5A, when the first reference template sample 4141 located at T2'(Wb-1, 0) is determined as the center sample C, the first block template sample 4041 located at T2(0, 0) can correspond to the first reference template sample 4141 to derive the prediction model.

[0110] In some other implementations, the linear candidate model may include a tap candidate model. A tap candidate model may include a spatial sample term and an offset term. The spatial sample term C may include a reference sample in the corresponding reference template region. Therefore, a tap candidate model can be represented by the following function: predLumaVal = C0 × C + C1, where coefficients C0 and C1 are two prediction coefficients. Coefficient C1 may also be determined as the offset term. When deriving the prediction model, the predLumaVal value may be the predicted brightness value of the block template sample determined based on the correspondence. The block template sample may be included in a block adjacency region corresponding to a reference adjacency region including the center sample C, which is currently applied to the derived prediction model.

[0111] Predictive coefficients can be derived by comparing previously reconstructed luminance values ​​in the block template region with predicted predLumaVal values ​​in the block template region. In some embodiments, difference minimization of the comparison can be performed by minimizing the MSE. In some embodiments, MSE minimization can be performed by computing the autocorrelation matrix. The autocorrelation matrix can be decomposed using LDL, and back substitution can be used to compute the convolution filter coefficients. In some embodiments, the decomposition can be Cholesky decomposition. In some embodiments, difference minimization can be performed by Gaussian elimination. It should be noted that the method for deriving the set of predictive coefficients can be changed without departing from the scope of this disclosure.

[0112] Referring to Figures 1 and 2, in order to derive a prediction model, decoder module 124 may pre-flip a reference template region based on an association type to correspond to a block template region. For example, the association type may specify a horizontal flip. Figures 6A to 6C are schematic diagrams illustrating the horizontal flip relationship between a block template region and a reference template region according to one or more examples of this disclosure. Figure 6A illustrates the correspondence between a first block adjacent region 403 and a second block adjacent region 404 in the block template region and a third reference adjacent region 413 and a second reference adjacent region 414 in the reference template region. The first block adjacent region 403 may be symmetrical to the third reference adjacent region 413 along a vertical symmetry line 4013. The second block adjacent region 404 may be symmetrical to the second reference adjacent region 414 along a vertical symmetry line 4013. Additionally, block unit 400 may also be symmetrical to reference block 410 along a vertical symmetry line 4013.

[0113] In some specific implementations, to facilitate the derivation of the prediction model, the decoder module 124 may pre-flip the reference template region along the vertical centerline of the reference block 410. Figure 6B illustrates the horizontal flipping relationship between the first adjacent region 403 and the second adjacent region 404 in the block template region and the flipped third reference adjacent region 413' and the flipped second reference adjacent region 414' in the reference template region within the decoder module 124. Therefore, the flipped third reference adjacent region 413' may be located to the left of the flipped reference block 410'. In addition, multiple flipped reference template samples in the flipped third reference template region 413' may be symmetrical with multiple original reference template samples in the third reference template region 413 along the vertical centerline 4014 of the reference block 410. Similarly, multiple flipped reference template samples in the flipped second reference template region 414' may also be symmetrical with multiple original reference template samples in the second reference template region 414 along the vertical centerline 4014 of the reference block 410. Therefore, the positional relationship between the flipped reference block 410' and the flipped reference template region can be the same as the positional relationship between the block unit 400 and the block template region, for deriving the prediction model. In some other specific embodiments, the positional relationship between the flipped reference block and the flipped reference template region generated by flipping the reference block 420 and the reference template region along the horizontal centerline of the reference block 420 can be the same as the positional relationship between the block unit 400 and the block template region.

[0114] In some specific implementations, to facilitate the derivation of the prediction model, the encoder module 114 can pre-flip the block template region along the vertical centerline of the block unit 400. Figure 6C illustrates the horizontal flipping relationship between the flipped first adjacent region 403' and the flipped second adjacent region 404' in the block template region and the third reference adjacent region 413 and the second reference adjacent region 414 in the reference template region within the encoder module 114. Therefore, the flipped first adjacent region 403' can be located to the right of the flipped block unit 400'. In addition, multiple flipped block template samples in the flipped block template region 403' can be symmetrical with multiple original block template samples in the block template region 403 along the vertical centerline 4015 of the block unit 400. Multiple flipped block template samples in the flipped second block template region 404' can also be symmetrical with multiple original block template samples in the second template region 404 along the vertical centerline 4015 of the reference block 410. Therefore, the positional relationship between the flipped block unit 400' and the flipped block template region can be the same as the positional relationship between the reference unit 410 and the reference template region, for deriving the prediction model. In some other specific embodiments, the positional relationship between the flipped block unit and the flipped block template region generated by flipping the block unit 400 and the block template region along the horizontal centerline of the block unit 400 can be the same as the positional relationship between the reference block 420 and the reference template region.

[0115] Referring again to Figure 3, at box 360, decoder module 124 can reconstruct block cells based on reference blocks using a prediction model with an association type.

[0116] Referring to Figures 1 and 2, in some specific implementations, the decoder module 124 can use a prediction model with an association type to predict block units based on a reference block to generate a prediction block. When a specific reference sample in the reference block is applied to the prediction model as a center sample C, the position of the corresponding block sample can be determined based on the correspondence, and the corresponding block sample has a predicted brightness value predLumaVal generated based on the specific reference sample.

[0117] When a specific reference sample in a reference block is applied as the center sample C to the prediction model, the predicted brightness value of the corresponding block sample can be generated. Referring to Figures 5A and 5B, the coordinates of the first block sample 4001 located on the upper left side of block cell 400 can be set to T0(0, 0). The coordinates of the reference sample located on the upper left side of reference block 410 can be set to T0'(0, 0). The coordinates of the reference sample located on the upper left side of reference block 420 can be set to T0''(0, 0). When a specific reference sample in a reference block is applied as the center sample C to the prediction model, the position of the center sample C can be located at the sample position T0'(i, j) or T0''(i, j) relative to the upper left reference template sample in the corresponding reference block. In addition, the block sample with the predLumaVal value can be located at the sample position T(m, n) relative to the upper left block template sample of the block cell. For example, referring to Figure 5A, when the first reference sample 4101 located at T0'(Wb-1, 0) is determined as the center sample C, the predLumaVal value of the first block sample 4001 located at T0(0, 0) can be generated. The position of the first block sample 4001 can be determined based on the correspondence.

[0118] To generate a prediction block, decoder module 124 may pre-flip a reference block based on an association type to correspond to a block unit. For example, the association type may specify a horizontal flip. Referring to Figures 6A and 6B, a plurality of flipped reference samples in flipped reference block 410' may be symmetrical to a plurality of original reference samples in reference block 410 along the vertical center line 4014 of reference block 410. Therefore, the positional relationship between the flipped reference samples in flipped reference block 410' may be the same as the positional relationship between a plurality of sample values ​​in block unit 400 for generating a prediction block. In some other embodiments, the positional relationship between the flipped reference samples in a flipped reference block generated by flipping reference block 420 along a horizontal center line may be the same as the positional relationship between sample values ​​in block unit 400 for generating a prediction block.

[0119] Decoder module 124 can determine multiple residual components from the bitstream for block units and add these residual components to the prediction block to reconstruct the block unit. Decoder module 124 can reconstruct all other block units in the image frame for use in reconstructing the image frame and video.

[0120] In some implementations, a block vector-based Local Illumination Compensation (LIC) flag for a block cell can be determined from video data to indicate whether a reference template region and a block template region were used to derive a prediction model for predicting or reconstructing the block cell. For example, when generating adjacent block vectors for determining adjacent blocks of a reference template region and a reference block using IBC merging mode or IBC Advanced Motion Vector Prediction (AMVP) mode, a block vector-based LIC flag for a block cell can be determined from video data. In some other implementations, when generating adjacent block vectors for specific adjacent blocks of a reference template region and a reference block using IBC merging mode, a block vector-based LIC flag for a block cell can be inherited from the specific adjacent block.

[0121] In some implementations, the RR-IBC flag can be determined from the bitstream to indicate whether the reference block indicated by the block vector is determined using the RR-IBC mode. Then, when the RR-IBC flag is determined to be true, the IBC-LIC flag can be further determined from the bitstream to indicate whether the reference template region and block template region generated using the RR-IBC mode are used to derive a prediction model for predicting or reconstructing block cells. In some other implementations, the IBC-LIC flag can be determined from the bitstream to indicate whether the reference template region and block template region are used to derive a prediction model for predicting or reconstructing block cells. Then, when the IBC-LIC flag is determined to be true, the RR-IBC flag can be further determined from the bitstream to indicate whether the block vector used to determine the reference template region to generate the prediction model is determined using the RR-IBC mode.

[0122] In some implementations, template type syntax can be determined from the bitstream to indicate the shape of block template regions and adjacent template regions. For example, the template type syntax may include a template type index that directly indicates which one or more of the first to third adjacent regions are included in the block template region. Additionally, the template type syntax may include more than one template type flag to determine the block template region. For example, a first template type flag can be used to determine whether both the first and second adjacent regions are included in the block template region. When the first template type flag is false, a second template type flag can be used to determine which of the first and second adjacent regions is included in the block template region. The template type syntax can be further determined from the bitstream when both the RR-IBC flag and the IBC-LIC flag are determined to be true. The template type syntax can be excluded from the bitstream when at least one of the RR-IBC flag and the IBC-LIC flag is determined to be false.

[0123] Then, method / procedure 300 can be completed.

[0124] Figure 7 is a flowchart illustrating a method / process 700 for decoding and / or encoding video data by an electronic device, specifically implemented according to one or more examples of this disclosure. Method / process 700 is an exemplary implementation, as various mechanisms for decoding video data may exist.

[0125] Method / process 700 may be performed by an electronic device using the configuration illustrated in Figures 1 and / or 2, wherein various elements of the method / process 700 may be described with reference to these figures. Each block illustrated in Figure 7 may represent one or more processes, methods, or subroutines performed by the electronic device.

[0126] The order in which the boxes appear in Figure 7 is for illustrative purposes only and should not be construed as limiting the scope of this disclosure; therefore, the order may differ from the illustrative order. Additional boxes may be added or fewer boxes may be used without departing from the scope of this disclosure.

[0127] At box 710, method / process 700 may begin by receiving video data (e.g., via decoder module 124, as shown in FIG2). The video data received by decoder module 124 may include a bitstream.

[0128] Referring to Figures 1 and 2, the second electronic device 120 can receive bitstreams from an encoder such as the first electronic device 110 (or other video provider) via a second interface 126.

[0129] At box 720, decoder module 124 can determine block units from the current frame included in the video data.

[0130] Referring to Figures 1 and 2, when the video data received by the decoder module 124 includes a bitstream, the decoder module 124 can determine the image frames included in the bitstream. The current frame can be one image frame determined according to the bitstream. The decoder module 124 can further divide the current frame according to the segmentation indication in the bitstream to determine block units. In some embodiments, the decoder module 124 can divide the current frame to generate multiple CTUs, and can further divide the current CTUs included in the CTUs to generate multiple partition blocks, and determine block units from the partition blocks according to the segmentation indication (e.g., based on any video coding standard). In some other embodiments, the decoder module 124 can divide the current frame to generate multiple slices, and further divide the current slices included in the slices to generate multiple CTUs. In addition, the decoder module 124 can further divide the current CTUs included in the CTUs to generate multiple partition blocks, and determine block units from the partition blocks according to the segmentation indication. The size of the block unit can be Wb×Hb. In some specific implementations, each of Wb and Hb can be a positive integer that can be equal to or different from each other (e.g., 4, 8, etc.).

[0131] At block 730, decoder module 124 can determine from the current frame multiple vector reference blocks of a block unit indicated by multiple block vector candidates of the block unit. In some specific implementations, each vector reference block may be indicated by a corresponding block vector candidate from the block vector candidates of the block unit.

[0132] Referring to Figures 1 and 2, decoder module 124 can determine a block vector list that includes block vector candidates for predicting block units. In some specific implementations, decoder module 124 can determine multiple adjacent blocks adjacent to the block unit. Decoder module 124 can also determine whether adjacent blocks are predicted and / or reconstructed based on multiple adjacent block vectors. Adjacent block vectors can be generated using either Intra-Block Copy (IBC) mode or Intra-Template Match Prediction (IntraTMP) mode for predicting or reconstructing adjacent blocks.

[0133] When predicting and / or reconstructing a specific neighboring block based on a specific neighboring block vector, the specific neighboring block vector can indicate the adjacent reference block in the current frame used to reconstruct the specific neighboring block. In other words, each neighboring block vector in the neighboring block vector can indicate the corresponding adjacent reference block in the current frame used to reconstruct the corresponding neighboring block. Additionally, the neighboring block vector can be determined as multiple block vector references used to determine block vector candidates in the block vector list.

[0134] Adjacent blocks can be reconstructed before reconstructing block units. Adjacent blocks may include at least one of the following: a first adjacent block located above the upper right corner of the block unit, a second adjacent block located to the left of the lower left corner of the block unit, a third adjacent block located to the upper right side of the block unit, a fourth adjacent block located to the lower left side of the block unit, and a fifth adjacent block located to the upper left side of the block unit.

[0135] In some other embodiments, the block vector reference may be generated based on multiple history-based block vectors. For example, multiple previous blocks reconstructed before a reconstructed block cell may be reconstructed based on multiple reconstruction schemes. When using block vectors to reconstruct one of the previous blocks, the block vectors of the previous blocks may be stored in a block vector table based on a first-in-first-out (FIFO) principle. The size of the block vector table may be equal to Nt. In some embodiments, Nt may be a positive integer, such as 6 or 12. In still other embodiments, the block vector reference may include multiple pairwise averaged block vectors generated based on two block vector references from other block vector references, and multiple filled block vectors generated using a filling process.

[0136] Decoder module 124 can determine block vector candidates based on a block vector reference including adjacent block vectors. Decoder module 124 can then determine the vector reference block indicated by the block vector candidates. In some embodiments, when IBC merging mode is allowed to predict block cells, the block vector reference can be directly determined as a block vector candidate. In some other embodiments, when IBC Advanced Motion Vector Prediction (AMVP) mode is allowed to predict block cells, the block vector reference can be combined with multiple block vector differences, and the combined result can be added to the block vector candidates.

[0137] Referring again to Figure 7, at box 740, the decoder module 124 can derive multiple filter models for the block unit based on a vector reference block. In some specific implementations, each filter model in the filter model can be derived based on a corresponding vector reference block in the vector reference block.

[0138] Referring to Figures 1 and 2, the decoder module 124 can determine a first template region adjacent to the block unit in the current frame. The first template region may include at least one of a first adjacent region located to the left of the block unit, a second adjacent region located above the block unit, and a third adjacent region located to the upper left of the block unit. Since the first to third adjacent regions can be reconstructed before reconstructing the block unit, the first template region can also be reconstructed before reconstructing the block unit. Therefore, when predicting and / or reconstructing a block unit, the decoder module 124 is allowed to directly determine the reconstruction result of the first template region.

[0139] In some implementations, the size of the first template region can be determined based on the size of the block cells. In some implementations, the size of the first template region can be predefined. For example, the height of the second adjacent region and the width of the first adjacent region can be equal to 1, 2, 3, 4, or other positive integers. In some implementations, the derivation of the size of the first template region in method / procedure 700 can be the same as the derivation in the block vector search process of the IBC search method.

[0140] In some other implementations, the size of the first template region can be determined based on the filter shape used at frame 740. For example, when the candidate model used to derive the filter model is a quadratic candidate model comprising five spatial sample terms C, N, S, E, and W, the size of the first template region can be equal to or less than the height of the five spatial sample terms C, N, S, E, and W. Therefore, the size of the first template region can be equal to 1, 2, or 3. In still other implementations, the size of the first template region can differ from the sizes of other template regions for different prediction stages. For example, the size of the first template region used to derive the filter model for reordering the filter model can differ from the size of the template region used to derive the linear filter model used in the motion compensation stage of inter-frame prediction.

[0141] Decoder module 124 can determine a first reference template region in the current frame for each vector reference block in the vector reference block. Each first reference template region can be adjacent to a corresponding vector reference block in the vector reference block. Each first reference template region can include at least one of a first reference adjacent region located to the left of the corresponding vector reference block, a second reference adjacent region located above the corresponding vector reference block, and a third reference adjacent region located to the upper left of the corresponding vector reference block. Since the first to third reference adjacent regions of the vector reference block can be reconstructed before the reconstruction block unit, the first reference template region of the vector reference block can be reconstructed before the reconstruction block unit. Therefore, when predicting and / or reconstructing a block unit, decoder module 124 can directly determine multiple reconstruction results of the first reference template region. The size of the first reference template region can be the same as the size of the first template region.

[0142] The filter model can be selected from multiple candidate models, including at least one of multiple linear candidate models and multiple nonlinear candidate models. Linear candidate models may include Nm-tap linear models. The number Nm is a positive integer and may be predefined. For example, the number Nm may be equal to 1, 2, 3, 4, 5, 7, or 9. Nonlinear candidate models may include quadratic, cubic, and quartic candidate models. In some embodiments, when the order of a candidate model is equal to 1, the candidate model may be a linear candidate model among the linear candidate models. In some other embodiments, when the order of a candidate model is greater than 1, the candidate model may be a nonlinear candidate model among the nonlinear candidate models. For example, since a quartic candidate model includes squared terms, it may be included in the nonlinear candidate models. It should be noted that other candidate models may be added without departing from the scope of this disclosure. Each filter model in the filter model may also be derived based on a corresponding first reference template region in the first template region and the first reference template region.

[0143] In some implementations, the nonlinear candidate model may include a quadratic candidate model. The quadratic candidate model comprises five spatial sample terms, a nonlinear term P, and a bias term B, and can be analogous to a convolutional cross-component model (CCCM). The five spatial sample terms may include: a center sample C, located at sample position (i, j) of the upper-left luminance sample relative to a specific vector reference block; a north sample N, located at sample position (i, j-1) of the upper-left luminance sample relative to a specific vector reference block and above the center sample C; and a south sample S, located at sample position (i, j+1) of the upper-left luminance sample relative to a specific vector reference block and below the center sample C. Additionally, the five spatial sample terms may also include: a west sample W, located at sample position (i-1, j) of the upper-left luminance sample relative to a specific vector reference block and to the left of the center sample C; and an east sample E, located at sample position (i+1, j) of the upper-left luminance sample relative to a specific vector reference block and to the right of the center sample C. Therefore, the 5-tap linear model can be represented by the following function: predLumaVal = C0×C + C1×N + C2×S + C3×E + C4×W + C5×P + C6×BP =(C×C + midVal)>>bitDepthB = 2 bitDepth-1The coefficients C0, C1, C2, C3, C4, C5, and C6 are seven filter coefficients used for the spatial sample terms C, N, S, W, and E, the nonlinear term P, and the bias term B. `predLumaVal` can be the predicted luminance sample located at the sample position (i, j) relative to the top-left luminance sample of the block cell. Additionally, the parameters `midVal` and `bias term B` can be set to intermediate luminance values, and `bitDepth` can be the bit depth of the sample in the bitstream. For example, for 10-bit content, the bias term B can be set to 512.

[0144] When decoder module 124 derives the filter model of the block unit, it can determine whether a spatial sample item is included in the first reference template region. Therefore, when excluding a specific spatial sample item from a particular first reference template region, the excluded item can be replaced by filling it with a sample adjacent to the excluded spatial sample item included in the particular first reference template region. For example, when center sample C is located at the top edge of the particular first reference template region, north sample N can be excluded from the particular first reference template region. Since center sample C, which is adjacent to north sample N, is included in the particular first reference template region, north sample N can be replaced by filling it with center sample C.

[0145] In some other implementations, the linear candidate model may include a tap candidate model. A tap candidate model may include a spatial sample term and an offset term. The spatial sample term C may include a reference sample in the corresponding first reference template region. Therefore, a tap candidate model can be represented by the following function: predLumaVal = C0 × C + C1, where coefficients C0 and C1 are two prediction coefficients, and predLumaVal may be a predicted luminance sample located at sample position (i, j) relative to the upper-left luminance sample of the block cell. Coefficient C1 may also be determined as the offset term.

[0146] Figures 8A and 8B are schematic diagrams illustrating the relative positions between a block unit and a first block template region, and the relative positions between a reference block and a first reference template region, according to one or more embodiments of the present disclosure. Figure 8A illustrates a first adjacent region 801 and a second adjacent region 802 in the first block template region of block unit 800, and a first reference adjacent region 811 and a second reference adjacent region 812 in the first reference template region of reference block 810. The first adjacent region 801 and the second adjacent region 802 may be directly adjacent to block unit 800, and the first reference adjacent region 811 and the second reference adjacent region 812 may be directly adjacent to reference block 810.

[0147] In some implementations, when a south sample S is included in reference block 810 and the center sample C is adjacent to reference block 810, the decoder module 124 can fill the south sample S by copying the center sample C during the filter modeling of the derived block unit. Similarly, when an east sample E is included in reference block 810 and the center sample C is adjacent to reference block 810, the decoder module 124 can fill the east sample E by copying the center sample C during the filter modeling of the derived block unit. In other implementations, when a south sample S is included in reference block 810, the decoder module 124 can directly use the reconstructed value of the south sample S without filling. Likewise, when an east sample E is included in reference block 810, the decoder module 124 can directly use the reconstructed value of the east sample E without filling.

[0148] Figure 8B illustrates the first adjacent region 803 and the second adjacent region 804 of block cell 800, and the first reference adjacent region 813 and the second reference adjacent region 814 of reference block 810. The first adjacent region 803 and the second adjacent region 804 can be separated from and adjacent to block cell 800. Additionally, the first reference adjacent region 813 and the second reference adjacent region 814 can be separated from and adjacent to reference block 810. To derive the filter model, the first adjacent region 803 and the second adjacent region 804 can be set as the first adjacent region and the second adjacent region of the first template region. Furthermore, the first reference adjacent region 813 and the second reference adjacent region 814 can be set as the first reference adjacent region and the second reference adjacent region of the first reference template region. Therefore, even if the center sample C is located at the bottom edge of the second reference adjacent region 814, the south sample S may not be included in reference block 810. Furthermore, even if the eastern sample E is located at the right edge of the first reference adjacency region 813, the eastern sample E may not be included in the reference block 810.

[0149] The prediction coefficients can be derived by comparing the previously reconstructed luminance values ​​in the first template region with the predicted predLumaVal values ​​in the first template region. In some embodiments, difference minimization of the comparison can be performed by minimizing the MSE. In some embodiments, MSE minimization can be performed by computing the autocorrelation matrix. The autocorrelation matrix can be decomposed using LDL, and back substitution can be used to compute the convolution filter coefficients. In some embodiments, the decomposition can be Cholesky decomposition. In some embodiments, difference minimization can be performed by Gaussian elimination. It should be noted that the method for deriving the set of prediction coefficients can be changed without departing from the scope of this disclosure.

[0150] In some other embodiments, decoder module 124 may use method / procedure 300 to derive the filter model for the block unit. In other words, the first template region may correspond to multiple flipped first reference template regions. Therefore, decoder module 124 may flip the first template region or flip the first reference template region to derive the filter model for the vector reference block. In some embodiments, when the number of vector reference blocks is equal to 1, decoder module 124 may pre-flip the first reference template region of the vector reference block to correspond to the first template region based on the association type in method / procedure 300. In some other embodiments, when the number of vector reference blocks is greater than 1, decoder module 124 may pre-flip the first template region of the block unit to simultaneously correspond to the first reference template region based on the association type in method / procedure 300. Although the first reference template region in the derivation process including the flipping process differs from the first reference template region in the derivation process excluding the flipping process, the derivation of prediction coefficients in the derivation process including the flipping process can be the same as the derivation of prediction coefficients in the derivation process excluding the flipping process.

[0151] Decoder module 124 can determine a block vector-based candidate list, comprising multiple block vector-based prediction candidates, based on a filter model used for predicting block units. The block vector-based candidate list may include at least one of multiple first block vector-based candidates, multiple second block vector-based candidates, multiple third block vector-based candidates, or multiple fourth block vector-based candidates. Each first block vector-based candidate may correspond to a corresponding block vector candidate in the block vector candidate list and a corresponding filter model in the filter model list. Each second block vector-based candidate may correspond only to a corresponding block vector candidate in the block vector candidate list, without using a filter model. Each third block vector-based candidate may correspond to a corresponding block vector candidate in the block vector candidate list and a corresponding filter model in the filter model derived using method / procedure 300. Each fourth block vector-based candidate may correspond only to a corresponding block vector candidate in the block vector candidate list, without using a filter model. In some implementations, the fourth block vector-based candidate can be generated by flipping the reference template region, and the second block vector-based candidate can be generated without flipping the reference template region. In some implementations, the block vector-based candidate list may include only the first block vector-based candidate, such that each block vector-based prediction candidate in the block vector-based prediction candidate list corresponds to the corresponding block vector candidate and the corresponding filter model.

[0152] Referring again to Figure 7, at box 750, decoder module 124 can determine multiple template matching costs computed using the filter model of the block unit. In some specific implementations, the template matching cost may include multiple first template costs, and each of the first template costs can be computed using a corresponding filter model in the filter model of the block unit.

[0153] Referring to Figures 1 and 2, the decoder module 124 can determine a second template region adjacent to the block unit in the current frame. The second template region may include at least one of a fourth adjacent region located to the left of the block unit, a fifth adjacent region located above the block unit, and a sixth adjacent region located to the upper left of the block unit. Since the fourth to sixth adjacent regions can be reconstructed before reconstructing the block unit, the second template region can also be reconstructed before reconstructing the block unit. Therefore, when predicting and / or reconstructing the block unit, the decoder module 124 is allowed to directly determine the reconstruction result of the second template region.

[0154] In some implementations, the size of the second template region can be determined based on the size of the block cells. In other implementations, the size of the second template region can be predefined. For example, the height of the fifth adjacent region and the width of the fourth adjacent region can be equal to 1, 2, 3, 4, or other positive integers.

[0155] In some other embodiments, the size of the second template region can be determined based on the size of the first template region. For example, the second template region may be included within the first template region. In other words, the second template region may not exceed the size of the first template region. In some embodiments, the second template region may be exactly the same as the first template region. Therefore, the size of the second template region may also be the same as the size of the first template region. For example, when the size of the first template region is equal to 4, the size of the second template region may also be equal to 4. Additionally, as shown in Figures 8A and 8B, the second template region may be separate from block unit 800. In some other embodiments, the second template region may be different from the first template region. For example, when the size of the first template region is equal to 4, the size of the second template region may be equal to 1, 2, or 3, which is smaller than the size of the first template region.

[0156] Decoder module 124 can determine a second reference template region in the current frame for each vector reference block in the vector reference block. Each second reference template region can be adjacent to a corresponding vector reference block in the vector reference block. Each second reference template region can include at least one of a fourth reference adjacent region located to the left of the corresponding vector reference block, a fifth reference adjacent region located above the corresponding vector reference block, and a sixth reference adjacent region located to the upper left of the corresponding vector reference block. Since the fourth to sixth reference adjacent regions of the vector reference block can be reconstructed before the reconstruction block unit, the second reference template region of the vector reference block can be reconstructed before the reconstruction block unit. Therefore, when predicting and / or reconstructing a block unit, decoder module 124 can directly determine multiple reconstruction results of the second reference template region.

[0157] In some specific implementations, the size of the second reference template region can be the same as the size of the second template block region. Therefore, when the size of the second template block region is the same as the size of the first template region, the size of the second reference template region can be the same as the size of the first reference template region. In other words, each first reference template region in the first reference template region can also be the same as the corresponding second reference template region. Therefore, as shown in Figures 8A and 8B, the second reference template region can also be directly adjacent to or separate from the reference block 810.

[0158] Decoder module 124 can determine a third reference template region in the current frame for each vector reference block in the vector reference block. Each third reference template region can also be adjacent to a corresponding vector reference block in the vector reference block. Each third reference template region can include at least one of the following: a fourth reference adjacent region located to the left of the corresponding vector reference block; a fifth reference adjacent region located above the corresponding vector reference block; a seventh reference adjacent region located to the right of the corresponding vector reference block; an eighth reference adjacent region located below the corresponding vector reference block; a ninth adjacent region located to the upper right of the corresponding vector reference block; and a tenth adjacent region located to the lower left of the corresponding vector reference block. Since the fourth, fifth, and seventh to tenth reference adjacent regions of the vector reference block can also be reconstructed before the reconstruction block unit, the third reference template region of the vector reference block can be reconstructed before the reconstruction block unit. Therefore, when predicting and / or reconstructing a block unit, decoder module 124 can also directly determine multiple reconstruction results of the third reference template region. The size of the third reference template region can also be the same as the size of the second template region. Each third reference template region in the current frame can be determined based on the association type of the corresponding block vector candidate in method / procedure 300. Additionally, as shown in Figures 8A and 8B, the third reference template region can also be directly adjacent to or separate from reference block 810.

[0159] In some specific implementations, the decoder module 124 may use a corresponding filter model to predict the second template region based on each reconstruction result in the reconstruction results of the second reference template region, thereby generating one of a plurality of first prediction results for the second template region. Each second reference template region may be determined by a corresponding block vector candidate from the block vector candidates. In other words, each of the first block vector-based candidates can be used to determine one of the first prediction results for the second template region. In some specific implementations, the decoder module 124 may directly set the reconstruction results in the second reference template region as a plurality of second prediction results for the second template region. In other words, each of the second block vector-based candidates can be used to determine one of the second prediction results for the second template region.

[0160] In some other embodiments, decoder module 124 may flip the reconstruction results in the third reference template region based on the association type in method / procedure 300. Each third reference template region may be determined by a corresponding block vector candidate from the block vector candidates. Then, decoder module 124 may predict the second block template region based on each flipped reconstruction result in the third reference template region, using the corresponding filter model derived by method / procedure 300, to generate one of a plurality of third prediction results for the second block template region. In other words, each of the third block vector-based candidates can be used to determine one of the third prediction results for the second block template region. In still other embodiments, decoder module 124 may directly set the flipped reconstruction results in the third reference template region as a plurality of fourth prediction results for the second block template region. In other words, each of the fourth block vector-based candidates can be used to determine one of the fourth prediction results for the second block template region.

[0161] When decoder module 124 determines the first and second prediction results of a block unit, it can determine whether a spatial sample item is included in the second reference template region. Therefore, when excluding a specific spatial sample item from a specific second reference template region, the excluded item can be replaced by copying samples adjacent to the excluded spatial sample item included in the specific second reference template region. Furthermore, when decoder module 124 determines the third and fourth prediction results of a block unit, it can determine whether a spatial sample item is included in the third reference template region. Therefore, when excluding a specific spatial sample item from a specific third reference template region, the excluded item can be replaced by copying samples adjacent to the excluded spatial sample item included in the specific third reference template region.

[0162] Decoder module 114 can determine the template matching cost of block vector-based prediction candidates by comparing multiple prediction results in the second template region with the reconstructed results in the second template region using a cost function. The prediction results may include at least one of a first prediction result, a second prediction result, a third prediction result, and a fourth prediction result. The cost function may include, but is not limited to, Sum of Absolute Differences (SAD), Sum of Absolute Transform Differences (SATD), Mean Absolute Difference (MAD), Mean Squared Error (MSD), and Structural Similarity (SSIM). It should be noted that any cost function may be used without departing from this disclosure.

[0163] In some specific implementations, when the block vector-based prediction candidates include first block vector-based candidates, the template matching cost may include a first template cost. Each first template cost in the first template cost may be calculated based on the first block vector-based candidate corresponding to the corresponding block vector candidate and the corresponding filter model. Additionally, each first template cost in the first template cost may be calculated using the corresponding filter model based on a corresponding second reference template region in the second block template region and the second reference template region. For example, when the block vector-based prediction candidates only include first block vector-based candidates, each template matching cost in the template matching cost may be calculated using a corresponding filter model in the filter model of the block unit.

[0164] In some specific implementations, when the block vector-based prediction candidate includes a second block vector-based candidate, the template matching cost may include multiple second template costs. Each second template cost can be directly computed based on the corresponding block vector candidate without using the block unit filter model. Alternatively, each second template cost can also be computed based on a corresponding second reference template region in the second block template region and the second reference template region without using the block unit filter model.

[0165] In some other implementations, when the block vector-based prediction candidate includes a third block vector-based candidate, the template matching cost may include multiple third template costs. Each of the third template costs may be computed based on the third block vector-based candidate corresponding to the corresponding block vector candidate and the corresponding filter model generated in method / procedure 300. Alternatively, each of the third template costs may be computed based on a corresponding third reference template region in the second block template region and the third reference template region using the corresponding filter model.

[0166] In some other specific implementations, when the block vector-based prediction candidate includes a fourth block vector-based candidate, the template matching cost may include multiple fourth template costs. Each of the fourth template costs can be computed directly based on the corresponding block vector candidate without using the block unit filter model. Alternatively, each of the fourth template costs can be computed based on a corresponding third reference template region in the second block template region and the third reference template region without using the block unit filter model.

[0167] Referring again to Figure 7, at box 760, decoder module 124 can determine the arrangement of filter models based on template matching cost.

[0168] Referring to Figures 1 and 2, the decoder module 124 can determine the arrangement of block vector-based prediction candidates based on template matching cost, and reorder the block vector-based prediction candidates based on this arrangement. In some implementations, the block vector-based prediction candidates can be reordered in ascending or descending order of template matching cost. Additionally, this arrangement can be used to adjust the original order of the block vector-based prediction candidates included in the block vector-based candidate list to generate an adjusted candidate list containing at least one of the block vector-based prediction candidates.

[0169] Before determining the placement based on template matching cost, the block vector-based prediction candidates can be sorted according to any rule. Then, the block vector-based prediction candidates can be reordered in ascending order of template matching cost. Therefore, when the template matching cost of a particular block vector-based prediction candidate is less than the template matching costs of other block vector-based prediction candidates, that particular block vector-based prediction candidate can be moved forward based on the placement, making it the first block vector-based prediction candidate. In other words, when the template matching cost of a particular block vector-based prediction candidate is the minimum among the template matching costs of all block vector-based prediction candidates, that particular block vector-based prediction candidate can be moved, making it the first block vector-based prediction candidate.

[0170] Decoder module 124 selects K block-vector-based prediction candidates with the minimum template matching cost from the block-vector-based prediction candidates and adds the selected K block-vector-based prediction candidates to an adjusted candidate list. The number K is a positive integer equal to or greater than 1 and can be equal to the number of block-vector-based prediction candidates in the adjusted candidate list. The number K can be equal to or less than the total number of block-vector-based prediction candidates. In other words, when the block-vector-based prediction candidates are reordered in ascending order of template matching cost to generate an arrangement, decoder module 124 can select from the first to the Kth block-vector-based prediction candidates based on the arrangement sorting.

[0171] In some specific implementations, the block vector-based prediction candidates may include at least one of a first block vector-based candidate, a second block vector-based candidate, a third block vector-based candidate, or a fourth block vector-based candidate. Therefore, the template matching cost may include at least one of a first template cost, a second template cost, a third template cost, or a fourth template cost. In other words, the arrangement of the block vector-based prediction candidates can be determined based on at least one of the first template cost, the second template cost, the third template cost, or the fourth template cost.

[0172] When the block vector-based prediction candidates include a first block vector-based candidate, the arrangement may include a first arrangement of the filter model determined based on a first template cost. Therefore, the block vector-based prediction candidates in the block vector-based candidate list may be ordered based on the first arrangement of the filter model. In some embodiments, the block vector-based prediction candidates may include only the first block vector-based candidate. Therefore, the arrangement may include only the first arrangement of the filter model determined based on the first template cost. The block vector-based prediction candidates in the block vector-based candidate list may also be ordered based only on the first arrangement of the filter model. In some other embodiments, the block vector-based prediction candidates may include a first block vector-based candidate and at least one of a second to a fourth block vector-based candidate. The arrangement may include a first arrangement of the filter model and at least one of a second to a fourth arrangement. Therefore, the block vector-based prediction candidates may be ordered based on at least one of the first and second to fourth arrangements of the filter model.

[0173] When the block vector-based prediction candidates include a first block vector-based candidate and a second block vector-based candidate, the arrangement may include a second arrangement of the block vector-based prediction candidates determined based on the first template cost and the second template cost. Therefore, the block vector-based prediction candidates in the block vector-based candidate list can be sorted according to the second arrangement of the block vector-based prediction candidates.

[0174] Referring again to Figure 7, at box 770, the decoder module 124 can reconstruct block cells based on the arrangement of the filter model.

[0175] Referring to Figures 1 and 2, decoder module 124 can reconstruct block cells according to block vector-based prediction candidates ordered in a block vector-based candidate list. In some embodiments, when the block vector-based prediction candidates include a first block vector-based candidate, decoder module 124 can reconstruct block cells according to the block vector-based prediction candidates ordered based on a first arrangement of the filter model. In some other embodiments, when the block vector-based prediction candidates include a first block vector-based candidate and a second block vector-based candidate, decoder module 124 can reconstruct block cells according to the block vector-based prediction candidates ordered according to a second arrangement of the block vector-based prediction candidates.

[0176] In some implementations, each block vector-based prediction candidate in the adjusted candidate list may have an index value. Therefore, since the number of block vector-based prediction candidates in the adjusted candidate list is equal to K, the index values ​​of the adjusted candidate list can be in the range of 0 to K-1. Since the index value of a prediction index may not be greater than K-1, block vector-based prediction candidates that are arranged after the Kth block vector-based prediction candidate can be excluded from the adjusted candidate list.

[0177] Decoder module 124 can select a block vector-based prediction candidate from an adjusted candidate list based on a prediction index, where the prediction index is arranged in a specific order. The prediction index of a block unit can indicate the index value of the selected block vector-based prediction candidate from the block vector-based prediction candidates in the adjusted candidate list. The prediction index can be included in the video data, allowing the decoder module to parse the video data to determine the prediction index.

[0178] In some other implementations, the first block-vector-based prediction candidate among the block-vector-based prediction candidates ordered by the arrangement in the adjusted candidate list may have the minimum template matching cost of the block-vector-based prediction candidates. Therefore, the decoder module 124 may directly select the first block-vector-based prediction candidate among the block-vector-based prediction candidates in the adjusted candidate list to predict block units without resolving the prediction index from the video data.

[0179] After determining the selected block vector-based prediction candidate, decoder module 124 can use the selected block vector-based prediction candidate to predict block units to determine the predicted block of the block unit. Decoder module 124 can also add multiple residual components to the predicted block to reconstruct the block unit. The residual components can be determined from the bitstream. Decoder module 124 can reconstruct all other block units in the current frame to reconstruct the current frame and video data.

[0180] In some implementations, a filter flag indicating whether to derive a filter model can be determined from video data for determining a first arrangement of the filter model based on a first template matching cost. When the filter flag is equal to 0, the decoder module 124 can ignore the derivation of the filter model. Therefore, a first block vector-based candidate can be excluded from the block vector-based prediction candidates. Alternatively, when the filter flag is equal to 1, the decoder module 124 can derive the filter model and calculate the first template matching cost for determining the first arrangement of the filter model.

[0181] Then, method / procedure 700 can be completed.

[0182] Figure 9 is a block diagram illustrating an encoder module 114 of the first electronic device 110 illustrated in Figure 1, specifically implemented according to one or more examples of the present disclosure. The encoder module 114 may include a prediction processor (e.g., prediction processing unit 9141), at least a first adder (e.g., first adder 9142) and a second adder (e.g., second adder 9145), a transform / quantization processor (e.g., transform / quantization unit 9143), an inverse quantization / inverse transform processor (e.g., inverse quantization / inverse transform unit 9144), a filter (e.g., filter unit 9146), a decoded image buffer (e.g., decoded image buffer 9147), and an entropy encoder (e.g., entropy coding unit 9148). The prediction processing unit 9141 of the encoder module 114 may also include a segmentation processor (e.g., segmentation unit 91411), an intra-frame prediction processor (e.g., intra-frame prediction unit 91412), and an inter-frame prediction processor (e.g., inter-frame prediction unit 91413). The encoder module 114 can receive source video and encode the source video to output a bitstream.

[0183] The encoder module 114 can receive a source video comprising multiple image frames, and then divide the image frames according to the encoding structure. Each image frame in the image frame can be divided into at least one image block.

[0184] The at least one image block may include a luminance block having multiple luminance samples and at least one chrominance block having multiple chrominance samples. The luminance block and the at least one chrominance block may be further subdivided to generate macroblocks, CTUs, CBs, their sub-partitions, and / or other equivalent coding units.

[0185] Encoder module 114 can perform additional sub-segments of the source video. It should be noted that the specific implementations disclosed are generally applicable to video encoding, regardless of how the source video is segmented before and / or during encoding.

[0186] During the encoding process, the prediction processing unit 9141 may receive the current image block of a specific image frame. The current image block may be a luma block or a chroma block in the chroma blocks of the specific image frame.

[0187] The segmentation unit 91411 can divide the current image block into multiple block units. The intra-frame prediction unit 91412 can perform intra-frame prediction coding of the current block unit relative to one or more adjacent blocks in the same frame as the current block unit to provide spatial prediction. The inter-frame prediction unit 91413 can perform inter-frame prediction coding of the current block unit relative to one or more blocks in one or more reference image blocks to provide temporal prediction.

[0188] The prediction processing unit 9141 can select one of the coding results generated by the intra-prediction unit 91412 and the inter-prediction unit 91413 based on a mode selection method (such as a cost function). The mode selection method can be a rate-distortion optimization (RDO) process.

[0189] The prediction processing unit 9141 can determine the selected coding result and provide the prediction block corresponding to the selected coding result to the first adder 9142 to generate a residual block, and provide it to the second adder 9145 to reconstruct the coded block unit. The prediction processing unit 9141 can also provide syntax elements, such as motion vectors, intra-frame mode indicators, segmentation information and / or other syntax information, to the entropy coding unit 9148.

[0190] Intra-prediction unit 91412 can perform intra-prediction on the current block unit. Intra-prediction unit 91412 can determine the intra-prediction mode for the reconstructed samples adjacent to the current block unit in order to encode the current block unit.

[0191] Intra-prediction unit 91412 can use various intra-prediction modes to encode the current block unit. Intra-prediction unit 91412 of prediction processing unit 9141 can select an appropriate intra-prediction mode from the selected modes. Intra-prediction unit 91412 can use a cross-component prediction mode to encode the current block unit to predict one of the two chrominance components of the current block unit based on the luma component of the current block unit. Intra-prediction unit 91412 can predict the first chrominance component of the two chrominance components of the current block unit based on the second chrominance component of the two chrominance components of the current block unit.

[0192] Inter-frame prediction unit 91413 can perform inter-frame prediction on the current block unit as an alternative to intra-frame prediction performed by intra-frame prediction unit 91412. Inter-frame prediction unit 91413 can perform motion estimation to estimate the motion of the current block unit to generate motion vectors.

[0193] The motion vector indicates the displacement of the current block cell within the current image block relative to the reference block cell within the reference image block. The inter-frame prediction unit 91413 may receive at least one reference image block stored in the decoded image buffer 9147 and estimate motion based on the received reference image block to generate a motion vector.

[0194] The first adder 9142 generates a residual block by subtracting the prediction block determined by the prediction processing unit 9141 from the original current block unit. The first adder 9142 may represent one or more components performing the subtraction.

[0195] The transform / quantization unit 9143 can apply a transform to the residual block to generate residual transform coefficients, and then quantize these residual transform coefficients to further reduce the bit rate. The transform can be one of DCT, DST, AMT, MDNSST, HyGT, signal-dependent transform, KLT, wavelet transform, integer transform, subband transform, and conceptually similar transforms.

[0196] Transformation can convert residual information from the pixel value domain to a transform domain such as the frequency domain. The degree of quantization can be modified by adjusting the quantization parameters.

[0197] The transform / quantization unit 9143 can perform a scan of a matrix including quantized transform coefficients. Alternatively, the entropy encoding unit 9148 can perform the scan.

[0198] The entropy coding unit 9148 can receive multiple syntax elements from the prediction processing unit 9141 and the transform / quantization unit 9143, including quantization parameters, transform data, motion vectors, intra-frame modes, segmentation information, and / or other syntax information. The entropy coding unit 9148 can encode the syntax elements into a bitstream.

[0199] Entropy coding unit 9148 can entropy code the quantized transform coefficients by performing CAVLC, CABAC, SBAC, PIPE coding, or another entropy coding technique to generate an encoded bitstream. The encoded bitstream can be sent to another device (e.g., a second electronic device 120, as shown in FIG1) or archived for later transmission or retrieval.

[0200] The inverse quantization / inverse transform unit 9144 can apply inverse quantization and inverse transform to reconstruct the residual block in the pixel domain for later use as a reference block. The second adder 9145 can add the reconstructed residual block to the prediction block provided by the prediction processing unit 9141 to produce a reconstructed block for storage in the decoded image buffer 9147.

[0201] Filtering unit 9146 may include a deblocking filter, a SAO filter, a bilateral filter, and / or an ALF to remove block artifacts from the reconstructed block. In addition to the deblocking filter, SAO filter, bilateral filter, and ALF, other filters (in-loop or post-loop) may also be used. For simplicity, such filters are not illustrated; these filters may filter the output of the second adder 9145.

[0202] The decoded image buffer 9147 may be a reference image memory that stores reference blocks to be used by the encoder module 914 for encoding video (such as in intra-frame coding or inter-frame coding modes). The decoded image buffer 9147 may include various memory devices, such as DRAM (e.g., including SDRAM), MRAM, RRAM, or other types of memory devices. The decoded image buffer 9147 may be on-chip along with other components of the encoder module 114, or it may be off-chip relative to these components.

[0203] The method / process 300 for decoding and / or encoding video data may be performed by a first electronic device 110. Referring to Figures 1 and 9, at block 310, the method / process 300 may begin by receiving video data via encoder module 114. The video data received by encoder module 114 may be video. At block 320, encoder module 114 may determine block units from the current frame included in the video data. In some embodiments, encoder module 114 may divide the current frame according to a segmentation instruction (e.g., based on any video coding standard) to generate multiple CTUs, and further divide the current CTUs included in the CTUs to generate multiple partition blocks, and determine block units from the partition blocks. In some other embodiments, encoder module 114 may divide the current frame to generate multiple slices, and further divide the current slices included in the slices to generate multiple CTUs. Additionally, encoder module 114 may further divide the current CTUs included in the CTUs according to a segmentation instruction to generate multiple partition blocks, and determine block units from the partition blocks.

[0204] At frame 330, encoder module 114 can determine the reference block indicated by the block vector from the current frame. Referring to Figures 1, 3, and 9, the determination process of decoder module 124 at frame 330 can also be performed by encoder module 114 at frame 330. Encoder module 114 can derive the block vector in Reconstructed Reordering (RR) Intra-Frame Block Copying (IBC) (RR-IBC) mode. Therefore, RR-IBC mode can be used to determine the reference block indicated by the block vector.

[0205] The adjacency candidate vectors determined by encoder module 114 can be the same as those determined by decoder module 114. Additionally, the flip direction determined by encoder module 114 can also be the same as that determined by decoder module. In some embodiments, when the vector direction of a specific adjacency candidate vector is the same as a specific flip direction, the specific adjacency candidate vector can be determined as a reference candidate vector in the vector candidate list for deriving the block vector. Furthermore, the association type corresponding to the specific adjacency candidate vector can specify the specific flip direction. In some other embodiments, the association type of the block cell corresponding to a specific adjacency candidate vector of a specific adjacent block can be inherited from the specific adjacent block. Therefore, when the vector direction of a specific adjacency candidate vector of a specific adjacent block is the same as the flip direction specified by the association type of the specific adjacent block, the specific adjacency candidate vector can be determined as a reference candidate vector in the vector candidate list for deriving the block vector.

[0206] In some implementations, when the number of reference candidate vectors is equal to one, a reference candidate vector can be determined as the reference vector used to derive a block vector to indicate a reference block. In other implementations, when the number of reference candidate vectors is greater than one, multiple reference candidate vectors can be determined, each reference vector used to derive a corresponding block vector in the block vectors to indicate a corresponding reference block in the reference blocks. The method for determining the block vector based on the reference vectors performed by the encoder module 114 can be the same as the method performed by the decoder module 124. Furthermore, the method for determining the reference block based on the block vectors performed by the encoder module 114 can be the same as the method performed by the decoder module 124. Therefore, the number of reference blocks can be equal to the number of reference candidate vectors in the vector candidate list.

[0207] At box 340, encoder module 114 can determine the correspondence between the block template region adjacent to the block unit and the reference template region adjacent to the reference block based on the association type. Referring to Figures 1, 3, and 9, the determination process of decoder module 124 at box 340 can also be performed by encoder module 114 at box 340.

[0208] The block template region determined by encoder module 114 can be the same as the block template region determined by decoder module 114. Furthermore, the method for determining the reference template region performed by encoder module 114 can be the same as the method for determining the reference template region performed by decoder module 124. Therefore, the determination method of decoder module 124 can be used to determine each reference template region adjacent to a corresponding reference block in the reference block. In other words, the number of reference template regions can be equal to the number of reference blocks.

[0209] The correspondence between block template regions and reference template regions can be determined based on the association type. Furthermore, the method for determining each correspondence between a block template region and its corresponding reference template region, performed by encoder module 114, can be the same as the method for determining the correspondence between a block template region and a reference template region, performed by decoder module 124. Therefore, the number of correspondences can be equal to the number of reference blocks and also equal to the number of reference template regions.

[0210] At box 350, encoder module 124 can derive the prediction model of the block cell based on the correspondence between the block template region and the reference template region. Referring to Figures 1, 3, and 9, the deriving process of decoder module 124 at box 350 can also be performed by encoder module 114 at box 350.

[0211] The method performed by encoder module 114 for deriving each predictive model from a predictive model based on a correspondence in the correspondence relation can be the same as the method performed by decoder module 124 for deriving predictive models based on correspondence relations. In some embodiments, the number of predictive models can be equal to the number of correspondence relations and also equal to the number of reference blocks. In some other embodiments, when encoder module 114 uses multiple candidate models to derive multiple predictive models, each correspondence can be used to derive more than one predictive model. For example, when the number of correspondence relations is equal to Ncr and the number of candidate models is equal to Ncm, the number of predictive models Npm can be equal to or less than Ncr × Ncm.

[0212] In some specific implementations, to facilitate the derivation of the prediction model, the encoder module 114 can pre-flip the block template region along the centerline of the block unit 400. For example, as shown in FIG6C, the encoder module 114 can pre-flip the block template region along the vertical centerline 4015 of the block unit 400. Therefore, the flipped first adjacent region 403' can be located to the right of the flipped block unit 400'. In addition, as shown in FIG6A and FIG6C, multiple flipped block template samples in the flipped block template region 403' can be symmetrical with multiple original block template samples in the block template region 403 along the vertical centerline 4015 of the block unit 400. Multiple flipped block template samples in the flipped second block template region 404' can also be symmetrical with multiple original block template samples in the second template region 404 along the vertical centerline 4015 of the reference block 410. Therefore, the positional relationship between the flipped block unit 400' and the flipped block template region can be the same as the positional relationship between the reference unit 410 and the reference template region, so as to derive the corresponding prediction model. In some other specific implementations, as shown in Figure 4B, the positional relationship between the flipped block unit and the flipped block template area generated by flipping the block unit 400 and the block template area along the horizontal center line of the block unit 400 can be the same as the positional relationship between the reference block 420 and the reference template area.

[0213] Referring again to Figure 3, at box 360, encoder module 124 can reconstruct the block cell using a prediction model with an association type based on the reference block. Referring to Figures 1, 3, and 9, the reconstruction process of decoder module 124 at box 360 can also be performed by encoder module 114 at box 360.

[0214] Encoder module 114 can predict block units based on a corresponding prediction model in the prediction model for each reference block in the reference block, to generate a corresponding prediction block among multiple prediction blocks. Alternatively, encoder module 114 can predict block units based on other prediction models to generate multiple prediction blocks. Encoder module 114 can select one of these prediction blocks based on a mode selection method (such as a cost function). Mode selection methods can include RDO processing, Sum of Absolute Differences (SAD) processing, Sum of Absolute Transform Differences (SATD) processing, Mean Absolute Difference (MAD) processing, Mean Squared Difference (MSD) processing, and Structural Similarity (SSIM) processing. Encoder module 114 can provide the selected encoding result to first adder 9142 to generate a residual block and to second adder 9145 to reconstruct the coded block unit. The reconstruction of block units by encoder module 114 can be the same as the reconstruction of block units by decoder module 124.

[0215] In some implementations, when selecting a prediction block corresponding to a prediction model in the prediction model to predict and reconstruct block units, encoder module 114 may also provide syntax elements (such as candidate indices) included in the bitstream for transmission to decoder module 124. In some implementations, the candidate index of a block unit can be used to determine a reference candidate vector among reference candidate vectors corresponding to a prediction model selected in the prediction model.

[0216] In some specific implementations, the block vector-based local illumination compensation (LIC) flag, RR-IBC flag, and template type syntax of the block cells received by the decoder module 124 in method / process 300 may be provided by the encoder module 114.

[0217] Encoder module 114 can predict and reconstruct all other block units in an image frame for use in reconstructing the image frame and video. Then, the method / process 300 for encoder module 114 can end.

[0218] The method / process 700 for decoding and / or encoding video data may be performed by a first electronic device 110. Referring to Figures 1 and 9, at block 710, the method / process 700 may begin by receiving video data via encoder module 114. The video data received by encoder module 114 may be video. At block 720, encoder module 114 may determine block units from the current frame included in the video data. In some embodiments, encoder module 114 may divide the current frame according to a segmentation instruction (e.g., based on any video coding standard) to generate multiple CTUs, and further divide the current CTUs included in the CTUs to generate multiple partition blocks, and determine block units from the partition blocks. In some other embodiments, encoder module 114 may divide the current frame to generate multiple slices, and further divide the current slices included in the slices to generate multiple CTUs. Additionally, encoder module 114 may further divide the current CTUs included in the CTUs according to a segmentation instruction to generate multiple partition blocks, and determine block units from the partition blocks.

[0219] At block 730, encoder module 114 can determine from the current frame multiple vector reference blocks of a block unit indicated by multiple block vector candidates of the block unit. In some specific implementations, each vector reference block can be indicated by a corresponding block vector candidate from the block vector candidates of the block unit. Referring to Figures 1, 3, and 9, the determination process of decoder module 124 at block 730 can also be performed by encoder module 114 at block 730.

[0220] At block 740, encoder module 114 can derive multiple filter models of block units based on vector reference blocks. In some specific implementations, each filter model in the filter model can be derived based on a corresponding vector reference block in the vector reference blocks. Referring to Figures 1, 3, and 9, the derivation process of decoder module 124 at block 740 can also be performed by encoder module 114 at block 740.

[0221] At block 750, encoder module 114 may determine multiple template matching costs computed using the filter model of the block unit. In some specific implementations, the template matching cost may include multiple first template costs, and each of the first template costs may be computed using a corresponding filter model in the filter model of the block unit. Referring to Figures 1, 3, and 9, the determination process of decoder module 124 at block 750 may also be performed by encoder module 114 at block 750.

[0222] At block 760, encoder module 114 can determine the arrangement of filter models based on template matching cost. Referring to Figures 1, 3, and 9, the determination process of decoder module 124 at block 760 can also be performed by encoder module 114 at block 760.

[0223] At block 770, encoder module 114 can reconstruct block cells based on the arrangement of the filter model. Referring to Figures 1, 3, and 9, the reconstruction process of decoder module 124 at block 770 can also be performed by encoder module 114 at block 770.

[0224] Encoder module 114 can predict block units based on each reference block in the reference block using a corresponding filter model from the filter models, to generate a corresponding prediction block from a plurality of prediction blocks. Alternatively, encoder module 114 can predict block units based on other prediction modes to generate multiple prediction blocks. Encoder module 114 can select one of these prediction blocks based on a mode selection method (such as a cost function). The mode selection method can be RDO processing, SAD processing, SATD processing, MAD processing, MSD processing, and SSIM processing. Encoder module 114 can provide the selected encoding result to first adder 9142 to generate a residual block, and to second adder 9145 to reconstruct the coded block unit. The reconstruction of block units by encoder module 114 can be the same as the reconstruction of block units by decoder module 124.

[0225] In some implementations, when selecting a prediction block corresponding to a filter model in a filter model to predict and reconstruct a block unit, encoder module 114 may also provide syntax elements (such as prediction indices) included in the bitstream for transmission to decoder module 124. Each block-vector-based prediction candidate in the adjusted candidate list may have an index value. Therefore, since the number of block-vector-based prediction candidates in the adjusted candidate list is equal to K, the index values ​​of the adjusted candidate list may be in the index range of 0 to K-1. Since the index value of the prediction index may not be greater than K-1, block-vector-based prediction candidates arranged after the Kth block-vector-based prediction candidate may be excluded from the adjusted candidate list. In some implementations, the prediction index of a block unit may be equal to one of the index values ​​used to determine the block-vector-based prediction candidate corresponding to a selected prediction block in the prediction block.

[0226] In some other implementations, when generating a selected prediction block based on the first block-vector-based prediction candidate from the adjusted candidate list, the encoder module 114 may not provide a prediction index to the decoder module 124. In other words, the decoder module 124 may directly select the first block-vector-based prediction candidate from the adjusted candidate list to predict the block unit without resolving the prediction index from the video data.

[0227] In some implementations, a filter flag may be encoded into the video data to indicate whether a filter model is derived for determining a first arrangement of the filter model based on a first template matching cost. When the encoder module 114 ignores the derivation of the filter model, the filter flag provided by the encoder module 114 may be equal to 0. Therefore, a first block vector-based candidate can be excluded from the block vector-based prediction candidates. Conversely, when the encoder module 114 derives the filter model and calculates the first template matching cost to determine the first arrangement of the filter model, the filter flag provided by the encoder module 114 may be equal to 1.

[0228] Encoder module 114 can predict and reconstruct all other block units in an image frame for use in reconstructing the image frame and video. Then, the method / process 700 for encoder module 114 can end.

[0229] The specific implementations disclosed are to be considered illustrative rather than restrictive in all respects. It should also be understood that this disclosure is not limited to the specific implementations disclosed; on the contrary, many rearrangements, modifications, and substitutions are possible without departing from the scope of this disclosure.

Claims

1. An electronic device for decoding video data, the electronic device comprising: At least one processor; and one or more non-transitory computer-readable media, said one or more non-transitory computer-readable media being coupled to said at least one processor and storing one or more computer-executable instructions, said one or more computer-executable instructions causing the electronic device, when executed by said at least one processor, to: receive the video data; Determine a block unit from the current frame included in the video data; determine a plurality of vector reference blocks for the block unit from the current frame, each of the plurality of vector reference blocks being indicated by a corresponding block vector candidate from a plurality of block vector candidates for the block unit; The block cell is derived from multiple filter models, each of which is derived based on a corresponding vector reference block among the multiple vector reference blocks; multiple first template matching costs are determined, each of which is calculated using a corresponding filter model among the multiple filter models of the block cell; a first arrangement of the multiple filter models is determined based on the multiple first template matching costs; and the block cell is reconstructed based on the first arrangement of the multiple filter models.

2. The electronic device of claim 1, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a plurality of adjacent blocks adjacent to the block unit; determine a plurality of adjacent block vectors, each of the plurality of adjacent block vectors indicating a corresponding adjacent reference block among a plurality of adjacent reference blocks in the current frame for reconstructing the corresponding adjacent block among the plurality of adjacent blocks; and determine the plurality of block vector candidates based on the plurality of adjacent block vectors.

3. The electronic device of claim 1, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a first block template region and a plurality of first reference template regions adjacent to the block unit, each of the plurality of first reference template regions being adjacent to a corresponding vector reference block among the plurality of vector reference blocks; and determine a second block template region and a plurality of second reference template regions adjacent to the block unit, each of the plurality of second reference template regions being adjacent to a corresponding vector reference block among the plurality of vector reference blocks, wherein: Each of the plurality of filter models is further derived based on a corresponding first reference template region among the first template region and the plurality of first reference template regions, and each of the plurality of first template matching costs is further calculated using the corresponding filter model among the plurality of filter models based on a corresponding second reference template region among the second template region and the plurality of second reference template regions.

4. The electronic device according to claim 3, wherein: The first template area is the same as the second template, and each of the plurality of first reference template areas is the same as a corresponding second template area among the plurality of second template areas.

5. The electronic device of claim 1, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a block vector-based candidate list comprising a plurality of block vector-based prediction candidates, wherein: Each of the block vector-based prediction candidates corresponds to a block vector candidate among the plurality of block vector candidates and a filter model among the plurality of filter models. The plurality of block vector-based prediction candidates in the block vector-based candidate list are ordered based on the first arrangement of the plurality of filter models, and the reconstruction of the block unit is also based on the plurality of block vector-based prediction candidates ordered in the block vector-based candidate list.

6. The electronic device of claim 1, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a plurality of second template matching costs, each of the plurality of second template matching costs being computed directly based on a corresponding block vector candidate among the plurality of block vector candidates, without using the plurality of filter models of the block unit; and determine a second arrangement of a plurality of block vector-based prediction candidates based on the plurality of first template matching costs and the plurality of second template matching costs, wherein reconstructing the block unit is also based on the second arrangement of the plurality of block vector-based prediction candidates.

7. The electronic device according to claim 6, wherein: The plurality of block vector-based prediction candidates includes at least one of a plurality of first block vector-based candidates or a plurality of second block vector-based candidates. Each of the plurality of first block vector-based candidates corresponds to a corresponding block vector candidate among the plurality of block vector candidates and a corresponding filter model among the plurality of filter models. Each of the plurality of second block vector-based candidates corresponds only to a corresponding block vector candidate among the plurality of block vector candidates, without using the plurality of filter models.

8. The electronic device of claim 1, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine, based on the plurality of first template matching costs, a filter flag indicating whether to derive the plurality of filter models for determining the first arrangement of the plurality of filter models.

9. A non-transitory machine-readable medium for an electronic device, the non-transitory machine-readable medium storing one or more computer-executable instructions for decoding video data, the one or more computer-executable instructions causing the electronic device, when executed by at least one processor of the electronic device, to: receive the video data; Determine a block unit from the current frame included in the video data; determine a plurality of vector reference blocks for the block unit from the current frame, each of the plurality of vector reference blocks being indicated by a corresponding block vector candidate from a plurality of block vector candidates for the block unit; The block cell is derived from multiple filter models, each of which is derived based on a corresponding vector reference block among the multiple vector reference blocks; multiple first template matching costs are determined, each of which is calculated using a corresponding filter model among the multiple filter models of the block cell; a first arrangement of the multiple filter models is determined based on the multiple first template matching costs; and the block cell is reconstructed based on the first arrangement of the multiple filter models.

10. The non-transitory machine-readable medium of claim 9, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a first block template region and a plurality of first reference template regions adjacent to the block unit, each of the plurality of first reference template regions being adjacent to a corresponding vector reference block of the plurality of vector reference blocks; and determine a second template region and a plurality of second reference template regions adjacent to the block unit, each of the plurality of second reference template regions being adjacent to a corresponding vector reference block of the plurality of vector reference blocks, wherein: Each of the plurality of filter models is further derived based on a corresponding first reference template region among the first template region and the plurality of first reference template regions, and each of the plurality of first template matching costs is further calculated using the corresponding filter model among the plurality of filter models based on a corresponding second reference template region among the second template region and the plurality of second reference template regions.

11. The non-transitory machine-readable medium according to claim 10, wherein: The first template area is the same as the second template area, and each of the plurality of first reference template areas is the same as a corresponding second template area among the plurality of second template areas.

12. The non-transitory machine-readable medium of claim 9, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a block-vector-based candidate list comprising a plurality of block-vector-based prediction candidates, wherein: Each of the block vector-based prediction candidates corresponds to a block vector candidate among the plurality of block vector candidates and a filter model among the plurality of filter models. The plurality of block vector-based prediction candidates in the block vector-based candidate list are ordered based on the first arrangement of the plurality of filter models, and the reconstruction of the block unit is also based on the plurality of block vector-based prediction candidates ordered in the block vector-based candidate list.

13. The non-transitory machine-readable medium of claim 9, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a plurality of second template matching costs, each of the plurality of second template matching costs being computed directly based on a corresponding block vector candidate among the plurality of block vector candidates, without using the plurality of filter models of the block units; and determine a second arrangement of a plurality of block vector-based prediction candidates based on the plurality of first template matching costs and the plurality of second template matching costs, wherein: The reconstruction of the block unit is also based on the second arrangement of the plurality of block vector-based prediction candidates, the plurality of block vector-based prediction candidates including at least one of a plurality of first block vector-based candidates or a plurality of second block vector-based candidates, each of the plurality of first block vector-based candidates corresponding to a corresponding block vector candidate among the plurality of block vector candidates and a corresponding filter model among the plurality of filter models, and each of the plurality of second block vector-based candidates corresponding to only one of the plurality of block vector candidates, without using the plurality of filter models.

14. The non-transitory machine-readable medium of claim 9, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine, based on the plurality of first template matching costs, a filter flag indicating whether to derive the plurality of filter models for determining the first arrangement of the plurality of filter models.

15. An electronic device for encoding video data, the electronic device comprising: At least one processor; and one or more non-transitory computer-readable media, said one or more non-transitory computer-readable media being coupled to said at least one processor and storing one or more computer-executable instructions, said one or more computer-executable instructions causing the electronic device, when executed by said at least one processor, to: receive the video data; Determine a block unit from the current frame included in the video data; determine a plurality of vector reference blocks for the block unit from the current frame, each of the plurality of vector reference blocks being indicated by a corresponding block vector candidate from a plurality of block vector candidates for the block unit; The block cell is derived from multiple filter models, each of which is derived based on a corresponding vector reference block among the multiple vector reference blocks; multiple first template matching costs are determined, each of which is calculated using a corresponding filter model among the multiple filter models of the block cell; a first arrangement of the multiple filter models is determined based on the multiple first template matching costs; and the block cell is reconstructed based on the first arrangement of the multiple filter models.

16. The electronic device of claim 15, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a first block template region and a plurality of first reference template regions adjacent to the block unit, each of the plurality of first reference template regions being adjacent to a corresponding vector reference block among the plurality of vector reference blocks; and determine a second template region and a plurality of second reference template regions adjacent to the block unit, each of the plurality of second reference template regions being adjacent to a corresponding vector reference block among the plurality of vector reference blocks, wherein: Each of the plurality of filter models is further derived based on a corresponding first reference template region among the first template region and the plurality of first reference template regions, and each of the plurality of first template matching costs is further calculated using the corresponding filter model among the plurality of filter models based on a corresponding second reference template region among the second template region and the plurality of second reference template regions.

17. The electronic device according to claim 16, wherein: The first template area is the same as the second template area, and each of the plurality of first reference template areas is the same as a corresponding second template area among the plurality of second template areas.

18. The electronic device of claim 15, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a block vector-based candidate list comprising a plurality of block vector-based prediction candidates, wherein: Each of the block vector-based prediction candidates corresponds to a block vector candidate among the plurality of block vector candidates and a filter model among the plurality of filter models. The plurality of block vector-based prediction candidates in the block vector-based candidate list are ordered based on the first arrangement of the plurality of filter models, and the reconstruction of the block unit is also based on the plurality of block vector-based prediction candidates ordered in the block vector-based candidate list.

19. The electronic device of claim 15, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a plurality of second template matching costs, each of the plurality of second template matching costs being computed directly based on a corresponding block vector candidate among the plurality of block vector candidates, without using the plurality of filter models of the block units; and determine a second arrangement of a plurality of block vector-based prediction candidates based on the plurality of first template matching costs and the plurality of second template matching costs, wherein: The reconstruction of the block unit is also based on the second arrangement of the plurality of block vector-based prediction candidates, the plurality of block vector-based prediction candidates including at least one of a plurality of first block vector-based candidates or a plurality of second block vector-based candidates, each of the plurality of first block vector-based candidates corresponding to a corresponding block vector candidate among the plurality of block vector candidates and a corresponding filter model among the plurality of filter models, and each of the plurality of second block vector-based candidates corresponding to only one of the plurality of block vector candidates, without using the plurality of filter models.

20. The electronic device of claim 15, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine, based on the plurality of first template matching costs, a filter flag indicating whether to derive the plurality of filter models for determining the first arrangement of the plurality of filter models.

Images